David A. Wheeler's Blog

Open Source Computer Emergency Response Team (oCERT)

Thu, 08 May 2008 11:01 GMT

Here's something new and interesting: the Open Source Computer Emergency Response Team (oCERT). Here's how they describe themselves: "The oCERT project is a public effort providing security handling support to Open Source projects affected by security incidents or vulnerabilities...".

They promise to keep things moving. They do permit embargo periods (where vulnerabilities are not publicly disclosing, giving time for developers to fix the problem first). More importantly, though, they have a maximum embargo time of two months; I think that's great, and important, because a lot of suppliers have abused embargo periods and failed to fix critical vulnerabilities as long as they're embargoed. These abuses often resulted in customers being exploited through mechanisms that the supplier knew about, but refused to fix in a timely manner.

Google is backing oCERT, which is certainly encouraging. Google even mentions my "three conditions" for securing software (thanks!):

people need to actually review the code
developers/reviewers need to know how to write secure code
once found, security problems need to be fixed quickly, and their fixes distributed quickly

Clearly, something like oCERT could help with these.

This ComputerWorld article on oCERT makes some interesting points. One minor point: They worry that oCERT is using the term "CERT" without permission, but oCERT reports that they do indeed have that permission.

OSS and the U.S. DoD - Questions and Answers

Tue, 11 Mar 2008 20:19 GMT

I've just posted Questions and Answers for 2008 "Open Source Software and DoD" Webinar. These are my attempts to answer the questions people sent me at my February "Open Source Software (OSS) and the U.S. Department of Defense (DoD)" Some of the questions were easy to answer, but some were surprisingly difficult. In some cases, I asked lawyers and got conflicting answers. But this is the best information that I could find on the topic.

For example, I explain in detail why In particular, it appears fairly clear that both the government and government contractors can release their results as open source software under the default DoD contract terms for software development (DFARS contracting clause 252.227-7014):

The government can release software as OSS once it receives "unlimited rights" to it. Unless other arrangements are made, the government has unlimited rights to software components when (1) it pays entirely for their development, or (2) five years after contract signature if it partly paid for their development. Before award, a contractor may identify the components that will have more restrictive rights (e.g., so the government can prefer proposals that give the government more rights). Where possible, software developed partly by government funds should broken into a set of smaller components at the "lowest practicable level" so the rules can be applied separately to each one. Of course, the software can only be released to the public as OSS if other laws are also met (such as classification, export control, patent law, and trademark law).
Normally a DoD contractor can release the software as OSS at any time, since it holds the copyright. This default can be overridden by the contract, e.g., DFARS 252.227-7020 assigns copyright to the government, and is included in some contracts. Again, this release can only occur if other laws are also met (such as classification, export control, patent law, and trademark law).

These are the usual defaults; negotiations can change things, so read the contract to see if the contract changes these defaults. For example, sometimes the government has copyright assigned to it, in which case it can release the software simply because it has the copyright.

I also point out that even when the government isn't the copyright holder, if it releases software under an OSS license it can still enforce its license. That's because, even when it's not the copyright holder, it can still enforce the license... and because the doctrine of unclean hands will impact those who refuse to obey the license.

Several people had questions about software developed by a government employee (which can't be copyrighted in the U.S.) and how that impacts OSS. The short impact is that there's no problem; government employees can still contribute to OSS projects, for example. I also discuss some of the export control issues (especially ITAR), and how to address them.

If there are mistakes, please let me know. Thanks!

OSS and the U.S. DoD - Webinar

Mon, 28 Jan 2008 17:59 GMT

I'm going to present a webinar on "Open Source Software (OSS) and the U.S. Department of Defense (DoD)" on Feb 11, 2008, 3:00-4:30pm EST. It is open to the public, at no charge. To find out how to sign up, see http://www.dwheeler.com/oss-dod-webinar2008.html.

Here's the summary: "Open source software (OSS) has become widespread, but there are many misconceptions about it - resulting in numerous missed opportunities. This presentation will clarify what OSS is (and isn't), rebut common misunderstandings about OSS, discuss the relationship of OSS and security, discuss how to find and evaluate OSS, and explain OSS licensing (including how to combine products and select a license). It will show why nearly all extant OSS is COTS software, and thus why it's illegal (as well as foolish) to ignore OSS options."

This presentation is hosted by the Data & Analysis Center for Software (DACS), which is technically managed by the Air Force Research Laboratory - Information Directorate (AFRL/IF).

Please sign up quickly, if you're interested. There were 45 registrants in the first half hour of its announcement.

Please point me to High-Assurance Free-Libre / Open Source Software (FLOSS) Components

Mon, 05 Nov 2007 14:19 GMT

I'm looking for High Assurance and Free-Libre / Open Source Software (FLOSS) components. Can anyone point me to ones I don't know about? A little context might help, I suppose...

A while back I posted a paper, High Assurance (for Security or Safety) and Free-Libre / Open Source Software (FLOSS). For purposes of the paper, I define “high assurance software” as software where there’s an argument that could convince skeptical parties that the software will always perform or never perform certain key functions without fail. That means you have to show convincing evidence that there are absolutely no software defects that would interfere with the software’s key functions. Almost all software built today is not high assurance; developing high assurance software is currently a specialist’s field.

High assurance and FLOSS are potentially a great match. We achieve high assurance in scientific analysis and mathematical proofs by subjecting them to peer review, and then worldwide review. FLOSS programs, unlike proprietary programs, can receive similar kinds of review, and many FLOSS programs achieve really good reliability figures for medium assurance. So it can be easily argued that stepping up to high assurance should be easier for FLOSS than for proprietary software. In addition, there are a vast number of FLOSS tools that support developing high assurance components, including PVS, ACL2, Isabelle/HOL, prover9, and Alloy (there's lots more, see the paper for more details).

Yet it's hard to find High Assurance Free-Libre / Open Source Software (FLOSS) components. To be fair, high assurance software components are exceedingly rare in the non-FLOSS world as well. But I suspect there are more than I've found, and I hope that people will help me by pointing them out to me. I'd like to know about such components for direct use, and also simply for use as demonstrations of how to actually develop high assurance software. Today, it's nearly impossible to explain how to develop high assurance software, because there are almost no fully-published examples. Existing "formal methods successes" papers generally don't publish everything including their specs, proofs, and code... which makes it impossible to really learn how to do it. And no, I don't require that proofs prove every line of machine code; but showing how correspondance demonstrations are made would be valuable, and that's currently not well demonstrated by working examples.

If you're curious about the general topic, take a peek at High Assurance (for Security or Safety) and Free-Libre / Open Source Software (FLOSS). I've collected lots of interesting information; hopefully it will be useful to some of you. And let me know of high-assurance FLOSS components that I don't already know about.

FLOSS License Slide - Modified for LGPL

Mon, 24 Sep 2007 22:31 GMT

My "Free-Libre / Open Source Software (FLOSS) FLOSS License Slide" helps people figure out if the most widely-used FLOSS licenses are compatible (that is, if you can combine their software to produce new works). The main body of the PDF text fits all on one page, which can be handy.

My thanks to Olaf Schmidt, who found an error of omission in the previous version of my "slide". He pointed out to me that the GNU Lessor General Public License (LGPL) 2.1 is even more compatible with various versions of the GPL than the plain reading of my "slide" originally noted. That's because the LGPL has explicit text noting that you can switch its license to GPL version 2 or later, and similarly LGPL version 3 explicitly says that you can also use GPL version 3 or later. I fixed this by adding one more arrow to my diagram, which was enough to capture this. I also noted that the previous version of the LGPL is version 2.1, not 2 (previous versions of the LGPL exist but are becoming uncommon). I also added some additional text, so I hope that the LGPL-related text is even clearer now.

Enjoy!

Navy: Open Source Software IS Commercial Software

Wed, 18 Jul 2007 17:52 GMT

On June 5, 2007, the U.S. Navy released some guidance on Open Source Software. In particular, they noted that if Open Source Software (OSS) met the U.S. law definition of "commercial item", it was a commercial item. (They actually say OSS that meets that definition is commercial off-the-shelf (COTS), not just a "commercial item" - presumably because they had in mind the off-the-shelf open source software.) I'm delighted to see this guidance, because I've been saying the same thing. This Navy memo was pretty clear, yet some people seemed to have really odd interpretations of it. In particular, GCN reported that "After several years of evaluation, the Navy Department has approved the use of open-source software in all Navy and Marine Corps information technology systems." This GCN article makes it seem like there's been some big change in direction, and I think that's a terrible misunderstanding of what's going on here.

I believe this 2007 Navy memo is not a change in policy or direction. This Navy memo merely tries to counter a widespread misunderstanding that is sometimes resulting in a failure to obey U.S. law, the U.S. government's Federal Acquisition Regulations (aka the FAR), and (by implication) the U.S. Department of Defense's (DoD) Defense Federal Acquisition Regulation Supplement (DFARS). This memo also serves as a restatement that Navy policy continues to obey existing DoD and U.S. government policies regarding open source software (OSS), as were already formally established in 2003-2004.

Below are some supporting details to justify those statements. I hope they will help put this memo in context. As is usual in any blog, my conclusions are just my own opinion, not the official position of any organization. On the other hand, I think I have really good evidence! So let's see...

Years ago some people had the strange idea that OSS was prohibited in the DoD or U.S. federal government, even though there was no such prohibition. This was particularly bizarre in the DoD, since a MITRE report (final publication early 2003) found that OSS use was already widespread and very helpful to the DoD. That MITRE report concluded that "Neither the survey nor the analysis supports the premise that banning or seriously restricting [Free / Open Source Software (FOSS)] would benefit DoD security or defensive capabilities. To the contrary, the combination of an ambiguous status and largely ungrounded fears that it cannot be used with other types of software are keeping FOSS from reaching optimal levels of use.": http://www.terrybollinger.com/dodfoss/dodfoss_pdf_hyperlinked.pdf

So in May 2003 an official DoD policy memo ("OSS in DoD") was released. It affirmed that OSS was fine as long as it met the applicable DoD requirements, just as any other kind of software had to meet the applicable DoD requirements: http://www.egovos.org/rawmedia_repository/822a91d2_fc51_4e6e_8120_1c2d4d88fa06?/document.pdf

This problem was government-wide, so in July 2004, OMB released a similar policy memo (M-04-16), which explicitly stated that U.S. federal government acquisition policy was neutral about using OSS vs. proprietary software. In particular, it said that government "policies are intentionally technology and vendor neutral, and to the maximum extent practicable, agency implementation should be similarly neutral.": http://www.whitehouse.gov/omb/memoranda/fy04/m04-16.html

Yet there still seemed to be some strange misunderstandings, in spite of these 2003 and 2004 policy memos explicitly stating that U.S. DoD and federal acquisition policies were neutral on the question of using OSS vs. proprietary software (they merely had to obey the usual requirements). More recently, these misunderstandings seem to revolve around a failure to read and understand the term "commercial item" as defined by U.S. Code Title 41, Chapter 7, section 403, as well as its corresponding FAR text. These define a "commercial item" as an item "customarily used by the general public or by non-governmental entities" (i.e., they have uses not unique to a government) and have been "sold, leased, or licensed to the general public"). It would seem obvious that if OSS meets the U.S. law/FAR definition for a commercial item, it is a commercial item for government acquisition purposes. And nearly all extant OSS does indeed meet this definition; nearly all extant OSS has non-government uses and are licensed to the public. In addition, almost all already-existing OSS software also meets the definition of commercial-off-the-shelf (COTS), since they are commercial items that are ALREADY available to the public ("off the shelf").

The problem was that some acquisition programs were redefining the term "commercial item" (and COTS) to exclude OSS competitors. These redefinitions were in contradiction to the existing DoD and federal government-wide explicit policy for neutrality regarding OSS, and in contradiction to the clear definition of "commercial item" given in U.S. law, the FAR, and by implication the DFARS. The Navy memo simply tries to correct this misunderstanding, as well as re-iterating that the existing DoD and federal government policy on OSS continues. This Navy memo was signed by Department of the Navy CIO Robert J. Carey on June 5, 2007, and titled "Department of the Navy Open Source Software Guidance". You can find the Navy memo here: http://oss-institute.org/Navy/DONCIO_OSS_User_Guidance.pdf

The main implication of this definition of "commercial item" is that (as required by law and the FAR) contractors and their subcontractors at all tiers must do market research of the commercial market and consider ALL their commercial options... including the OSS options. This is certainly NOT a special preference for OSS, and ALL evaluation characteristics for software are still valid (e.g., functionality, total cost of ownership, quality, security, support, and flexibility). But in cases where the OSS option is the better option, by policy the U.S. government intends to take advantage of it.

This approach makes sense, given the major changes that are happening in the software industry. In many market segments OSS programs are the #1 or #2 product by market share, and OSS in aggregate now represents billions of dollars of development effort. Many companies are developing OSS and/or selling commercial support for OSS, including Red Hat, Novell, Sun Microsystems, IBM, and MySQL AB. Microsoft competes with some OSS programs, business models, and licenses, but in other areas Microsoft uses, develops, and encourages OSS (Microsoft's Windows includes BSD-developed networking applications; Microsoft OSS projects include WiX and IronPython; Microsoft runs the "Codeplex" website to encourage OSS development on Windows). In areas where they are appropriate some major OSS programs have received the relevant Common Criteria or FIPS 140-2 IT security certificates.

OSS potentially affects how acquisition programs acquire software, but acquisition programs should expect to be affected by changes in relevant commercial industries. This was anticipated; the DoD policy memo "Commercial Acquisitions" (Jan. 5, 2001) explains that the benefits of commercial item acquisition include "increased competition; use of market and catalog prices; and access to leading edge technology and 'non-traditional' business segments". In other words, DoD policy anticipates that there WILL be "non-traditional business segments" - and its policy is to embrace and exploit such changes where appropriate. (Given its growth and breadth, it's become increasingly difficult to argue that OSS is "non-traditional" anyway.) AT&L's "Commercial Item Handbook" (November 2001) explains that this broad definition of "commercial item" is intentional, because it "enables the Government to take greater advantage of the commercial marketplace." See: http://www.acq.osd.mil/dpap/Docs/cihandbook.pdf

In other words, U.S. contractors must consider all their options, and then select the best one. They are not allowed to arbitrarily ignore a relevant commercial industry sector, and are specifically not allowed to ignore OSS options.

If you're interested in this topic, you might also be interested in some related articles of mine, such as Open Source Software (OSS) in U.S. Government Acquisitions and “Commercial” is not the opposite of Free-Libre / Open Source Software (FLOSS): Nearly all FLOSS is Commercial.

FLOSS License Slide - Released!

Wed, 11 Jul 2007 23:17 GMT

There are a large number of Free-Libre / Open Source Software (FLOSS) licenses, but only a few are widely used. Unsurprisingly, the widely-used licenses tend to be compatible -- that is, it’s possible to combine software under the different licenses to produce a larger work. But many people have trouble figuring out when they can be combined, or how.

So I've created a little figure which I call the “FLOSS license slide” to make it easier to see if FLOSS licenses can be combined in many common cases, and if so, what the basic ramifications are. I've crafted it so that the figure and explanatory text all fit in a page, which can be handy.

You can look at the FLOSS license slide in one of three formats:

I had released a draft of this earlier, and got some nice feedback. I added "public domain"; there seemed to be enough questions about public domain software that it made sense to add that to the figure. Yes, I know that technically "public domain" isn't a license, but it's much easier to understand and explain by treating it as if it were one. Some of the earlier text was not clear enough; hopefully this one is clearer. For example, GPLv2+ and the Affero GPL 3 licenses are compatible via GPLv3, but some people had trouble understanding why. Now the text notes that they are compatible via the GPL version 3, which will hopefully make that clearer. I also added the HTML format.

I'm not a lawyer, and if you need formal legal advice you need to consult your own attorney. But for many people, this is the information you needed, in a conveniently small format, so here it is. Enjoy!

Increasing Government Interest in Free-Libre / Open Source Software (FLOSS)

Sun, 08 Jul 2007 16:10 GMT

There's more on the web now about governments and Free-Libre / Open Source Software (FLOSS), both on my own site and other sites. I think this suggests more and more interest by governments in FLOSS.

I've now posted Open Source Software (OSS) in U.S. Government Acquisitions, which is a slightly-updated version of my article which was published in the "DoD Software Tech News" issue devoted to free-libre / open source software (FLOSS). As I noted earlier, this issue has lots of good articles. I was asked to create the article (not the other way around), so I think the very existance of this issue is evidence of increasing interest, not just a personal interest of mine.

I should probably mention why I'm posting a revision. After I submitted my article, the Department of the Navy CIO Robert J. Carey signed a June 5, 2007 memorandum titled “Department of the Navy Open Source Software Guidance” which notes that FLOSS needs to be considered a commercial item when it meets the U.S. government's standard definition of a commercial item (nearly all extant FLOSS does). That is the same basic point that I raised in my presentation and paper, and I would have gladly mentioned the Navy memo... had it been released at the time. So my modified version now points to the Navy memo, as well as making a few tweaks based on other feedback. This realization that FLOSS programs are (almost always) commercial items is an important point in the U.S. government. Why? Because as noted in Linux.com, that means that extant FLOSS software must be considered when the U.S. government acquires software the same way as other commercial software is considered (i.e., it must be considered before starting a new project to write their own). The Navy memo's assertion of this makes it worth posting an update.

But think of this updated essay as a only sampler... for the rest of the articles, you'll still need to go read the "DoD Software Tech News" issue. Getting the issue does require registration, but registration is free and in this case it's worthwhile.

Anyway, it's not just one issue about FLOSS in one magazine. Here's more evidence it's not just me - on July 5, 2007, the article Open Source Government: good-will needed was posted by Roberto Galoppini's Commercial Open Source Software blog. He points in turn to various articles, including Matt Asay's "Open source in government: Leadership needed", which then leads you back to a very interesting research paper: "Open-Source Collaboration in the Public Sector: The Need for Leadership and Value" by Michael P. Hamel. There's lots of interesting discussion in those articles about how governments can use FLOSS, and in particular how governments can use FLOSS components and approaches more effectively. I'll throw in my own two cents here. I would certainly agree that leadership is important in any project, including FLOSS projects; the leadership of Linus Torvalds of the Linux kernel is well-documented, for example. But there are different kinds and levels of leadership. Roberto Galoppini says, "I think that first we need politicians with good-will, willing to put their intellectual potential to work for the overall desires of the general public..."; while that's good, often what's important is the ability to lead concept into practice. It's easy to "lead" by saying words, but I think that often what's needed is the kind of leadership that rolls up its sleeves and makes actual projects produce useful results. Focusing on specific, measurable products can often get better results. Certainly politicians with goodwill are a good thing, but they can provide value primarily by providing "cover" for those who actually do the hands-on leading of projects, but the latter make or break such efforts.

While I can quibble with some of the stuff Hamel says, some of the his statements seem dead-on with what I see. Hamel notes that "participants in both groups also believe that the creation of value, or products that are appropriate and effective in addressing members’ wants and needs, is critical"... to which I say, Amen. Hamel concludes that "Collaborations with a strong leadership structure, and more importantly a single leader who is persistent, passionate and willing to spend a great deal of time maintaining and improving the organization are much more likely to succeed. Value is also a critical component, and requires that efforts meet the wants and needs of members and clients, whether they be in the form of software, documentation, research or even policy advocacy." Focusing on a few most useful projects is critical: "a conscious effort to focus energy on a small number of projects in early stages may be an important component in creating value for members of collaborative efforts." A FLOSS project requires collaboration to be successful, and collaboration requires that the project gain the trust of potential users/developers; "In this research I found that leadership, face-to-face contact, and the legal framework were the primary factors leading to trust. A willingness and ability to evolve, which may be tied to creating products of value to clients and members, might also be an important factor in developing a successful collaboration." Those statements, at least, seem very sound. These aren't new ideas, to be sure, but it sure is easy to lose sight of them.

I think there's a vast opportunity here for governments to use FLOSS. Which is odd in a sense, because in fact, FLOSS is already widely used in governments. MITRE determined back in 2003 that FLOSS was already critically important to the U.S. Department of Defense (DoD). But that widespread use is small compared to its potential, and to how many commercial organizations use FLOSS, and there are a lot of reasons for that. Governments have long aquisition times, often tend to avoid doing anything "risky" (aka "different than the way we did it before", abetted because there's no competitor to force them to improve), and many proprietary vendors target governments to try to prevent them from using competing FLOSS products. Which means that even when it makes sense to use FLOSS, it can often take much longer for FLOSS components and approaches to enter into government use. But it has already entered into significant use, and I expect that to accelerate over the years.

There are some other goodies on my web site if you're interested in this topic (government and FLOSS). My essay is basically a text version of my March 2007 presentation on Free-libre / Open Source Software (FLOSS). It includes snippets from much longer papers that give statistics about FLOSS, explain how to evaluate FLOSS, discuss FLOSS and security, and explain why most FLOSS is commercial software. There's even a hint of my essay Make Your Open Source Software GPL-Compatible. Or Else, since I note the widespread use of the GPL.

Again, if you have a true PHB, I can't help you. But many managers are trying to the right thing, and just need reasonable information so that they can do the right thing.

"DoD Software Tech News" posts open source software issue

Tue, 19 Jun 2007 12:44 GMT

The U.S. Department of Defense (DoD)'s software technology magazine "DoD Software Tech News" has posted a whole issue devoted to free-libre / open source software (FLOSS). If you're trying to get FLOSS seriously considered by acquisition or management people, this may be what you need. This issue includes essays by David A. Wheeler (that's me!), Terry Bollinger, John M. Weathersby, Mark Lucas (on Geospatial FLOSS), Peter Gallagher, Matt Asay (Alfresco), and Andrew Gordon. Free registration required. My essay is basically a text version of my March 2007 presentation on Free-libre / Open Source Software (FLOSS); it includes snippets from my papers that give statistics about FLOSS, explain how to evaluate FLOSS, discuss FLOSS and security, and explain why most FLOSS is commercial software. There's even a hint of my essay Make Your Open Source Software GPL-Compatible. Or Else, since I note the widespread use of the GPL.

After I submitted my article, but before it got published, the Department of the Navy CIO Robert J. Carey signed a June 5, 2007 memorandum titled “Department of the Navy Open Source Software Guidance” which notes that FLOSS needs to be considered a commercial item when it meets the U.S. government's standard definition of a commercial item (and nearly all extant FLOSS meets that definition). That is the same basic point that I raised in my presentation. It feels nice to have made a key statement, and then find that the U.S. Navy officially confirms it. As noted in Linux.com, that means that extant FLOSS software must be considered when the U.S. government acquires software, the same way as other commercial software is considered (i.e., it must be considered before starting a new project to write their own).

If you have a true PHB, I can't help you. But many managers just need some honest information, and this is at least one way to get it to them.

FLOSS License Slide

Mon, 11 Jun 2007 22:48 GMT

You can look at the FLOSS license slide in one of two formats:

This is currently a draft, because I'd like to hear comments before "finishing" this. Also, it's based on the "final draft" of GPLv3; things could change before that revision is complete (though I doubt it). There's no end in trying to add other licenses, but if there's a big mistake in this document, I'd like to know. I'm not a lawyer, and if you need formal legal advice you need to consult your own attorney. But for many people, this is the information you needed, so here it is.

Funny "I'm Linux" ads from Novell

Sat, 26 May 2007 17:53 GMT

This came out a little while ago, but I just saw them and they're hilarous. Here are three ads from Novell that spoof Apple's "I'm a Mac" ads, and I thought they were really funny and well-done.

Here are links to the Hi-Res Novell Videos:

Here's some background about how these ads were created.

April 2007 release of "Why OSS/FS? Look at the Numbers!"

Thu, 12 Apr 2007 19:27 GMT

Finally, I've released a new version of "Why Open Source Software / Free Software (OSS/FS, FLOSS, FOSS)? Look at the Numbers!" This paper continues to provide "quantitative data that, in many cases, using open source software / free software (abbreviated as OSS/FS, FLOSS, or FOSS) is a reasonable or even superior approach to using their proprietary competition according to various measures. This paper’s goal is to show that you should consider using OSS/FS when acquiring software."

It's been a while; my last release was November 14, 2005. The ChangeLog has all the details, but here are some of the highlights:

Updated webserver stats, and noted issues with the Go Daddy change and lighttpd.
Noted Kenneth van Wyk's article about Linux security
Added quotes from Microsoft's Bill Hilf, from "Cracking Open the Door to Open Source" by Carolyn A. April, "Redmond" magazine, March 2007, pp. 26-36.
Added link to Andy Tanenbaum's article about Ken Brown and ADTI.
Added a link to an approved European Parliament resolution, A5-0264/2001, which calls "on the Commission and Member States to promote software projects whose source text is made public (open-source software), as this is the only way of guaranteeing that no backdoors are built into programmes [and calls] on the Commission to lay down a standard for the level of security of e-mail software packages, placing those packages whose source code has not been made public in the 'least reliable' category" (5 September, 2001; 367 votes for, 159 against and 39 abstentions).
Added a reference to the Forrester report "Open Source Becoming Mission-Critical In North America And Europe" by Michael Goulde that says "Firms Should Consider Open Source Options For Mission-Critical Applications".
Added references to a major new European Commission-sponsored study, "Study on the Economic impact of open source software on innovation and the competitiveness of the Information and Communication Technologies (ICT) sector in the EU", November 20, 2006. This is a major new study; "Our findings show that, in almost all the cases, a transition toward open source reports of savings on the long term". There is LOTS of quantitative information here.
Added reference to Communications of the ACM (CACM) Jan. 2007, "Increased Security through Open Source" It doesn't say anything new, and it omits the many quantitative studies cited here, but it's a prestigious journal that says it.
Added reference to mail server market survey: Sendmail and Postfix and #1 and #2 in the market.
Added references to defectivebydesign.org and to Raymond/Landley's "World Domination 201" into desktop section.
IE vs. Firefox unsafe days in 2006. Eek... it's scary.
Added Survey - Linux use on mission-critical systems
Added Danish cities demand more openness
Added "The war is over and Linux won" (Server war)
Added Evergreen, an open source, enterprise-class library management developed by the Georgia Public Library Service.
Added reference to TCO savings on OSS/FS databases, from "Open source databases '60 percent cheaper'" article
Added info Firefox use which keeps growing. See http://marketshare.hitslink.com/report.aspx?qprid=3 and http://www.techweb.com/wire/security/193104314
Added reference to IDC survey
Referenced "Trusting Trust" attack. Here's the text: "An Air Force evaluation by Karger and Schell first publicly described this very nasty computer attack, which Ken Thompson ably demonstrated and described in his classic 1984 paper "Reflections on Trusting Trust". Thompson showed that because we use software to create other software, if an attacker subverts the software-creating programs, no amount of auditing any program can help you - the subverted programs can hide whatever they want to! This has been called the "uncounterable attack", and some have said that it's impossible to secure computers simply because this attack is possible. Some have even said that all those security audits of OSS/FS are worthless, because subverted tools could insert attacks the auditors couldn't see. But it turns out that the trusting trust attack can be countered. My 2005 paper Countering Trusting Trust through Diverse Double-Compiling (DDC), published by ACSAC, shows how the "uncounterable" trusting trust attack can be countered. But there's a catch: the DDC defense only works if you can get the source code for your software creation tools, including the operating system, compiler, and so on. That kind of information is typically only available for OSS/FS programs! Thus, even in the case of the dangerous "trusting trust" attack, OSS/FS has a security advantage."
Added a note about Symphony OS (innovative user interface).
Added quote from Bellovin to history section. OSS was the norm in many communities before the mid-1970s.
Added stats from onestat.com re: Firefox usage
Added EMA study
Added Spyware stats, IE vs. Firefox, from University of Washington.
Added new reports on security flaw fixing time: http://blogs.washingtonpost.com/securityfix/2006/02/a_time_to_patch.html and http://www.heinz.cmu.edu/%7Ertelang/disclosure_jan_06.pdf.
Added "Deliverable D3: Results and policy paper from survey of government authorities". There's lots of other good stuff there.
Added reference to another paper on innovation.
Added reference to "Why open source projects are not publicised" by Ingrid Marson, ZDNet UK, November 25, 2005.

As I mentioned earlier, I wish I'd used the term "FLOSS" (Free-Libre / Open Source Software) as my all-encompassing term in this paper. FLOSS is much easier to say than some of the alternatives, and the term "Free Software" is widely misunderstood as being "no cost". However, I've used the term OSS/FS all over in the paper, and it's awkward to change now (and people might not find the document they were looking for), so I haven't changed it here.

Enjoy!

Presentation and audio of "Open Source Software" online

Tue, 20 Mar 2007 18:27 GMT

Earlier this month I gave a presentation about open source software (aka OSS, Free Software, or FLOSS) at a conference near Washington, DC. You can now download the March 2007 presentation "Open Source Software" in PDF format; you can also get it in OpenDocument format. For the OpenDocument version, make sure you have the fonts you need. Those are just the slides; I've separately made the audio available in several formats: [OGG (Vorbis)], [MP3], and [FLAC]. You should be able to understand the presentation just from the audio, but looking at the slides while listening to the audio is even better. For the audio, I recommend using the Ogg Vorbis format - it's the smallest file, and it has very good quality. The FLAC format is lossless, and is useful for recoding later (it's much smaller than WAV or AIFF while still not losing anything). The MP3 format is useful if your player can't handle Ogg Vorbis yet (complain to your manufacturer!) - while MP3 is an ISO standard, MP3 isn't an open standard because it's patent-encumbered.

The conference was titled "Open Source - Open Standards - Open Architecture", and was put on by the non-profit Association for Enterprise Integration (AFEI) (a member of the NDIA family of associations). A lot of people were particularly surprised to learn that essentially all open source software (FLOSS) are commercial off-the-shelf (COTS) software, a point I make in more detail in my essay ‘Commercial’ is not the opposite of Free-Libre / Open Source Software (FLOSS). Basically, the U.S. government’s own laws (particularly Title 10 section 101) and regulations (particularly the Federal Acquisition Regulation) make it clear that nearly all open source software is commercial off-the-shelf (COTS). There are two kinds of COTS software products: proprietary software and open source software.

"Commercial" is not the opposite of Free-Libre / Open Source Software (FLOSS)

Wed, 27 Dec 2006 17:31 GMT

Announcing "Commercial" is not the opposite of Free-Libre / Open Source Software (FLOSS) -- a new and hopefully useful essay!

Why this new essay? When I talk with with other people about Commercial Free-Libre / Open Source Software (FLOSS), I still hear a lot of people mistakenly use the term "commercial software" as if it had the opposite meaning of "open source software", Free-Libre Software, OSS/FS, or FLOSS. That's in spite of (1) the rise in commercial support for FLOSS, (2) official definitions of "commercial item" that include FLOSS, and (3) FLOSS licenses and projects clearly approving commercial support.

This confusion -- that FLOSS and commercial software are opposites -- is a dreadful mistake. Speakers who differentiate between FLOSS and commercial products, as if they were opposites, are simply unable to understand what is happening in the software industry. And if you cannot understand something, you cannot make good decisions about it.

If you wish to understand the 21st century (and beyond), you need to understand the basics of what controls software... because software controls everything else.

So, this essay "Commercial" is not the opposite of Free-Libre / Open Source Software (FLOSS) explains why it's so important to understand that the word "commercial" is not the opposite of FLOSS, and then gives examples to justify the claim.

Enjoy!

GPL, BSD, and NetBSD - why the GPL rocketed Linux to success

Fri, 01 Sep 2006 01:50 GMT

Charles M. Hannum (one of the 4 originators of NetBSD) has posted a sad article about serious problems in the NetBSD project, saying "the NetBSD Project has stagnated to the point of irrelevance." You can see the article or an LWN article about it.

There are still active FreeBSD and OpenBSD communities, and there's much positive to say about FreeBSD and OpenBSD. I use them occasionally, and I always welcome a chance to talk to their developers - they're sharp folks. Perhaps NetBSD will partly revive. But systems based on the Linux kernel ("Linux") absolutely stomp the *BSDs (FreeBSD, OpenBSD, and NetBSD) in market share. And Linux-based systems will continue to stomp on the *BSDs into the foreseeable future.

I think there is one primary reason Linux-based systems completely dominate the *BSDs' market share - Linux uses the protective GPL license, and the *BSDs use the permissive ("BSD-style") licenses. The BSD license has been a lot of trouble for all the *BSDs, even though they keep protesting that it's good for them. But look what happens. Every few years, for many years, someone has said, "Let's start a company based on this BSD code!" BSD/OS in particular comes to mind, but Sun (SunOS) and others have done the same. They pull the *BSD code in, and some of the best BSD developers, and write a proprietary derivative. But as a proprietary vendor, their fork becomes expensive to self-maintain, and eventually the company founders or loses interest in that codebase (BSD/OS is gone; Sun switched to Solaris). All that company work is then lost forever, and good developers were sucked away during that period. Repeat, repeat, repeat. That's enough by itself to explain why the BSDs don't maintain the pace of Linux kernel development. But wait - it gets worse.

In contrast, the GPL has enforced a consortia-like arrangement on any major commercial companies that want to use it. Red Hat, Novell, IBM, and many others are all contributing as a result, and they feel safe in doing so because the others are legally required to do the same. Just look at the domain names on the Linux kernel mailing list - big companies, actively paying for people to contribute. In July 2004, Andrew Morton addressed a forum held by U.S. Senators, and reported that most Linux kernel code was generated by corporate programmers (37,000 of the last 38,000 changes were contributed by those paid by companies to do so; see my report on OSS/FS numbers for more information). BSD license advocates claim that the BSD is more "business friendly", but if you look at actual practice, that argument doesn't wash. The GPL has created a "safe" zone of cooperation among companies, without anyone having to sign complicated legal documents. A company can't feel safe contributing code to the BSDs, because its competitors might simply copy the code without reciprocating. There's much more corporate cooperation in the GPL'ed kernel code than with the BSD'd kernel code. Which means that in practice, it's actually been the GPL that's most "business-friendly".

So while the BSDs have lost energy every time a company gets involved, the GPL'ed programs gain every time a company gets involved. And that explains it all.

That's not the only issue, of course. Linus Torvalds makes mistakes, but in general he's a good leader; leadership issues are clearly an issue for some of the BSDs. And Linux's ability early on to support dual-boot computers turned out to be critical years ago. Some people worried about the legal threats that the BSDs were under early on, though I don't think it had that strong an effect. But the early Linux kernel had a number of problems (nonstandard threads, its early network stack was terrible, etc.), which makes it harder to argue that it was "better" at first. And the Linux kernel came AFTER the *BSDs - the BSDs had a head start, and a lot of really smart people. Yet the Linux kernel, and operating systems based on it, jumped quickly past all of them. I believe that's in large part because Linux didn't suffer the endless draining of people and effort caused by the BSD license.

Clearly, some really excellent projects can work well on BSD-style licenses; witness Apache, for example. It would be a mistake to think that BSD licenses are "bad" licenses, or that the GPL is always the "best" license. But others, like Linux, gcc, etc., have done better with copylefting / "protective" licenses. And some projects, like Wine, have switched to a protective (copylefting) license to stem the tide of loss from the project. Again, it's not as simple as "BSD license bad" - I don't think we fully understand exactly when each license's effects truly have the most effect. But clearly the license matters; this as close to an experiment in competing licenses as you're likely to get.

Obviously, a license choice should depend on your goals. But let's look more carefully at that statement, maybe we can see what type of license tends to be better for different purposes.

If your goal is to get an idea or approach widely used to the largest possible extent, a permissive license like the BSD (or MIT) license has much to offer. Anyone can quickly snap up the code and use it. Much of the TCP/IP code (at least for tools) in Windows was originally from BSD, I believe; there are even some copyright statements still in it. BSD code is widely used, and even when it isn't used (the Linux kernel developers wrote their own TCP/IP code) it is certainly studied. But don't expect the public BSD-licensed code to be maintained by those with a commercial interest in it. I haven't noticed a large number of Microsoft developers being paid to improve any of the *BSDs, even though they share the same code ancestries in some cases.

If your goal is to have a useful program that stays useful long-term, then a protective ("copylefting") license like the LGPL or GPL licenses has much to offer. Protective licenses force the cooperation that is good for everyone in the long term, if a long-term useful project is the goal. For example, I've noticed that GPL projects are far less likely to fork than BSD-licensed projects; the GPL completely eliminates any financial advantage to forking. The power of the GPL license is so strong that even if you choose to not use a copylefting license, it is critically important that an open source software project use a GPL-compatible license.

Yes, companies could voluntarily cooperate without a license forcing them to. The *BSDs try to depend on this. But it today's cutthroat market, that's more like the "Prisoner's Dilemma". In the dilemma, it's better to cooperate; but since the other guy might choose to not cooperate, and exploit your naivete, you may choose to not cooperate. A way out of this dilemma is to create a situation where you must cooperate, and the GPL does that.

Again, I don't think license selection is all that simple when developing a free-libre/open source software (FLOSS) program. Obviously the Apache web server does well with its BSD-ish license. But packages like Linux, gcc, Samba, and so on all show that the GPL does work. And more interestingly, they show that a lot of competing companies can cooperate, when the license requires them to.

Ohloh, SLOCCount, and evaluating Free-libre / Open Source Software (FLOSS)

Tue, 18 Jul 2006 02:39 GMT

A new start-up company named Ohloh has recently appeared, and is mentioned in many articles such as those from Ars Technica, eWeek, and ZDNet. This start-up will do analysis of free-libre / open source software (FLOSS) projects for customers, as well as do analysis of in-house proprietary software. To do the analysis, they'll use a suite of FLOSS tools. Some articles suggest that they'll also help customers determine which FLOSS programs best fit their needs, by basing their recommendations on this analysis (at least in part). Exactly what this start-up will do is hard to figure out from the news reports. That's understandable... this is a start-up, and no doubt the start-up itself will adjust what it does depending on who shows up as customers. But the general niche they're trying to fill seems clear enough.

This seems like a very reasonable business idea. Indeed, companies like IBM, Sun, and HP have been helping customers select programs and develop systems for a long time, and they often recommend and help transition customers to FLOSS programs. IBM has said they invested over a billion dollars in Linux, and have already recouped it, so IBM shows that this can be a lucrative business model. Ohloh will have some stiff competition if they want to muscle into the job of giving recommendations, but I love to see competition; hooray! And I have stated for years that I'd love to see more analysis of FLOSS programs, so I'm happy that it's happening.

I am amused, though. Some of the articles about this start-up seem to suggest that this kind of data was impossible to get before. Yet after a quick web search, you'll discover that a lot of what they discuss is already here on my website, and has been for years. The article in Ars Technica says Ohloh intends to "analyze open-source software projects and provide customers with detailed information about them, including how much it would cost to duplicate the project given an average programmer salary of US$55,000 per year. The Linux kernel, for example, clocked in at nearly 4.7 million lines of code, has had 1,434 man-years of coding effort put in so far, and would have cost approximately US$79 million in salaries." I'm all for analyzing code, but to do that kind of analysis, you can just run my program SLOCCount. You can even see my papers discussing the results of SLOCCount applied to the Linux kernel, or my More than a Gigabuck paper (which analyzed a whole Linux distribution). In fact, I'll bet that they are using SLOCCount as one of their tools, even though it appears to be uncredited (shame on you!). After all, they state that they're using FLOSS programs to do their analysis, and some of the reports they're generating include exactly the kind of data that only SLOCCount provides. This is not a license problem - I'm glad they've found it useful! And to be fair, they're not just using SLOCCount; they're also gathering other data such as check-in rates, and they're grouping the results as a nice prepackaged web page. There are other tools that do some similar analysis of project activity, of course, such as CIA. Still, I think making information like this more accessible is really valuable... so good show!

If they plan to go beyond number-generating, and move into the business of giving specific advice, you could do the same thing yourself. Just go read my paper, How to Evaluate Open Source Software / Free Software Programs. That paper outlines steps to take, and the kind of information to look for. Doing things well is much harder than just following some process, of course, but at least you'll have an idea of what should be done.

So it turns out that the basic tools, and basic approaches, for doing this kind of analysis and making recommendations already exist. Is that a problem for this start-up? Not really. Not every company has the skills, knowledge, or time to do these kinds of analyses. It's quite reasonable to hire someone who specializes in gathering a particular kind of knowledge or doing a particular kind of analysis... people who work in one area tend to get good at it. I don't know how well they'll do, and execution always matters, but their idea seems reasonable enough.

Frankly, what's more interesting to me is who's starting the company - it's basically lots of former Microsoft folks! It's headed by two former Microsoft executives: Scott Collison (former director of platform strategy at Microsoft) and Jason Allen (a former development manager for XML Web Services). Investors include Paul Maritz (who was a member of the Microsoft executive committee and manager of the overall Microsoft company from 1986 to 2000) and Pradeep Singh (who spent nine years at Microsoft in various management positions). Years ago, Microsoft's Halloween documents revealed their deep concern about FLOSS, and how they were going to try to bury it. Hmm, it doesn't seem to be very buried. I've no idea where this will end up, but it sure is interesting.

The Wisdom of Crowds and Free-Libre / Open Source Software

Sat, 24 Jun 2006 21:21 GMT

I just came across an interesting short essay by Dr. Les Hatton titled "Open source inevitably good"; it appears it was published in the July 2005 IT week. He has some intriguing conclusions about free-libre / open source software (FLOSS).

But first, a little about Dr. Hatton, to show that he is no lightweight. Dr. Hatton holds the Chair in Forensic Software Engineering at the University of Kingston, UK, he is a fellow of the British Computer Society, and was voted in "World's leading Scholars of Systems and Software Engineering (1993-2002)" by U.S. Journal of Systems and Software. His work in computer science has primarily been in the field of software failure, especially the design and execution of experiments to determine the cause and reduce the likelihood of failure in software systems. He's particularly known for his work on safer language subsets, such as "Safer C". One paper of his I especially like is "EC--, a measurement based safer subset of ISO C suitable for embedded system development" - in this, he measures the common mistakes made in C by professional developers, and then proposes simple rules to reduce their likelihood (if you write software in C, it's definitely worth reading). In any case, here is someone who understands software development, and in particular has carefully studied why software fails and how to prevent such failures in the future.

In his essay "Open source inevitably good", Hatton starts by first examining James Surowiecki's interesting book "The Wisdom of Crowds: Why the Many are smarter than the Few". It turns out that crowds working together regularly beat the experts; there's both good evidence for this (with legions of examples), and good mathematical underpinnings justifying this too. For this to happen, two simple conditions must be met: they must all have some knowledge, and they must act effectively independently.

He notes that while the "many eyeballs" theory of Raymond still operates, this "wisdom of the crowds" also has a strong effect. In short, FLOSS software development often appears chaotic because much of it uses a "survival of the fittest" development approach; several different ideas are tried, and then the most successful approach is selected by many others. When viewed through the lens of the "wisdom of crowds", this is an entirely sensible thing to do. He concludes this startling way: "High quality open source isn't a surprise, it's inevitable."

Obviously, there has to be a crowd for this concept to hold. But there are many FLOSS projects where it's obvious that there is a crowd, and where the results are really very good. So take a peek at Les Hatton's "Open source inevitably good". It's an interesting and provocative piece that will make you think.

Autonumbering supported in Firefox 1.5!

Fri, 02 Jun 2006 21:31 GMT

Here's another reason to use Firefox as your web browser, besides the fact that Firefox has a better security record and that Firefox has better support for web standards in general. Firefox 1.5 has added autonumbering support, and sites like mine are starting to use it. If you're using a non-compliant web browser, like the current version of Internet Explorer, you're missing out. But let's back up a bit to the basics: HTML.

HTML has been a spectacularly successful standard for sharing information - web pages around the world use it. I write a lot of my papers directly in HTML, because it's easy, using HTML makes them easily accessible to everyone, and it's a completely open standard.

But HTML has several weaknesses if you're writing long or technical reports. One especially important one is automatic numbering of headers: the original HTML specification can't do it. When you're reading a long report, it can be hard to keep track of where you are, so having every heading numbered (such as "section 2.4.3") is really helpful. This can be solved by having programs directly insert the heading numbers numbers into the HTML text, but that's a messy and kludgy solution. It'd be much better if browsers automatically added numbered headings where appropriate, so that the HTML file itself is simple and clean.

The W3C (the standards group in charge of HTML and related standards) agreed that automatic numbering was important, and included support for automatic numbering in the Cascading Style Sheets (CSS) standard way back in 1998. CSS is an important support standard for HTML, so that should have been it... but it wasn't. Both Netscape and Microsoft decided to not fully implement the standard, nor try to fix the standard so that they would implement it. Soon afterwards Microsoft gained dominant market share, and then let their browser stagnate (why bother improving it, since there was no competition?). It looked like we, the users, would never get basic capabilities in HTML like auto-numbering.

I'm happy to report that Firefox 1.5 has added support for auto-numbering headings and other constructs too. So I've modified my CSS file for papers and essays so it auto-number headings; I've released my CSS file under the MIT/X license, so anyone else can use it. If you develop web content you may want to look at examples like mine for a reason, because...

It turns out that the story is more complicated. In the process of implementing auto-numbering, the Firefox developers found a serious problem with the CSS specification. Oops! The Mozilla Firefox bug #3247 and David Flanagan's blog discuss this further. The Firefox developers talked with the W3C, and the W3C ended up creating "CSS 2.1", an updated/patched version of the CSS2 standard that is in the process of being formally released.

What this means is that the examples for autonumbering in the "official" original CSS2 standard won't actually work! Instead, you need to follow a slightly different approach as defined in the patched CSS2.1 specification.

Technical stuff: The basic problem involves scoping issues. To solve it, the counter-reset property must be in the main heading names (like h1, h2, etc.), and not in the "before" sections (like h1:before, h2:before, etc.) - in spite of all the examples in the original CSS2 spec. You can put counter-increment in either place, though the spec puts them in the :before sections so I have too.

Now people have yet another reason to upgrade to Firefox. Firefox has had better standards support for some time; there are now many sites that won't display properly (or as well) if your browser doesn't support the standards well. But here is a clear and functionally important difference.

I'm a big believer in standards, but they can only help users if they are implemented, and they will only be implemented if users demand standards compliance. I think that the more people switch to standards-compliant browsers, and the more that sites use standards (to encourage people to switch), the more pressure it will bring on the other browser makers to catch up. And that would be great for all computer users.

More broadly, this is also a good example of why it's important to have implementations try out standards before they are frozen; they help avoid mistakes like this. Today, essentially every successful open standard is implemented by free-libre/open source software (FLOSS) - this makes sure that the standard is implementable, helps all understand what the standard means, and also helps other developers understand at least one way to implement it. This doesn't mean standards aren't important; standards are vital! And this also shows that when a mistake is made by a standards body, life is not over; standards bodies can work with implementors to fix problems. In fact, this shows that the best standards are those created from an interplay between standards developers and implementors, where standards are then made official after actual implementation experience.

I'll be speaking at some Linux User Groups (LUGs)

Wed, 17 May 2006 00:40 GMT

A while ago I was asked to speak at some of the Linux User Groups (LUGs) in the Washington, DC area, and I agreed to do so. Here are the current plans, if you are interested in hearing me speak:

May 17, 2006, 7pm: DCLUG, Washington, DC. "FLOSS and security." Get directions to their meeting location: 2025 M Street NW, Washington DC.
July 1, 2006, 10am: NovaLUG. Meeting is at Washington Technology Park/CSC (formerly Dyncorp), 15000 Conference Center Drive, Chantilly, VA. (I've just sent an email to confirm, but that's the date I have.)
July 12, 2006, 7pm: Columbia LUG. "Open Standards and Security (and OpenDocument too)" Meeting is at HP, 8890 McGaw Rd Ste 100, Columbia, MD.

Plans may change, but this is the information I have for now.

High Assurance (for Security or Safety) and Free-Libre / Open Source Software (FLOSS)

Sat, 06 May 2006 10:52 GMT

Recently I spoke at an Open Group conference and gave my presentation on Open Source Software and Software Assurance (Security). While there, someone asked a very interesting question: "What is the relationship between high assurance and open source software?" That's a fair question, and although I gave a quick answer, I realized that a longer and more thoughtful answer was really needed.

So I've just posted a paper to answer the question: High Assurance (for Security or Safety) and Free-Libre / Open Source Software (FLOSS). For purposes of the paper, I define “high assurance software” as software where there’s an argument that could convince skeptical parties that the software will always perform or never perform certain key functions without fail. That means you have to show convincing evidence that there are absolutely no software defects that would interfere with the software’s key functions. Almost all software built today is not high assurance; developing high assurance software is currently a specialist’s field. But I think all software developers should know a little about high assurance. And it turns out there are lots of connections between high assurance and FLOSS.

The relationships between high assurance and FLOSS are interesting. Many tools for developing high assurance software are FLOSS, which I can show by examining the areas of software configuration management, testing, formal methods, analysis implementation, and code generation. However, while high assurance components are rare, FLOSS high assurance components are even rarer. This is in contrast to medium assurance, where there are a vast number of FLOSS tools and FLOSS components, and the security record of FLOSS components is quite impressive. The paper then examines why this is the circumstance. The most likely reason for this appears to be that decision-makers for high assurance components are not even considering the possibility of FLOSS-based approaches. The paper concludes that in the future, those who need high assurance components should consider FLOSS-based approaches as a possible strategy.

Anyway, it's a thought piece; if you're interested in making software that is REALLY reliable, I hope you'll find it interesting.

Again, the paper is here: High Assurance (for Security or Safety) and Free-Libre / Open Source Software (FLOSS).