I've analyzed the BROWSER convention by Eric S. Raymond and found some security problems, as well as some other issues.
Raymond's specification doesn't say whether or not the shell (/bin/sh) is invoked, but his sample code does call the shell. For example, his patch to urlview calls C's system() call, which calls the shell. As a result, certain URLs provided by external parties could cause arbitrary code to execute. An attacker can create a hypertext link like " ; /bin/rm -fr / ; " and increase his victim's free disk space :-). This can even happen indirectly through a carefully-chosen filename. Also, the convention doesn't handle some filenames well as-is.
Here's my security analysis - it primarily captures information I've gathered from various sources that led to the proposed algorithm.
I investigated several areas. Many characters are not legal in URI/URLs; conceivably a browser could be exploited this way. Filenames can essentially have any character but ASCII 0, and clearly arbitrary filenames must not cause programs to be executed instead. Also, some browsers get the reference request indirectly (e.g., Netscape and Mozilla's "-remote" scheme), so their communication path needs to be examined. If the shell is called, any shell metacharacters must be escaped before sending them to the shell,
Thus, I looked carefully at the syntax of URI/URLs, shell metacharacters, and Netscape's transfer mechanism.
First, I examined the various specifications on URI/URL syntax, particularly RFC 1738 (http://www.ietf.org/rfc/rfc1738.txt) and and RFC 2396 (http://www.ietf.org/rfc/rfc2396.txt). On Linux systems you can see my summary about URIs in the uri(7) manual page. Basically, URIs have reserved and unreserved characters. The reserved characters are:
If it's already a URL, the '#' needs to go through unimpeded so it can identify fragments, and '%' needs to go through so users can escape variables. Also, if it's already a URL, then the '%' has a special meaning. We want users to be able to send their own %hh values in the URL, so we'll want '%' to go through unchanged. We also want the reserved characters to go through unchanged, so those characters must (in general) not be URL-escaped. Any other characters should be escaped using %hh, and in fact any unreserved characters can also be escaped this way.
If the name was originally a filename, not a URL, there are additional complications (e.g., the name comes from the file system or a user typing in a "file name"). One complication is that filenames can have characters such as "%" or "#" in them; these characters have special meaning in URLs. Indeed, filenames can have all sorts of "nasty" characters, including space, newline, and so on. Filenames should not have ASCII 0.
One concern is characters with their high-bit set (international characters). Many systems will probably pass through the high-bit characters fine. I do have some concerns that some programs may have problems with them. For example, I believe some old shells aren't 8-bit clean, which might possibly be exploitable. Probably the correct solution here is to upgrade the shell, and disable the convention until that system has upgraded. Such shells are a hazard anyway. Besides, increasing numbers of users use languages other than English, and those languages generally require such characters anyway. However, this still makes me (and others) concerned.
Since browsers have to deal with all sorts of malformed URI/URLs, I decided that it's the browser's duty to protect itself from bad characters, and I didn't make URL escaping mandatory. However, I encouraged escaping illegal characters, since perhaps by escaping them, the intended URI/URL can be viewed. This escaping would make it possible for someone to insert an overlong UTF-8 sequence and ASCII 0, but they could use %hh to do this anyway; again, browsers have to protect themselves against this. This convention can only safely deliver information, and let users determine what programs can be trusted to safely process the information.
I then re-examined my book's section on "Limit Call-outs to Valid Values", which lists the known shell metacharacters, to handle the shell. You can see that at http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/limit-call-outs.html As noted there, they are:
I also looked at browser implementations. It particular, I looked up information on Netscape's command line options to see what happens to them (see http://developer.netscape.com/docs/manuals/deploymt/options.htm, and especially http://home.netscape.com/newsref/std/x-remote.html which discusses remote control). It turns out that Netscape's ``-remote'' command invokes Xt. It uses XChangeProperty and the X protocol to send information to Netscape; the data is an Xt action to invoke, with optional arguments. Typical use would invoke "openURL(url,new-window)". Thus, "," and ")" have special effects here, since they are the designated separator and terminator of the function calls. The "," merely allows additional parameters to be added, and the openURL() call's extra parameter only permits a window name. I haven't found a useful attack from this end; experimentation with Netscape 4.7 got nowhere, and looking at Mozilla's source code shows that "new-window" is now a special value (so at best an attacker could cause a new window to be created and a different location to be viewed). The Mozilla code is the basis of Netscape 6, and the code for remote execution (at least in February 2001) was in: nsGtkMozRemoteHelper.cpp. The character ")" closes a function, but the Mozilla source code shows that all characters after the ")" character are truncated, so there's no way to supply an "additional" command to execute and use as an exploit. I don't have Netscape 4.7's source code, but experimentation suggests that it does the same thing. Note that these characters can be escaped using URL escaping, but the comma (,) is a reserved character in URLs and in theory isn't supposed to be escaped. I've found no general solution for this - this convention just can't pass unmolested commas. If necessary, URL-escaping ")" and "," could be done in the BROWSER command without changing any applications.
I looked at GNOME's "gnome-help-caller" code in help-caller.c. Its calling conventions mean it should be called using "echo %s | gnome-help-caller", but that doesn't work. Also, it calls help_browser_simple_browser_fetch_url or help_browser_simple_browser_show_url, which aren't the ``usual'' redirectable calls to view a URL. Invoking gnome-help-browser directly (from the command line) works, though it doesn't get redirected through GNOME's Virtual File System.
KDE's "kdehelp" doesn't properly handle URL escapes, but it's really an error in its implementation. As long as there's a way to work around it, it'd be better to design the convention so that once kdehelp is fixed programs can simply work with it.
All of the browsers handle references beginning with "/" as an absolute pathname (it's not a legal absolute URL anyway) when passed as a "URL". They vary in how they handle the characters "%" and "#"; Netscape and Mozilla treat them specially (as in a URL), while the other browsers don't treat them specially at all.
Originally, I proposed maximally escaping characters, even ones that weren't required to be escaped (e.g., ")"), but it was noted that this wasn't really necessary for security. The real issue was escaping characters special to the shell if the shell is to execute the command. And, since I have to ``shell escape'' (prepend a backslash) to some characters it's going through the shell, I may as well do it for all, and leave the data as minimally mangled as possible. By doing the "shell escape" just before calling the shell (after all other processing), and using all shell metacharacters, in theory I should cover them all. The only character not covered is ASCII 0, which I can simply require to be forbidded in the specification: it's not permitted in either format.
One nice thing about not calling the shell is that this approach would be more portable to other systems (e.g., Windows). This is because the URL escaping approaches are independent of the operating system, but the shell escaping process is obviously dependent on the command shell used.
On the downside, this change wouldn't be "backward compatible" with Raymond's original proposal. It also eliminates the benefits of having a shell, namely, small scripts and variable substitutions can be embedded directly into the BROWSER variable. You'd also have to parse the text out anyway (separating parameters by whitespace), which isn't hard but is tedious in some languages such as C, making the process at best no simpler.
I've currently decided to leave the shell request in the convention, but actually I'd prefer it were removed.
If there's a zero-length field in the BROWSER variable, it should be ignored. This is an easy error to make (":netscape:lynx" has three values, the first of which is empty). However, by the rules above, this would get translated into " %s", resulting in an attempt to run the URL. An attacker might try to create hypertext links with nasty names ("halt" comes to mind).
Frankly, the idea of quietly adding a "%s" where it wasn't supposed to be is annoying. Why not just require "%s" if you want it, rather than quietly adding it? However, I can't think of a strong reason to be incompatible with the original specification, so I've left it as-is. If you want to throw away the %s, you can always write a command that throws it away, e.g., "/bin/true %s; real-command".
In general, don't implement the "BROWSER" variable in a program that may be used as setuid/setgid; the BROWSER variable (and other environment variables) in such a case are coming from an untrusted user. If you really must support this kind of functionality in your setuid/setgid program, you should extract just the environment values you need, completely erase your environment, and repopulate your environment using only (1) selected and checked values from the user, and/or (2) values from some configuration file not modifiable by untrusted users. Don't just reload all variables a user sends you; specifically select and filter the ones you'll allow.
The approach described here at least eliminates the threat of someone creating a link that forces the user to run an arbitrary local command, so we've significantly reduced the risk.
A user could create a URL with illegal characters; the specification doesn't prohibit it (though it suggests escaping it). As long as the shell characters are escaped, I haven't found a security problem with this.
A user could also create an overly long reference, or a URI that is syntactically invalid. It's assumed that the shell and brower can handle these; a browser with a buffer overflow possibility on its URI input routines is badly off anyway.
There's an ambiguity in the spec: in a filename, what do "%" and "#" mean? Different browsers treat them differently. I have proposed a particular interpretation in the specification. I haven't found an attack based on this.
As noted before, this convention may cause problems with shells that aren't 8-bit clean. Upgrade your shell! Such a shell probably can't handle international filenames correctly anyway.
Essentially I decided that the key is simply to ``shell escape'' all shell metacharacters and standard separators. Also, since it's known that some shells will do bad things with characters above 127, they need to be URL-escaped, so we may as well URL-escape all characters that can't be legal in URIs. It's assumed that any browsers will not have a security problem caused by any absolute reference given to them (including buffer overflows, illegal characters, %00 for ASCII 0, etc.).
For the actual algorithm being proposed, see the secure BROWSER specification.
In some future version of this document I may try to demonstrate more rigorously that the algorithm correctly handles all cases. However, at least this document captures some of the constraints the algorithm has to deal with, hopefully in enough detail to show that it's effective.
There are several other alternatives which aren't compatible with the original BROWSER convention proposal, since they eliminate the shell. This would be better for security -- calling through the shell is tricky. Also, this would permit programs to pass local filenames unmolested - it's quite silly to have to add "file:", URL-escape, then shell-escape a filename, all of which have to be undone by the receiving program.
The approach I like the most lists the commands, colon-separated, in BROWSER, and the commands can have options (space-separated); the URL is always added at the end as an additional parameter, and the sequence is executed directly (no call through the shell). Thus, "BROWSER=lynx:call_netscape:call_netscape -r" would work. The advantage here is that there's no attempt to call through the shell, nor any complexity of dealing with "%s". Since correctly handling Netscape and Mozilla requires a program anyway (to handle "," and ")"), it's not really a big deal.
An even simpler approach would be to not permit options, or even only list a single program name. I believe this is too inflexible - allowing a colon-separated list of commands, and permitting options, does improve flexibility.
Another approach would be passing the URL in via standard input or an environment variable. This has some advantages, but frankly almost no browsers work this way now, so this creates extra complexity that doesn't seem worth it.