Eric Raymond has proposed the BROWSER convention for Unix-like systems, which lets users specify their browser preferences and lets developers easily invoke those browsers. In general, this is a great idea. Unfortunately, as specified it has horrendous security flaws; documents containing hypertext links like "; /bin/rm -fr ~" will erase all of a user's files when the user selects it! Also, it's ambiguously specified, it doesn't permit a colon (:) in commands, and it doesn't discuss some implementation issues (especially if filenames are allowed and how to handle non-absolute references).
This document defines the ``secure BROWSER'' (SB) convention, which is backwards-compatible with the original BROWSER specification, fixes these problems, and is still easy to implement. It also provides a simple example, implementation notes, a detailed discussion of how to implement the convention, a discussion on how to convert non-absolute references into absolute references, discussion of how this could be expanded to serve non-Unix-like systems, and "tricks" that users can use to perform all sorts of tasks using the Secure BROWSER convention. Technically URLs are a subset of URIs, so in this document the term "URI" is used instead of URL (with occasional references to URLs).
This is a DRAFT specification. Comments welcome, please email dwheeler@dwheeler.com (no spam please). This specification and related information (include a security analysis) are available at http://www.dwheeler.com/browse.
In particular, a major debate is whether it's worth calling the shell. The shell call may be removed as being too dangerous. Options include: (1) BROWSER only having a list of program names, (2) BROWSER listing programs with constant arguments (no need for %s), and (3) Using "%" replacements but avoiding the shell. The "%" replacements are increasingly looking undesirable; they take more work to program, and handling Netscape/Mozilla properly requires writing a short program anyway (so they don't seem to be helpful). Below are "compatible" and "alternative" definitions - the "compatible" one is compatible with the original definition, but it's complicated to implement; the "alternative" one is simpler, but requires that the "BROWSER" variable have a different format. Comments welcome.
An implementation of this convention must convert any reference to an ``escaped absolute reference''. An ``absolute reference'' is either an absolute local pathname (a filename beginning with "/" and not containing any high-bit characters or control characters) or an absolute URI/URL (which begins with lower case letters followed by a ":"). In an absolute URI/URL, all characters except for the following must be URL-escaped (replaced with %hh, where hh is its hexadecimal value):
An absolute reference must not include the NIL character (0). Any other kind of reference must be either rejected or converted into an absolute reference before executing the command parts. For example, filenames with high-bit characters must be converted to the "file:" URI. This absolute reference is then converted into an ``escaped absolute reference'' by inserting a backslash before each of the following characters (to prevent certain shell-based security attacks):
The BROWSER environment variable contains a colon-separated series of browser command parts. These command parts are tried in order, executing each directly (e.g., using execvp(3), and not using system(3) or /bin/sh), until one succeeds (returns 0); empty command parts (e.g., "::") are ignored. A command part begins with the filename of the command to run, followed by 0 or more parameters (separated by one or more spaces). After the command part, the absolute reference (e.g., URI) to be browsed is added.
An implementation of this convention must convert any reference to an ``absolute reference''. An ``absolute reference'' is either an absolute local pathname (a filename beginning with "/") or an absolute URI/URL (which begins with lower case letters followed by a ":"). An absolute reference must not include the NIL character (0). An implementation may choose to URL-escape any characters not legal in URLs, but it doesn't have to do so. Any other kind of reference must be either rejected or converted into an absolute reference before executing the command parts.
For example, users can set their BROWSER variable by running the following command in a Bourne-like shell:
Note that the "%s" was changed to the escaped absolute reference in the first case, and that an implied " %s" was added in the second case. Note that the "?" is preceded with a backslash when handed to the shell, to prevent it from being (mis)interpreted. The shell will remove the backslash and hand the final browser the intended value.
Before discussing implementation details, a few notes on how to implement the EB convention may help:
If you're developing for a desktop environment that already has calls to invoke a browser, just use those calls and don't try to implement this convention directly; the desktop environment configuration should then determine if and how the BROWSER convention should be used. Where possible, make the library implementors do the work! Thus, GNOME developers should continue to call gnome_url_show() in libgnome. However, Gtk+ users who don't use GNOME libraries will want to use this BROWSER convention directly, since Gtk+ by itself doesn't have a standard mechanism for invoking browsers.
In general, don't implement this convention in a program that may be setuid/setgid; the BROWSER variable (and other environment variables) in such a case are coming from an untrusted user. If you really must support this kind of functionality in your setuid/setgid program, you should use normal security practices. In particular, extract just the environment values you need, completely erase your environment, and repopulate your environment using only (1) selected and checked values from the user, and/or (2) values from some configuration file not modifiable by untrusted users. Don't just reload all variables a user sends you, such as BROWSER; specifically select and filter the ones you'll allow.
This approach does assume that your shell won't have security problems with characters whose decimal value is greater than 127 (i.e., it's "8-bit clean"). If your /bin/sh implementation can't handle such characters, it's time to upgrade your shell. Alternatively, it may be that your obsolete shell won't be a security risk, but you'll need to do that analysis. As a temporary measure, you could disable this convention for any programs that take data from untrusted sources using:
Desktop environments should then set up their initialization routines so that, when the desktop environment is started up, the desktop environment sets BROWSER to the name of their helper program. This way, programs that aren't specifically designed for that desktop environment will nevertheless use that environment's browsing system whenever they want to view a URI. Note that this means that users can log into different desktop environments at different times and get the "correct" results each time. Desktop environment variables should allow users to select a non-default BROWSER value (e.g., if they want to use KDE's help system inside GNOME).
Desktop environment developers who have their own library conventions for invoking browsers should modify their libraries so they can optionally support the BROWSER convention as well. GNOME, for example, has such a convention: gnome_url_show(). Desktop environments should allow users to determine if they want to use the BROWSER convention or not as part of their internal convention, and if so, if it will be tried before or after the "standard" approach in the desktop environment. Desktop environments could make this setting controllable for each scheme (with a default), they could have a separate setting for "unknown" schemes, or they could simply have those two settings globally apply. By making it possible to use the BROWSER scheme, users can use BROWSER for actions such as URI filtering even if they normally control browser selection through their desktop environment.
Fundamentally, each application takes the data it receives and transforms it into the correct absolute URI or absolute pathname (using information it has, such as the correct current directory). This is then used to call the browser.
Rationale: We need to be able to specify multiple browser commands so programs obeying this convention can do the right thing in either X or console environments, trying X first. Specifying multiple commands may also be useful for people who share files like .profile across multiple systems. Skipping empty strings avoids security problems - otherwise it would be converted into trying to run the reference. If the entire series is an empty string, then by these definitions any attempt fails; this is a convenient way to disable BROWSER functionality. By running commands through the shell, shell capabilities such as variable substitution can be used; there are disadvantages to calling the shell too (anyone failing to filter their text will be a security liability).
Rationale: We need %s because some popular browsers have remote-invocation syntax that requires the URL to be somewhere other than the end. Unless %% reduces to %, it won't be possible to have a literal percent sign in the string. Unless %c reduces to :, it won't be possible to have a literal colon. I considered adding support for additional conventions (some inspired by Netscape), such as %h for a hostname, %p for a portname, %b for a base URI, %r for the relative URI, %f for a filename, %S for the scheme, and so on. For example, by default Netscape's telnet protocol is "xterm -e telnet %h %p". However, requiring all programs to implement these is a lot more work; the same effect can be accomplished by creating separate programs that do the job. Be careful: "Hello %%sister" becomes "Hello %sister" and not "Hello %URI".
Raymond's original proposal doesn't explicitly say if the shell should be called, but the sample patches I looked at did so, so I'm assuming that calling the shell with each fragment is the intended result. In some senses it'd be safer if the shell wasn't called, but you would still have to determine where the parameter breaks occurred. Not calling the shell would make the process more portable. On the other hand, calling the shell makes the BROWSER command fragments more capable; they can include whole commands and meta-variables where useful. Also, the BROWSER variable value is system-unique anyway, so even if it's implemented through a shell on a Unix-like system and not on an MS-DOS system, it won't matter since you can't exchange their BROWSER values anyway. I've decided that backwards-compatibility is desirable unless there's a clear reason not to be, so I've left the shell call in.
It's not well-documented, but web browsers for Unix-like systems typically handle anything beginning with "/" as a filename, and anything beginning with "[a-z]+:" as a URI. In this document I've defined the union of these two possibilities as an "absolute reference". I've tested this and found that the following browsers all do this: lynx 2.8.4, Netscape 4.7 (using the openURL() remote invocation call), Mozilla 0.7 (also using openURL()), links 0.92 (when the reference is provided as a parameter), GNOME's gnome-help-browser (when the reference is provided as a parameter), and KDE's help browser (when invoked using "kfmclient openURL"). This is very reasonable behavior, and essentially universal, so this secure BROWSER convention takes advantage of it. After all, if there's no "base document" it's illegal for a URI to begin with a "/", so determining that it's a filename is unambiguous. If a browser/viewer can't handle it, you can write a wrapper to return "fail" for any reference format that particular viewer can't handle.
There is an ambiguity/difference of opinion by browsers of how to handle some special characters in absolute pathnames. In lynx, kdehelp, and gnome-help-browser, '%' and '#' are not considered special characters in absolute pathnames. Conversely, in Netscape and Mozilla, '%' and '#' in absolute pathnames are considered special characters with their usual URL meanings; they also can't handle the characters "," and ")" using their remote invocation syntax unless they're URL-escaped. Thus, it's best to avoid these characters in filenames. As far as I can tell, this ambiguity is not a security problem; these characters are interpreted by the browser itself and the browser has to protect itself from bad characters and syntax anyway.
This convention might, in the future, be modified to define one interpretation ("are any characters special in pathnames?") as the "correct" one, and browsers using a "different" interpretation will need to be wrapped. For now, I would suggest that if an absolute reference begins with "/", then no characters (including '%' and '#') are special - if a browser assumes otherwise, then as part of the browser call (in the BROWSER command) those characters should be escaped. Thus, a system to make sure that these special characters could be passed to Netscape could be entirely placed in the BROWSER command, doing something like this: BROWSER='/usr/bin/invoke_netscape %s:/usr/bin/lynx'
Then, create the program invoke_netscape with the following contents:
Here's an algorithm to convert other URI/URLs into absolute URI/URLs. First, determine the "base document" (e.g., the starting document's absolute address) if any. In HTML files a base document value may be specified; you can learn more about base documents from the HTML 4 specification section on bases. Then do the following:
| URI Type | Reference's Prefix | Computed Prefix (non-empty base) | Computed Prefix (empty base) |
|---|---|---|---|
| Network path | // | {base-scheme}: | file: |
| Absolute path | / | {base-scheme}://{base-host} | file: |
| Relative path, fragment only | # (or empty) | {base} | file:{current directory/} |
| Relative path, current dir | ./ | {base}/../ | file:{current directory/} |
| Normal relative path | (default) | {base}/../ |
http:// or ftp:// or file:{current directory/} |
In this table, {base-scheme} means the scheme of the base document (e.g., "http"), {base-host/} means the hostname of the base document including any usernames and passwords and ending in a slash (e.g., "www.yahoo.com"), and {current-directory/} means the current directory, ending in a slash (thus, if the current directory isn't "/", you'll need to append a slash).
Note that if there's no base, a normal relative path is completely ambiguous (does "xyz.com" mean "http://xyz.com", or file "./xyz.com"?). It's reasonable to reject these, but you could try instead to intuit the intent. Perhaps you could prefix "ftp://" if the reference starts with "ftp.". If you think the user meant a web address, prefix it with "http://", otherwise prefix using the file prefix as shown.
This is easier to show by example, so here are some examples. In the table below, the first column shows the different types of URIs, the same list as the previous table. The second column shows an example of that type of URI. The third column shows that example transformed into an absolute URI if the ``base document'' is http://www.dwheeler.com/browse/index.html. The forth column shows that same example transformed into an absolute URI if there is no base document and your home directory is /home/dwheeler.
| URI Type | Input | Resulting Reference (non-empty base) | Resulting Reference (empty base) |
|---|---|---|---|
| Absolute URI | ftp://www.yahoo. com/r/ww | ftp://www.yahoo. com/r/ww | ftp://www.yahoo. com/r/ww |
| Network path | //www.yahoo. com/r/ww | http://www.yahoo. com/r/ww | file://www.yahoo. com/r/ww |
| Absolute path | /etc/passwd | http://www.dwheeler. com/etc/passwd | file:/etc/passwd |
| Relative path, fragment only | #section3 | http://www.dwheeler. com/browse/index. html#section3 | file:/home/dwheeler/ #section3 |
| Normal relative path, current dir | ./hello | http://www.dwheeler. com/browse/ index.html/../ ./hello | file:/home/ dwheeler/./hello |
| Normal relative path | hello | http://www.dwheeler. com/browse/ index.html/../ hello | http://hello or
ftp://hello or file:/home/ dwheeler/hello |
Note that this entire process as stated so far allows characters that aren't legal in URIs to be passed on. Unfortunately, some old shells have problems with characters beyond 127, and many characters are illegal anyway. To limit URI/URLs to only legal characters, you'll need to URL-escape many characters. A URL-escape simply replaces a character with "%hh", where hh is its hexadecimal value. To eliminate illegal characters in URI/URLs, URL-escape all characters except the following, even if you've been given an absolute URI/URL:
The key requirement for implementing this convention is that there be (1) some way to get the BROWSER value, (2) some way to safely invoke a program with arguments (which may or may not involve escape characters), and (3) some way for browsers to differentiate between ``absolute pathnames'' and ``absolute URIs'' when given an absolute reference. Note that when detecting if a reference is a pathname, a program should use the operating system's conventions and not a "portable" set; a program on a Unix-like system could not use "C:\hello" as an absolute pathname.
The next two subsections show how this convention could be modified to suport MS-DOS/Windows and MacOS - showing that this convention is even reasonable to support in portable libraries. This doesn't mean you should - many OS's have other ways of doing this.
It would certainly be possible to extend the approach to MS-DOS/Windows. MS-DOS/Windows has environment variables, so it's easy to get BROWSER. In MS-DOS/Windows it would probably be better to not call through a shell, but instead not do any escaping and invoke the call directly. It would be possible to call through a ``shell'', but the list of ``shell escape characters'' is different and the name of the standard shell has changed over time.
The key difference is that in MS-DOS/Windows, absolute pathnames can look like "C:\hello", that is, they can start with a single letter (lower or upper case) followed by a colon, and backslashes as well as forward slashes can be used as directory separators. Luckily, absolute pathnames and absolute URIs can still be disambiguated, because there are no single-letter URI schemes; thus, a full pathname begins with "/", "\", or "[a-zA-Z]:". Accepting backslashes ("\") is easy, although when converting filenames to URIs they must be converted to "/". Also, in a URI, the ":" following a drive letter (e.g., "C:") becomes a vertical bar ("|").
MS-DOS/Windows also supports some ``device names'' which have multiple letters and end in ":", such as "CON:". These can be safely ignored; it's hard to imagine when these device names should be accessible through this convention. If necessary, they could be supported through the URI convention, e.g., "file://CON:". An alternative would be declaring that anything with one or more alphanumeric characters and ending in a colon is a device name. Thus, "^[A-Za-z0-9]+:$" would be a device name and accepted as such. This is because real URIs have at least one character after the colon.
Usually lists in Windows/MS-DOS are separated by ";" not ":"; since other lists often do this, most libraries can just reuse their mechanisms to implement this.
In the Windows world you're more likely to invoke a browser via COM or DCOM. However, it'd be possible to modify the convention so that this would work.
Although more difficult (and requiring assumptions about the system volume names), it appears that this convention can also be modified so it can be implemented on MacOS. MacOS typically doesn't use environment variables, but that could be "faked". Also, programs on MacOS can clearly execute other programs. However, file naming is more complex.
The MacOS typically uses the colon ":" as the directory separator, and under MacOS a ``full pathname'' begins with a volume name or volume reference number. The volume reference number changes each time a volume is mounted, so this isn't as useful ("0" is the default volume). Unfortunately, on the surface this naming scheme is ambiguous - letters followed by a colon could be either a URI or a full pathname, making it difficult to rationally use this convention! For example, does "http:hello" mean a URI with the http protocol, or a full pathname to the http volume? However, I understand that file names under MacOS are case-sensitive. Moreover, volume names are very often given names that begin with an uppercase letter and/or include characters other than lower case letters.
Thus, a convention could be required on MacOS systems: ``all volume names must include one non-lower-case letter''. If this convention were enforced, then any reference matching the pattern "^[a-z]+:" would be URI. An (illegal) partial pathname would match "^:", and a full pathname would be any other pattern with a colon inside it. A reference without a colon anywhere is also a partial pathname, and thus it's illegal. Officially, a partial pathname doesn't need to start with ":"; this is outrageously ambiguous though, and need not be supported.
Many URIs follow the first colon with a "/" (though not all do!); if necessary, this information might be useful to disambiguate pathnames and URIs. This is more complex and less certain, however, so I would prefer to avoid that approach.
If you're curious, here's the information about MacOS that I used to derive this approach. The following is information from ``Inside Macintosh, Volume VI'' by Apple Computer, 1991, published by Addison-Wesley, in the section ``Identifying files, directories, and volumes'':
A file specification (in HFS) is specified as a:Programs can set the "vRefNum" (volume reference number) and/or "dirID" (directory ID) in the MacOS interface, but thankfully these parameters are ignored when a full pathname is given."A full pathname consists of the name of the volume, the names of all directories between the root directory and the target, and the name of the target. A full pathname starts with a character other than a colon and contains at least one colon. If the first character is a colon, or if the pathname contains no colons, it is a partial pathname. If a partial pathname starts with the name of a parent directory, the first character in the pathname must be a colon. If a partial pathname contains only the name of the target file or directory, the leading colon is optional."
- full pathname,
- volume reference number and partial pathname,
- working directory reference number and partial pathname, or
- a volume reference number, directory ID, and partial pathname.
If you want to see the URL without the backslashes, do this:
This specification is (C) 2001 David A. Wheeler, all rights reserved. You may copy it unchanged. You can see more information at http://www.dwheeler.com/browse.