Safely Creating Temporary Files in Shell Scripts
0. Abstract
This paper discusses how a programmer can write shell scripts that securely create temporary files in world/group writable directories. After explaining why it is important to be careful with temporary files I give some hints on how to identify and fix vulnerable shell scripts. This paper concentrates on how things are done. I intentionally leave out lots of gory details in order to make this document shorter and easier to understand for people that just want to write secure code with as little extra effort as possible.1. Introduction
1.1 Why I wrote this
There are actually many documents and papers out there that describe how to securely create temporary files, so why yet another one? Unfortunately, most of these documents refer mainly to C programs even though many temporary files are created by shell scripts. Such shell scripts are wrappers for other programs or configuration scripts. They are inside Makefiles, they are install scripts included in RPM packages or just small tools written by the admin. Due to the nature of these scripts they very often handle temporary files. In contrast to C/C++ there is no standard mechanism for creating temporary files. In order to compensate for this, programmers have to reinvent the wheel time and time again which is very error prone. As a result many scripts handle temporary files insecurely, creating a security risk that is often underestimated.
1.2 Symlink Exploits: The Basic Idea
Programmers often do not think that creating temporary files in a secure way is important. After all it is just a tiny shell script. I will give a very brief introduction of how this can be exploited by an attacker in order to underline the importance of secure tempfile handling:
The basic idea of a symlink exploit is to predict where an application will create its temporary file and put a symbolic link (symlink) at that place. When the application now tries to work with a tempfile it will actually work with the file that the symlink points to. The most simplistic thing a (local) attacker can do with this is to have the symlink point to important files such as /bin/bash. This will cause the program (assuming it has root privileges) to overwrite /bin/bash with its temporary data, making the system unusable.
It is also possible to use this type of exploit to gain complete root privileges. If an attacker is somehow able to influence what data the insecure program writes to the tempfile (now /bin/bash) he can as a result modify the contents of /bin/bash or any other executable on the system. Obviously he will then get root privileges the next time the modified executable is run by root. There are more advanced concepts for exploiting insecure tempfile creation (not all of them involve symlinks), but these simple examples should demonstrate that insecure file creation may cause a lot of trouble.
2. Common mistakes
2.1 Predictable filename
In order to make it easier for you to identify vulnerable code I will show you some examples of how you should NOT create your temporary files.
Very often shell scripts create their temporary files without any checks at all:
sort /some/file > /tmp/tempfile
This will make an exploit trivial: Just create a symlink called /tmp/tempfile and let it point to a file you want to overwrite.
Other vulnerable programs will create temporary files with a name like /tmp/tempfile.$$. The $$ variable contains the process ID (PID) of the shell running the shell script. This is very useful if you have several shell scripts running in parallel: Each one will have its unique tempfile name (because it has a unique PID) so there will be no conflicts. Unfortunately this does not improve security. An attacker knows that the PID of your program will most likely be between 1 and 33000. So the attacker will create 33000 symlinks, knowing that one of them will work.
2.2 Race conditions
The obvious solution to the problems from 2.1 seems to be this:
rm -rf /tmp/tempfile.$$
sort /some/file > /tmp/tempfile.$$
"rm" will delete the symlink itself (not the file that the symlink points to), so that the "sort" command will be safe. It may sound good, yet the script is still insecure because now you have a race condition. Once the symlink gets deleted the attacker will know that "rm" was successfully run by the victim. Now he re-creates the symlink in the hope that the sort command has not yet created the tempfile. The attacker cannot be 100% sure that his exploit will be successful, but he can keep trying until it works.
2.3 Identifying vulnerable code
There are many variations of the above examples so one could think this kind of vulnerability is hard to find. Actually, it is much easier to detect than buffer overflows or format string vulnerabilities. Just do a
grep -RH /tmp *
if you suspect that your package contains vulnerable code. If you feel really bored some day just try and scan your entire system, you might be surprised! Check if the files found by grep handle their tempfiles securely. If they do not, the next chapter will tell you how to fix this.
3. Writing secure code
3.1 Don't use tempfiles
This may sound a little stupid at first, yet there is some truth to it: If you don't do anything you can't mess up anything. Sometimes programs use tempfiles even though it is not necessary. Most commands can write their results to standard output instead of a file. Assuming that you have just a few bytes or kilobytes of temporary data it could actually be much more convenient for you as a programmer to store it in a variable rather than a file. This way you can sometimes avoid any risk of insecurely creating tempfiles simply by not creating them at all.
3.2 Use mktemp
If you have to use tempfiles then mktemp could be what you are looking for. Realizing that the creation of tempfiles in shell scripts is a security problem, Todd C. Miller wrote mktemp, a tool that will do this for you. The man page gives you some good examples of how to use it securely, so I am not going to explain that. If you write shell scripts for an environment where mktemp is available you should definitely use it to create your temporary files.
Just one little problem: Not all Linux distributions come with mktemp in their default install, so you cannot rely on mktemp to be available. Other Unixes (e.g. Solaris) don't necessarily have it either. If you want your script to be able to run on a wide variety of systems you will have to get along without mktemp. If maximum compatibility is not that much of an issue for you, you can just tell your users to install mktemp before they use your software. Mktemp will compile almost everywhere, so that should not be a big issue for most people.
3.3 Notes on mktemp
What you should know is that Suse Linux comes with a rather old version of mktemp. This older version of mktemp does not understand all of the command line options that are described on the mktemp website. It only understands the -d -q and -u options. If you try to use some of the newer options mktemp will simply exit and signal an error.
Similarly, the mktemp that comes with Mac OS X and FreeBSD 4.9/5.2 only supports -d -q -u and -t. Mktemp on Debian supports -d -q -u -t -p but not the -V option. So you should be careful when you use the new -V -p and -t options. It is probably better to stick to -d -q and -u for now to ensure portability. Optionally you may ask you distributor of choice to update his mktemp package.
3.4 Creating tempdirs with mkdir
Ok, so you are writing scripts for systems that may not have mktemp. Let's look for something to replace it that is available on all platforms. The mkdir command will do the job. "mkdir /tmp/tempdir.$$" will create the directory /tmp/tempdir.$$ and signal an error if it already existed. This is similar to what "mktemp -d" would do. But mkdir lacks some of the luxury that mktemp provides, so the code gets a little more complicated:
(umask 077 && mkdir /tmp/tempdir.$$) || exit 1
Explanation: The umask command ensures that when mkdir creates the directory it will have an access mode of 700. Running chmod 700 afterwards would be a bit less secure. Because the umask is in parentheses it will not affect the umask for anything other than the mkdir command. If mkdir signals an error for whatever reason, the program exits. This approach is used by many shell scripts and is secure against symlink exploits. The code should work on every decent operating system that deserves to be called Unix-like.
Now that you have created yourself a secure temporary directory you can create new files/directories/whatever inside this tempdir without having to worry about security: You own the directory, you are the only one that has access to it. This makes it easy to fix existing code. All you need to do is add the code for the secure tempdir creation, adjust a few pathnames and you are finished! Just don't forget to clean up after yourself and delete the tempdir with "rm -rf" when you are finished with it.
3.5 Mkdir deluxe
While the above code is pretty good there is still room for improvements. First you should find out if the admin really wants you to use /tmp/ for your temporary data. For this purpose there is a shell variable called TMPDIR. If TMPDIR is set you should use this directory to create your temporary files. That way the admin can for example improve security by assigning each user a separate (=safe) directory for temporary data.
Using $$ is a convenient way of creating unique names for tempfiles. And even though the filename can be predicted the directory is still created in a secure way by the mkdir command. The question that remains is: Secure against what? Well, it is secure against symlink exploits and the like. But it is not secure against a DoS attack. An attacker simply keeps your application from creating its temporary directory by imitating (!) a symlink attack. That way your program always exits and never completely does what it is supposed to do. Depending on the program, this alone can be a security threat.
This problem can be avoided by appending $RANDOM to the filename. This variable contains a random integer between 0 and 32767. In order to get filenames that are as unpredictable as the ones created by mktemp you will need to append $RANDOM to your filename three times for a total of 35184372088832 (=2^45) different possible combinations. So here comes the code to create a temporary directory:
tmp=${TMPDIR-/tmp}
tmp=$tmp/somedir.$RANDOM.$RANDOM.$RANDOM.$$
(umask 077 && mkdir $tmp) || {
echo "Could not create temporary directory! Exiting." 1>&2
exit 1
}
Explanation:
 Line one will set tmp to $TMPDIR if $TMPDIR is defined. If $TMPDIR is undefined "/tmp" is used as a default. Line 2 builds the actual directory name using $RANDOM three times. It is important that you put some kind of separator like the "." between the numbers. Without the "." there would be no difference between the random numbers 12, 34 and 56 (=123456) and the random numbers 123, 45 and 6 (=123456). As a result the number of different combinations would decrease considerably, thereby making your directory name more predictable. You should replace "somedir" with a more meaningful name in your own script. If anything goes wrong, an error message will be printed (to stderr, where it belongs) and the program exits. If everything is ok, then $tmp will contain the name of your temporary directory.
The only trouble you may have with this is $RANDOM. This variable is a feature of the Bourne Again Shell (bash) and not included in the normal Bourne Shell (sh). But on every Linux I have seen (and on MacOS X) both "sh" and "bash" are just the good old GNU bash that will happily provide you with $RANDOM. On other systems it may not be available.
But even then there is no reason to worry. In those cases $RANDOM will simply be an uninitialized variable that defaults to "". Hence your directory will be "/tmp/somedir....123" if your PID is 123. This is also the reason why I append the $$ to the directory name: To make sure I get unique directory names even if $RANDOM is not supported. On systems that don't provide $RANDOM this program will act just like the one given in chapter 3.4.
3.6 Mixing mktemp and mkdir
Many shell scripts try to use mktemp to create their tempdirs. If mktemp is not available they will resort to using mkdir. At first glance this makes perfect sense: try to use the secure mktemp and only use mkdir if you have to. But once you think about it you will realize that this is rather illogical. Because for you as a programmer there are two possible scenarios that you have to take into account when writing a shell script:
1. mktemp is always available
In this case there is no point in including the code that uses mkdir. It will never be used and is therefore dead code that should be removed.
2. mktemp may not be available.
That means now you have to include the mkdir code. Of course this code is secure enough for your purpose. If it is not, then you got a vulnerability that you should fix right now! So if you already have code that uses mkdir and is sufficiently secure, why would you want to use mktemp? Your program works perfectly fine without it and you get no security benefit from using it. Obviously, mktemp is unnecessary in this scenario and the code using it should be removed.
There may be exceptions to this, but 99% of the time you should use either mktemp or mkdir but NOT both.
If you really do need to mix mktemp and mkdir (which is unlikely) make sure you don't repeat a very common mistake:
#try mktemp
tempfile=`mktemp /tmp/secureXXXXXXXX`
#if mktemp didn't work, use insecure name
test -n $tempfile && $tempfile=/tmp/insecure$$
do_whatever > $tempfile
You would expect this code to be secure on all systems that provide mktemp. Wrong! This one is vulnerable on every system. An attacker will prepare a symlink attack against the /tmp/insecure$$ file name. Then he will fill up the /tmp partition with data. As /tmp is full, mktemp will fail and the insecure file name will be used. You may now object that there are quotas and the like. But you as a programmer cannot expect everybody to set up quotas for /tmp. And even if there is a quota an intelligent attacker can still flood /tmp in most cases. A solution could look like this:
#look for mktemp
mymktemp=`which mktemp`
test -n $mymktemp && {
#mktemp available
}
test -z $mymktemp && {
#no mktemp
}
This code checks if mktemp can be found and reacts accordingly. But as stated above, you should hardly ever need to do this.
