This is sfs.info, produced by makeinfo version 4.2 from sfs.texi. START-INFO-DIR-ENTRY * SFS: (sfs). Self-certifying file system END-INFO-DIR-ENTRY This file documents SFS, the self-certifying file system. Copyright 1999-2002 David Mazie`res Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the Free Software Foundation.  File: sfs.info, Node: vidb, Next: funmount, Prev: sfssd, Up: Command reference `vidb' command ============== `vidb' manually edits an SFS user-authentication file *note sfs_users::, acquiring locks to prevent concurrent updates from overwriting each other. The usage is: vidb [-r] [-w] [-e EDITOR] SFS-USERS-FILE `vidb' has the following options: `-r' Recovers from a previous edit session. `vidb' makes a copy of of SFS-USERS-FILE to be edited, named by appending `.tmp' to the file name. If an editing session crashes, vidb will refuse to run, as the old temporarly file may contain useful data that should not be deleted. With `-r', however, `vidb' removes any old temporary file. `-w' The point of `vidb' is to avoid concurrent edits to the database and the corresponding inconsistencies that might result. Ordinarily, if the database is already being edited, `vidb' will just exit with an error message. The `-w' flag tells `vidb' to wait until it can acquire the lock on the database, and then run. `-e EDITOR' Specifies the editor to use for editing the file. The default is to use the command specified by the enviornment variable `EDITOR'. If there is no environment variable and `-e' is not specified, `vidb' uses `vi'. Note: `vidb' should really recreate any publicly-readable versions of user authentication databases (either by parsing `sfsauthd_config' for `-pub=...' options to `Userfile' directives or signaling `sfsauthd'). Currently you must manually kill `sfssd' or `sfsauthd' for this to happen.  File: sfs.info, Node: funmount, Next: sfsrwsd, Prev: vidb, Up: Command reference `funmount' command ================== The `funmount' command is executed as follows: funmount PATH `funmount' forcibly attempts to unmount the file system mounted on PATH. It is roughly equivalent to running `umount -f PATH'. However, on most operating systems the `umount' command does a great deal more than simply execute the `unmount' system call--for instance it may attempt to read the attributes of the file system being unmounted and/or contact a remote NFS server to notify it of the unmount operation. These extra actions make `umount' hang when a remote NFS server is unavailable or a loopback server has crashed, which in turn causes the client to become ever more wedged. `funmount' can avoid such situations when you are trying to salvage a machine with bad NFS mounts without rebooting it. SFS will get very confused if you ever unmount file systems from beneath it. SFS's `nfsmounter' program tries to clean up the mess if the client software ever crashes. Running `funmount' will generally only make things worse by confusing `nfsmounter'. If `/a' is a mount point, and `/a/b' is another mount point, unmounting `/a' before `/a/b' will cause the latter file system to become "lost." Once a file system is lost, there is no way to unmount it without rebooting. Worse yet, on some operating systems, commands such as `df' may hang because of a lost file system. Many operating systems will not let you unmount a file system (even forcibly) if a process is using the file system's root directory (for instance as a current working directory). Under such circumstances, `funmount' may fail. To unmount the file system you must find and kill whatever process is using the directory. Utilities such as `fstat' and `lsof' may be helpful for identifying processes with a particular file system open.  File: sfs.info, Node: sfsrwsd, Next: sfsauthd, Prev: funmount, Up: Command reference `sfsrwsd' daemon ================ /usr/local/lib/sfs-0.7.2/sfsrwsd [-f CONFIG-FILE] `sfsrwsd' is the program implementing the SFS read-write server. Ordinarily, you should never run `sfsrwsd' directly, but rather have `sfssd' do so. Nonetheless, you must create a configuration file for `sfsrwsd' before running an SFS server. *Note sfsrwsd_config::, for what to put in your `sfsrwsd_config' file. `-f CONFIG-FILE' Specify an alternate `sfsrwsd' configuration file, *Note sfsrwsd_config::. The default, if `-f' is unspecified, is `/etc/sfs/sfsrwsd_config'.  File: sfs.info, Node: sfsauthd, Prev: sfsrwsd, Up: Command reference `sfsauthd' daemon ================= /usr/local/lib/sfs-0.7.2/sfsauthd [-u SOCKFILE] [-f CONFIG-FILE] `sfsauthd' is the program responsible for authenticating users. `sfsrwsd' and other daemons communicate with `sfsauthd', forwarding it authentication requests from `sfsagent' processes on remote client machines. `sfsauthd' informs requesting daemons of whether authentication requests are valid, and if so what local credentials to associate with the remote user agent. The `sfskey' program also communicates directly with remote `sfsauthd' processes when retrieving and updating users keys (with `sfskey add', `update', `register', and more). `-f CONFIG-FILE' Specify an alternate `sfssauthd_config' configuration file, *Note sfsauthd_config::. The default, if `-f' is unspecified, is `/etc/sfs/sfsauthd_config'. `-u PATH' Bind unix domain socket PATH, and accept TCP connections passed over connections to that socket. This option allows `sfssd' to communicate with already running `sfsauthd' commands using a directive like `Service 2 -u PATH' in `sfssd_config' *Note sfssd_config::.  File: sfs.info, Node: Security, Next: Contacts, Prev: Command reference, Up: Top Security considerations *********************** SFS shares files between machines using cryptographically protected communication. As such, SFS can help eliminate security holes associated with insecure network file systems and let users share files where they could not do so before. That said, there will very likely be security holes attackers can exploit because of SFS, that they could not have exploited otherwise. This chapter enumerates some of the security consequences of running SFS. The first section describes vulnerabilities that may result from the very existence of a global file system. The next section lists bugs potentially present in your operating system that may be much easier for attackers to exploit if you run SFS. Finally the last section attempts to point out weak points of the SFS implementation that may lead to vulnerabilities in the SFS software itself. * Menu: * new vulnerabilities:: Vulnerabilities created by SFS * exposed vulnerabilities:: Vulnerabilities exploitable because of SFS * implementation vulnerabilities:: Vulnerabilities in the SFS implementation  File: sfs.info, Node: new vulnerabilities, Next: exposed vulnerabilities, Prev: Security, Up: Security Vulnerabilities created by SFS ============================== Facilitating exploits --------------------- Many security holes can be exploited much more easily if the attacker can create an arbitrary file on your system. As a simple example, if a bug allows attackers to run any program on your machine, SFS allows them to supply the program somewhere under `/sfs'. Moreover, the file can have any numeric user and group (though of course, SFS disables setuid and devices). `.' in `path' ------------- Another potential problem users putting the current working directory `.' in their PATH environment variables. If you are browsing a file system whose owner you do not trust, that owner can run arbitrary code as you by creating programs named things like `ls' in the directories you are browsing. Putting `.' in the PATH has always been a bad idea for security, but a global file system like SFS makes it much worse. symbolic links from untrusted servers ------------------------------------- Users need to be careful about using untrusted file systems as if they were trusted file systems. Any file system can name files in any other file system by symbolic links. Thus, when randomly overwriting files in a file system you do not trust, you can be tricked, by symbolic links, into overwriting files on the local disk or another SFS file system. As an example of a seemingly appealing use of SFS that can cause problems, consider doing a `cvs' checkout from an untrusted CVS repository, so as to peruse someone else's source code. If you run `cvs' on a repository you do not trust, the person hosting the repository could replace the `CVSROOT/history' with a symbolic link to a file on some other file system, and cause you to append garbage to that file. This `cvs' example may or may not be a problem. For instance, if you are about to compile and run the software anyway, you are placing quite a bit of trust in the person running the CVS repository anyway. The important thing to keep in mind is that for most uses of a file system, you are placing some amount of trust in in the file server. *Note resvgids::, to see how users can run multiple agents with the `newaid' command. One way to cut down on trust is to access untrusted file servers under a different agent with different private keys. Nonetheless, this still allows the remote file servers to serve symbolic links to the local file system in unexpected places. Leaking information ------------------- Any user on the Internet can get the attributes of a LOCAL-DIRECTORY listed in an `Export' directive (*note export::). This is so users can run commands like `ls -ld' on a self-certifying pathname in `/sfs', even if they cannot change directory to that pathname or list files under it. If you wish to keep attribute information secret on a LOCAL-DIRECTORY, you will need to export a higher directory. We may later reevaluate this design decision, though allowing such anonymous users to get attributes currently simplifies the client implementation.  File: sfs.info, Node: exposed vulnerabilities, Next: implementation vulnerabilities, Prev: new vulnerabilities, Up: Security Vulnerabilities exploitable because of SFS ========================================== NFS server security ------------------- The SFS read-write server software requires each SFS server to run an NFS server. Running an NFS server at all can constitute a security hole. In order to understand the full implications of running an SFS server, you must also understand NFS security. NFS security relies on the secrecy of file handles. Each file on an exported file system has associated with it an NFS file handle (typically 24 to 32 bytes long). When mounting an NFS file system, the `mount' command on the client machine connects to a program called `mountd' on the server and asks for the file handle of the root of the exported file system. `mountd' enforces access control by refusing to return this file handle to clients not authorized to mount the file system. Once a client has the file handle of a directory on the server, it sends NFS requests directly to the NFS server's kernel. The kernel performs no access control on the request (other than checking that the user the client claims to speak for has permission to perform the requested operation). The expectation is that all clients are trusted to speak for all users, and no machine can obtain a valid NFS file handle without being an authorized NFS client. To prevent attackers from learning NFS file handles when using SFS, SFS encrypts all NFS file handles with a 20-byte key using the Blowfish encryption algorithm. Unfortunately, not all operating systems choose particularly good NFS file handles in the first place. Thus, attackers may be able to guess your file handles anyway. In general, NFS file handles contain the following 32-bit words: * A file system ID (containing the device number) * The inode number (i-number) of the file * A generation number that changes when the i-number is recycled In addition NFS file handles can contain the following words: * A second file system ID word (for a 64-bit fsid) * The length of the file handle data * The i-number of the exported directory * The generation number of the exported directory * Another copy of the file system ID (for the exported directory?) * One or more unused 0 words Many of these words can be guessed outright by attackers without their needing to interact with any piece of software on the NFS server. For instance, the file system ID is often just the device number on which the physical file system resides. The i-number of the root directory in a file system is always 2. The i-number and generation number of the root directory can also be used as the i-number and generation number of the "exported directory". On some operating systems, then, the only hard thing for an attacker to guess is the 32-bit generation number of some directory on the system. Worse yet, the generation numbers are sometimes not chosen with a good random number generator. To minimize the risks of running an NFS server, you might consider taking the following precautions: * Many operating systems ship with a program called `fsirand' that re-randomizes all generation numbers in a file system. Running `fsirand' may result in much better generation numbers than, say, a factory install of an operating system. * In general, you should try to block all external NFS traffic from reaching your machine. If you have a firewall, consider filtering ports 111 and 2049 for both TCP and UDP. If your server's operating system comes with some sort of IP filtering, you might filter any traffic to port 2049 that does not come from the loopback interface (though on some OSes, this could prevent you from acting as an NFS client if you are still using NFS on your local network--try it to see). * Most operating systems allow you to export NFS file systems "read-mostly"--i.e. read-write to a small number of servers and read-only to everyone else. The read-only requirement typically is enforced by the kernel. Thus, if you can export file systems read-write to `localhost' for SFS, but read-only to any client on which an attacker may have learned an NFS file handle, you may be able to protect the integrity of your file system under attack. (Note, however, that unless you filter forged packets at your firewall, the attacker can put whatever source address he wants on an NFS UDP packet.) See the `mountd' or `exports' manual page for more detail. *Note: under no circumstances should you make your file system "read-only to the world," as this will let anyone find out NFS file handles. You want the kernel to think of the file system as read-only for the world, but `mountd' to refuse to give out file handles to anybody but `localhost'.* `mountd -n'. ------------ The `mountd' command takes a flag `-n' meaning "allow mount requests from unprivileged ports." *Do not ever run use this flag*. Worse yet, some operating systems (notably HP-UX 9) always exhibit this behavior regardless of whether they `-n' flag has been specified. The `-n' option to `mountd' allows any user on an NFS client to learn file handles and thus act as any other user. The situation gets considerably worse when exporting file systems to `localhost', however, as SFS requires. Then everybody on the Internet can learn your NFS file handles. The reason is that the `portmap' command will forward mount requests and make them appear to come from `localhost'. `portmap' forwarding -------------------- In order to support broadcast RPCs, the `portmap' program will relay RPC requests to the machine it is running on, making them appear to come from `localhost'. That can have disastrous consequences in conjunction with `mountd -n' as described previously. It can also be used to work around "read-mostly" export options by forwarding NFS requests to the kernel from `localhost'. Operating systems are starting to ship with `portmap' programs that refuse to forward certain RPC calls including mount and NFS requests. Wietse Venema has also written a `portmap' replacement that has these properties, available from `ftp://ftp.porcupine.org/pub/security/index.html'. It is also a good idea to filter TCP and UDP ports 111 (`portmap') at your firewall, if you have one. Bugs in the NFS implementation ------------------------------ Many NFS implementations have bugs. Many of those bugs rarely surface when clients and servers with similar implementation talk to each other. Examples of bugs we've found include servers crashing when the receive a write request for an odd number of bytes, clients crashing when they receive the error `NFS3ERR_JUKEBOX', and clients using uninitialized memory when the server returns a `lookup3resok' data structure with `obj_attributes' having `attributes_follow' set to false. SFS allows potentially untrusted users to formulate NFS requests (though of course SFS requires file handles to decrypt correctly and stamps the request with the appropriate Unix uid/gid credentials). This may let bad users crash your server's kernel (or worse). Similarly, bad servers may be able to crash a client. As a precaution, you may want to be careful about exporting any portion of a file system to anonymous users with the `R' or `W' options to `Export' (*note export::). When analyzing your NFS code for security, you should know that even anonymous users can make the following NFS RPC's on a LOCAL-DIRECTORY in your `sfsrwsd_config' file: `NFSPROC3_GETATTR', `NFSPROC3_ACCESS', `NFSPROC3_FSINFO', and `NFSPROC3_PATHCONF'. On the client side, a bad, non-root user in collusion with a bad file server can possibly crash or deadlock the machine. Many NFS client implementations have inadequate locking that could lead to race conditions. Other implementations make assumptions about the hierarchical nature of a file system served by the server. By violating these assumptions (for example having two directories on a server each contain the other), a user may be able to deadlock the client and create unkillable processes. `logger' buffer overrun ----------------------- SFS pipes log messages through the `logger' program to get them into the system log. SFS can generate arbitrarily long lines. If your `logger' does something stupid like call `gets', it may suffer a buffer overrun. We assume no one does this, but feel the point is worth mentioning, since not all logger programs come with source. To avoid using `logger', you can run `sfscd' and `sfssd' with the `-d' flag and redirect standard error wherever you wish manually.  File: sfs.info, Node: implementation vulnerabilities, Prev: exposed vulnerabilities, Up: Security Vulnerabilities in the SFS implementation ========================================= Resource exhaustion ------------------- The best way to attack the SFS software is probably to cause resource exhaustion. You can try to run SFS out of file descriptors, memory, CPU time, or mount points. An attacker can run a server out of file descriptors by opening many parallel TCP connections. Such attacks can be detected using the `netstat' command to see who is connecting to SFS (which accepts connections on port 4). Users can run the client (also `sfsauthd') out of descriptors by connecting many times using the setgid program `/usr/local/lib/sfs-0.7.2/suidconnect'. These attacks can be traced using a tool like lsof, available from `ftp://vic.cc.purdue.edu/pub/tools/unix/lsof'. SFS enforces a maximum size of just over 64 K on all RPC requests. Nonetheless, a client could connect 1000 times, on each connection send the first 64 K of a slightly larger message, and just sit there. That would obviously consume about 64 Megabytes of memory, as SFS will wait patiently for the rest of the request. A worse problem is that SFS servers do not currently flow-control clients. Thus, an attacker could make many RPCs but not read the replies, causing the SFS server to buffer arbitrarily much data and run out of memory. (Obviously the server eventually flushes any buffered data when the TCP connection closes.) Connecting to an SFS server costs the server tens of milliseconds of CPU time. An attacker can try to burn a huge amount of the server's CPU time by connecting to the server many times. The effects of such attacks can be mitigated using hashcash, *Note HashCost::. Finally, a user on a client can cause a large number of file systems to be mounted. If the operating system has a limit on the number of mount points, a user could run the client out of mount points. Non-idempotent operations ------------------------- If a TCP connection is reset, the SFS client will attempt to reconnect to the server and retransmit whatever RPCs were pending at the time the connection dropped. Not all NFS RPCs are idempotent however. Thus, an attacker who caused a connection to reset at just the right time could, for instance, cause a `mkdir' command to return `EEXIST' when in fact it did just create the directory. Injecting packets on the loopback interface ------------------------------------------- SFS exchanges NFS traffic with the local operating system using the loopback interface. An attacker with physical access to the local ethernet may be able to inject arbitrary packets into a machine, including packets to 127.0.0.1. Without packet filtering in place, an attacker can also send packets from anywhere making them appear to come from 127.0.0.1. On the client, an attacker can forge NFS requests from the kernel to SFS, or forge replies from SFS to the kernel. The SFS client encrypts file handles before giving them to the operating system. Thus, the attacker is unlikely to be able to forge a request from the kernel to SFS that contain a valid file handle. In the other direction however, the reply does not need to contain a file handle. The attacker may well be able to convince the kernel of a forged reply from SFS. The attacker only needs to guess a (possibly quite predictable) 32-bit RPC XID number. Such an attack could result, for example, in a user getting the wrong data when reading a file. On the server side, you also must assume the attacker cannot guess a valid NFS file handle (otherwise, you already have no security--*note NFS security::). However, the attacker might again forge NFS replies, this time from the kernel to the SFS server software. To prevent such attacks, if your operating system has IP filtering, it would be a good idea to block any packets either from or to 127.0.0.1 if those packets do not come from the loopback interface. Blocking traffic "from" 127.0.0.1 at your firewall is also a good idea. Causing deadlock ---------------- On BSD-based systems (and possibly others) the buffer reclaiming policy can cause deadlock. When an operation needs a buffer and there are no clean buffers available, the kernel picks some particular dirty buffer and won't let the operation complete until it can get that buffer. This can lead to deadlock in the case that two machines mount each other. Getting private file data from public workstations -------------------------------------------------- An attacker may be able to read the contents of a private file shortly after you log out of a public workstation if the he can then become root on the workstation. There are two attacks possible. First, the attacker may be able to read data out of physical memory or from the swap partition of the local disk. File data may still be in memory if the kernel's NFS3 code has cached it in the buffer cache. There may also be fragments of file data in the memory of the `sfsrwcd' process, or out on disk in the swap partition (though `sfsrwcd' does its best to avoid getting paged out). The attacker can read any remaining file contents once he gains control of the machine. Alternatively, the attacker may have recorded encrypted session traffic between the client and server. Once he gains control of the client machine, he can attach to the `sfsrwcd' process with the debugger and learn the session key if the session is still open. This will let him read the session he recorded in encrypted form. To minimize the risks of these attacks, you must kill and restart `sfscd' before turning control of a public workstation over to another user. Even this is not guaranteed to fix the problem. It will flush file blocks from the buffer cache by unmounting all file systems, for example, but the contents of those blocks may persist as uninitialized data in buffers sitting on the free list. Similarly, any programs you ran that manipulated private file data may have gotten paged out to disk, and the information may live on after the processes exit. In conclusion, if you are paranoid, it is best not to use public workstations. Setuid programs and devices on remote file systems -------------------------------------------------- SFS does its best to disable setuid programs and devices on remote file servers it mounts. However, we have only tested this on operating systems we have access to. When porting SFS to new platforms, It is worth testing that both setuid programs and devices do not work over SFS. Otherwise, any user of an SFS client can become root.  File: sfs.info, Node: Contacts, Next: Concept Index, Prev: Security, Up: Top How to contact people involved with SFS *************************************** Please report any bugs you find in SFS to . You can send mail to the authors of SFS at . There is also a mailing list of SFS users and developers at . To subscribe to the list, send mail to .  File: sfs.info, Node: Concept Index, Prev: Contacts, Up: Top Concept Index ************* * Menu: * /etc/exports: Quick server setup. * ___gmp_default_allocate: Build Problems. * aid: newaid. * Caffeine: Overview. * Certification paths: sfskey. * configure: Building. * dirsearch <1>: sfskey. * dirsearch: dirsearch. * Disk Full: Build Problems. * Dynamic server authentication: sfskey. * EDEADLK: sfscd. * HOSTID: Overview. * Internal compiler error: Build Problems. * NFS security: exposed vulnerabilities. * nfsmounter <1>: System overview. * nfsmounter: funmount. * Resource deadlock avoided: sfscd. * Self-certifying pathname: Overview. * sfs_config: sfs_config. * sfs_srp_params: sfs_srp_params. * sfs_users: sfs_users. * sfsauthd_config: sfsauthd_config. * sfscd_config: sfscd_config. * sfsrwsd_config: sfsrwsd_config. * sfssd_config: sfssd_config. * SRP: sfskey. * Virtual memory exhausted: Build Problems.