linux备份软件rsync-3.0.4.tar.gz

-*- indented-text -*- Notes towards a new version of rsync Martin Pool <mbp@samba.org>, September 2001. Good things about the current implementation: - Widely known and adopted. - Fast/efficient, especially for moderately small sets of files over slow links (transoceanic or modem.) - Fairly reliable. - The choice of runnning over a plain TCP socket or tunneling over ssh. - rsync operations are idempotent: you can always run the same command twice to make sure it worked properly without any fear. (Are there any exceptions?) - Small changes to files cause small deltas. - There is a way to evolve the protocol to some extent. - rdiff and rsync --write-batch allow generation of standalone patch sets. rsync+ is pretty cheesy, though. xdelta seems cleaner. - Process triangle is creative, but seems to provoke OS bugs. - "Morning-after property": you don't need to know anything on the local machine about the state of the remote machine, or about transfers that have been done in the past. - You can easily push or pull simply by switching the order of files. - The "modules" system has some neat features compared to e.g. Apache's per-directory configuration. In particular, because you can set a userid and chroot directory, there is strong protection between different modules. I haven't seen any calls for a more flexible system. Bad things about the current implementation: - Persistent and hard-to-diagnose hang bugs remain - Protocol is sketchily documented, tied to this implementation, and hard to modify/extend - Both the program and the protocol assume a single non-interactive one-way transfer - A list of all files are held in memory for the entire transfer, which cripples scalability to large file trees - Opening a new socket for every operation causes problems, especially when running over SSH with password authentication. - Renamed files are not handled: the old file is removed, and the new file created from scratch. - The versioning approach assumes that future versions of the program know about all previous versions, and will do the right thing. - People always get confused about ':' vs '::' - Error messages can be cryptic. - Default behaviour is not intuitive: in too many cases rsync will happily do nothing. Perhaps -a should be the default? - People get confused by trailing slashes, though it's hard to think of another reasonable way to make this necessary distinction between a directory and its contents. Protocol philosophy: *The* big difference between protocols like HTTP, FTP, and NFS is that their fundamental operations are "read this file", "delete this file", and "make this directory", whereas rsync is "make this directory like this one". Questionable features: These are neat, but not necessarily clean or worth preserving. - The remote rsync can be wrapped by some other program, such as in tridge's rsync-mail scripts. The general feature of sending and retrieving mail over rsync is good, but this is perhaps not the right way to implement it. Desirable features: These don't really require architectural changes; they're just something to keep in mind. - Synchronize ACLs and extended attributes - Anonymous servers should be efficient - Code should be portable to non-UNIX systems - Should be possible to document the protocol in RFC form - --dry-run option - IPv6 support. Pretty straightforward. - Allow the basis and destination files to be different. For example, you could use this when you have a CD-ROM and want to download an updated image onto a hard drive. - Efficiently interrupt and restart a transfer. We can write a checkpoint file that says where we're up to in the filesystem. Alternatively, as long as transfers are idempotent, we can just restart the whole thing. [NFSv4] - Scripting support. - Propagate atimes and do not modify them. This is very ugly on Unix. It might be better to try to add O_NOATIME to kernels, and call that. - Unicode. Probably just use UTF-8 for everything. - Open authentication system. Can we use PAM? Is SASL an adequate mapping of PAM to the network, or useful in some other way? - Resume interrupted transfers without the --partial flag. We need to leave the temporary file behind, and then know to use it. This leaves a risk of large temporary files accumulating, which is not good. Perhaps it should be off by default. - tcpwrappers support. Should be trivial; can already be done through tcpd or inetd. - Socks support built in. It's not clear this is any better than just linking against the socks library, though. - When run over SSH, invoke with predictable command-line arguments, so that people can restrict what commands sshd will run. (Is this really required?) - Comparison mode: give a list of which files are new, gone, or different. Set return code depending on whether anything has changed. - Internationalized messages (gettext?) - Optionally use real regexps rather than globs? - Show overall progress. Pretty hard to do, especially if we insist on not scanning the directory tree up front. Regression testing: - Support automatic testing. - Have hard internal timeouts against hangs. - Be deterministic. - Measure performance. Hard links: At the moment, we can recreate hard links, but it's a bit inefficient: it depends on holding a list of all files in the tree. Every time we see a file with a linkcount >1, we need to search for another known name that has the same (fsid,inum) tuple. We could do that more efficiently by keeping a list of only files with linkcount>1, and removing files from that list as all their names become known. Command-line options: We have rather a lot at the moment. We might get more if the tool becomes more flexible. Do we need a .rc or configuration file? That wouldn't really fit with its pattern of use: cp and tar don't have them, though ssh does. Scripting issues: - Perhaps support multiple scripting languages: candidates include Perl, Python, Tcl, Scheme (guile?), sh, ... - Simply running a subprocess and looking at its stdout/exit code might be sufficient, though it could also be pretty slow if it's called often. - There are security issues about running remote code, at least if it's not running in the users own account. So we can either disallow it, or use some kind of sandbox system. - Python is a good language, but the syntax is not so good for giving small fragments on the command line. - Tcl is broken Lisp. - Lots of sysadmins know Perl, though Perl can give some bizarre or confusing errors. The built in stat operators and regexps might be useful. - Sadly probably not enough people know Scheme. - sh is hard to embed. Scripting hooks: - Whether to transfer a file - What basis file to use - Logging - Whether to allow transfers (for public servers) - Authentication - Locking - Cache - Generating backup path/name. - Post-processing of backups, e.g. to do compression. - After transfer, before replacement: so that we can spit out a diff of what was changed, or kick off some kind of reconciliation process. VFS: Rather than talking straight to the filesystem, rsyncd talks through an internal API. Samba has one. Is it useful? - Could be a tidy way to implement cached signatures. - Keep files compressed on disk? Interactive interface: - Something like ncFTP, or integration into GNOME-vfs. Probably hold a single socket connection open. - Can either call us as a separate process, or as a library. - The standalone process needs to produce output in a form easily