diff --git a/NEWS b/NEWS index 3ee18a5..1cbbe9b 100644 --- a/NEWS +++ b/NEWS @@ -1,243 +1,115 @@ -NEWS for rsync 3.1.0 (28 Sep 2013) -Protocol: 31 (changed) -Changes since 3.0.9: - - OUTPUT CHANGES: - - - Output numbers in 3-digit groups by default (e.g. 1,234,567). See the - --human-readable option for a way to turn it off. See also the daemon's - "log format" parameter and related command-line options (including - --out-format) for a modifier that can be used to request digit-grouping - or human-readable output in log escapes. (Note that log output is - unchanged by default.) - - - The --list-only option is now affected by the --human-readable setting. - It will display digit groupings by default, and unit suffixes if higher - levels of readability are requested. Also, the column width for the size - output has increased from 11 to 14 characters when human readability is - enabled. Use --no-h to get the old-style output and column size. - - - The output of the --progress option has changed: the string "xfer" was - shortened to "xfr", and the string "to-check" was shortened to "to-chk", - both designed to make room for the (by default) wider display of file - size numbers without making the total line-length longer. Also, when - incremental recursion is enabled, the string "ir-chk" will be used - instead of "to-chk" up until the incremental-recursion scan is done, - letting you know that the value to check and the total value will still - be increasing as new files are found. - - - Enhanced the --stats output: 1) to mention how many files were created - (protocol >= 28), 2) to mention how many files were deleted (a new line - for protocol 31, but only output when --delete is in effect), and 3) to - follow the file-count, created-count, and deleted-count with a subcount - list that shows the counts by type. The wording of the transferred count - has also changed so that it is clearer that it is only a count of regular - files. +NEWS for rsync 3.1.1 (22 Jun 2014) +Protocol: 31 (unchanged) +Changes since 3.1.0: BUG FIXES: - - Fixed a bug in the iconv code when EINVAL or EILSEQ is returned with a - full output buffer. + - If the receiver gets bogus filenames from the sender (an unexpected + leading slash or a ".." infix dir), exit with an error. This prevents a + malicious sender from trying to inject filenames that would affect an + area outside the destination directories. - - Fixed some rare bugs in --iconv processing that might cause a multibyte - character to get translated incorrectly. + - Fixed a failure to remove the partial-transfer temp file when interrupted + (and rsync is not saving the partial files). - - Fixed a bogus "vanished file" error if some files were specified with - "./" prefixes and others were not. + - Changed the chown/group/xattr-set order to avoid losing some security- + related xattr info (that would get cleared by a chown). - - Fixed a bug in --sparse where an extra gap could get inserted after a - partial write. + - Fixed a bug in the xattr-finding code that could make a non-root-run + receiver not able to find some xattr numbers. - - Changed the way --progress overwrites its prior output in order to make - it nearly impossible for the progress to get overwritten by an error. + - Fixed a bug in the early daemon protocol where a timeout failed to be + honored (e.g. if the remote side fails to send us the initial protocol + greeting). - - Improved the propagation of abnormal-exit error messages. This should - help the client side to receive errors from the server when it is exiting - abnormally, and should also avoid dying with an "connection unexpectedly - closed" exit when the closed connection is really expected. + - Fixed unintended inclusion of commas in file numbers in the daemon log. - - The sender now checks each file it plans to remove to ensure that it - hasn't changed from the first stat's info. This helps to avoid losing - file data when the user is not using the option in a safe manner. + - We once again send the 'f' sub-flag (of -e) to the server side so it + knows that we can handle incremental-recursion directory errors properly + in older protocols. - - Fixed a data-duplication bug in the compress option that made compression - less efficient. This improves protocol 31 onward, while behaving in a - compatible (buggy) manner with older rsync protocols. + - Fixed an issue with too-aggressive keep-alive messages causing a problem + for older rsync versions early in the transfer. - - When creating a temp-file, rsync is now a bit smarter about it dot-char - choices, which can fix a problem on OS X with names that start with "..". + - Fixed an incorrect message about backup-directory-creation when using + --dry-run and the backup dir is not an absolute path. - - Rsync now sets a cleanup flag for --inplace and --append transfers that - will flush the write buffer if the transfer aborts. This ensures that - more received data gets written out to the disk on an aborted transfer - (which is quite helpful on a slow, flaky connection). + - Fixed a bug where a failed deletion and/or a failed sender-side removal + would not affect the exit code. - - The reads that map_ptr() now does are aligned on 1K boundaries. This - helps some filesystems and/or files that don't like unaligned reads. + - Fixed a bug that caused a failure when combining --delete-missing-args + with --xattrs and/or --acls. - - Fix an issue in the msleep() function if time jumps backwards. + - Fixed a strange dir_depth assertion error that was caused by empty-dir + removals and/or duplicate files in the transfer. - - Fix daemon-server module-name splitting bug where an arg would get split - even if --protect-args was used. + - Fixed a problem with --info=progress2's output stats where rsync would + only update the stats at the end of each file's transfer. It now uses + the data that is flowing for the current file, making the stats more + accurate and less jumpy. - ENHANCEMENTS: - - - Added the --remote-option=OPT (-M OPT) command-line option that is useful - for things like sending a remote --log-file=FILE or --fake-super option. - - - Added the --info=FLAGS and --debug=FLAGS options to allow finer-grained - control over what is output. Added an extra type of --progress output - using --info=progress2. - - - The --msgs2stderr option can help with debugging rsync by allowing the - debug messages to get output to stderr rather than travel via the socket - protocol. - - - Added the --delete-missing-args and --ignore-missing-args options to - either delete or ignore user-specified files on the receiver that are - missing on the sender (normally the absence of user-specified files - generates an error). - - - Added a "T" (terabyte) category to the --human-readable size suffixes. - - - Added the --usermap/--groupmap/--chown options for manipulating file - ownership during the copy. - - - Added the "%C" escape to the log-output handling, which will output the - MD5 checksum of any transferred file, or all files if --checksum was - specified (when protocol 30 or above is in effect). - - - Added the "reverse lookup" parameter to the rsync daemon config file to - allow reverse-DNS lookups to be disabled. - - - Added a forward-DNS lookup for the daemon's hosts allow/deny config. Can - be disabled via "forward lookup" parameter (defaults to enabled). - - - Added a way for more than one group to be specified in the daemon's - config file, including a way to specify that you want all of the - specified user's groups without having to name them. Also changed the - daemon to complain about an inability to set explicitly-specified uid/gid - values, even when not run by a super-user. - - - The daemon now tries to send the user the error messages from the - pre-xfer exec script when it fails. - - - Improved the use of alt-dest options into an existing hierarchy of files: - If a match is found in an alt-dir, it takes precedence over an existing - file. (We'll need to wait for a future version before attribute-changes - on otherwise unchanged files are safe when using an existing hierarchy.) - - - Added per-user authorization options and group-authorization support to - the daemon's "auth users" parameter. - - - Added a way to reference environment variables in a daemon's config file - (using %VAR% references). + - Fixed an itemize bug that affected the combo of --link-dest, -X, and -n. - - When replacing a non-dir with a symlink/hard-link/device/special-file, - the update should now be done in an atomic manner. + - Fixed a problem with delete messages not appearing in the log file when + the user didn't use --verbose. - - Avoid re-sending xattr info for hard-linked files w/the same xattrs - (protocol 31). + - Improve chunked xattr reading for OS X. - - The backup code was improved to use better logic maintaining the backup - directory hierarchy. Also, when a file is being backed up, rsync tries - to hard-link it into place so that the upcoming replacement of the - destination file will be atomic (for the normal, non-inplace logic). + - Removed an attempted hard-link xattr optimization that was causing a + transfer failure. This removal is flagged in the compatibility code, so + if a better fix can be discovered, we have a way to flip it on again. - - Added the ability to synchronize nano-second modified times. + - Fixed a bug when the receiver is not configured to be able to hard link + symlimks/devices/special-file items but the sender sent some of these + items flagged as hard-linked. - - Added a few more default suffixes for the "dont compress" settings. + - We now generate a better error if the buffer overflows in do_mknod(). - - Added the checking of the RSYNC_PROTECT_ARGS environment variable to allow - the default for the --protect-args command-line option to be overridden. + - Fixed a problem reading more than 16 ACLs on some OSes. - - Added the --preallocate command-line option. + - Fixed the reading of the secrets file to avoid an infinite wait when + the username is missing. - - Allow --password-file=- to read the password from stdin (filename "-"). + - Fixed a parsing problem in the --usermap/--groupmap options when using + MIN-MAX numbers. - - Rsync now comes packaged with an rsync-ssl helper script that can be - used to contact a remote rsync daemon using a piped-stunnel command. - It also includes an stunnel config file to run the server side to - support ssl daemon connections. See the packaging/lsb/rsync.spec - file for one way to package the resulting files. (Suggestions for - how to make this even easier to install & use are welcomed.) + - Switched Cygwin back to using socketpair "pipes" to try to speed it up. - - Improved the speed of some --inplace updates when there are lots of - identical checksum blocks that end up being unusable. + - Added knowledge of a few new options to rrsync. - - Added the --outbuf=N|L|B option for choosing the output buffering. - - - Repeating the --fuzzy option now causes the code to look for fuzzy - matches inside alt-dest directories too. - - - The --chmod option now supports numeric modes, e.g. --chmod=644,D755 - - - Added some Solaris xattr code. - - - Made an rsync daemon (the listening process) exit with a 0 status when - it was signaled to die. This helps launchd. - - - Improved the RSYNC_* environment variables for the pre-xfer exec script: - when a daemon is sent multiple request args, they are now joined into a - single return value (separated by spaces) so that the RSYNC_REQUEST - environment variable is accurate for any "pre-xfer exec". The values in - RSYNC_ARG# vars are no longer truncated at the "." arg (prior to the - request dirs/files), so that all the requested values are also listed - (separately) in RSYNC_ARG# variables. - - EXTRAS: - - - Added an "instant-rsyncd" script to the support directory, which makes - it easy to configure a simple rsync daemon in the current directory. - - - Added the "mapfrom" and "mapto" scripts to the support directory, which - makes it easier to do user/group mapping in a local transfer based on - passwd/group files from another machine. - - - There's a new, improved version of the lsh script in the support dir: - it's written in perl and supports -u without resorting to using sudo - (when run as root). The old shell version is now named lsh.sh. - - - There is a helper script named rsync-slash-strip in the support directory - for anyone that wants to change the way rsync handles args with trailing - slashes. (e.g. arg/ would get stripped to arg while arg/. would turn into - arg/). - - INTERNAL: - - - The I/O code was rewritten to be simpler and do bigger buffered reads - over the socket. The I/O between the receiver and the generator was - changed to be standard multiplexed-I/O (like that over the socket). - - - The sender tries to use any dead time while the generator is looking for - files to transfer in order to do sender-side directory scanning in a more - parallel manner. + ENHANCEMENTS: - - A daemon can now inform a client about a daemon-configured timeout value - so that the client can assist in the keep-alive activity (protocol 31). + - Tweaked the temp-file naming when --temp-dir=DIR is used: the temp-file + names will not get a '.' prepended. - - The filter code received some refactoring to make it more extendible, to - read better, and do better sanity checking. + - Added support for a new-compression idiom that does not compress all the + matching data in a transfer. This can help rsync to use less cpu when a + transfer has a lot of matching data, and also makes rsync compatible with + a non-bundled zlib. See the --new-compress and --old-compress options in + the manpage. - - Really big numbers are now output using our own big-num routine rather - than casting them to a double and using a %.0f conversion. + - Added the support/rsync-no-vanished wrapper script. - - The pool_alloc library has received some minor improvements in alignment - handling. + - Made configure more prominently mention when we failed to find yodl (in + case the user wants to be able to generate manpages from *.yo files). - - Added init_stat_x() function to avoid duplication of acl/xattr init code. + - Have manpage mention how a daemon's max-verbosity setting affects info + and debug options. Also added more clarification on backslash removals + for excludes that contain wildcards. - - The included zlib was upgraded from 1.2.3 to 1.2.8. + - Have configure check if for the attr lib (for getxattr) for those systems + that need to link against it explicitly. - - Rsync can now be compiled to use an unmodified zlib library instead of - the tweaked one that is included with rsync. This will eventually - become the default, at which point we'll start the countdown to removing - the included zlib. Until then, feel free to configure using: + - Change the early dir-creation logic to only use that idiom in an + inc-recursive copy that is preserving directory times. e.g. using + --omit-dir-times will avoid these early directories being created. - ./configure --with-included-zlib=no + - Fix a bug in cmp_time() that would return a wrong result if the 2 times + differed by an amount greater than what a time_t can hold. DEVELOPER RELATED: - - Added more conditional debug output. + - We now include an example systemd file (in packaging/systemd). - - Fixed some build issues for android and minix. + - Tweaked configure to make sure that any intended use of the included popt + and/or zlib code is put early in the CFLAGS. diff --git a/OLDNEWS b/OLDNEWS index a3b2ab7..ba02b07 100644 --- a/OLDNEWS +++ b/OLDNEWS @@ -1,3 +1,247 @@ +NEWS for rsync 3.1.0 (28 Sep 2013) +Protocol: 31 (changed) +Changes since 3.0.9: + + OUTPUT CHANGES: + + - Output numbers in 3-digit groups by default (e.g. 1,234,567). See the + --human-readable option for a way to turn it off. See also the daemon's + "log format" parameter and related command-line options (including + --out-format) for a modifier that can be used to request digit-grouping + or human-readable output in log escapes. (Note that log output is + unchanged by default.) + + - The --list-only option is now affected by the --human-readable setting. + It will display digit groupings by default, and unit suffixes if higher + levels of readability are requested. Also, the column width for the size + output has increased from 11 to 14 characters when human readability is + enabled. Use --no-h to get the old-style output and column size. + + - The output of the --progress option has changed: the string "xfer" was + shortened to "xfr", and the string "to-check" was shortened to "to-chk", + both designed to make room for the (by default) wider display of file + size numbers without making the total line-length longer. Also, when + incremental recursion is enabled, the string "ir-chk" will be used + instead of "to-chk" up until the incremental-recursion scan is done, + letting you know that the value to check and the total value will still + be increasing as new files are found. + + - Enhanced the --stats output: 1) to mention how many files were created + (protocol >= 28), 2) to mention how many files were deleted (a new line + for protocol 31, but only output when --delete is in effect), and 3) to + follow the file-count, created-count, and deleted-count with a subcount + list that shows the counts by type. The wording of the transferred count + has also changed so that it is clearer that it is only a count of regular + files. + + BUG FIXES: + + - Fixed a bug in the iconv code when EINVAL or EILSEQ is returned with a + full output buffer. + + - Fixed some rare bugs in --iconv processing that might cause a multibyte + character to get translated incorrectly. + + - Fixed a bogus "vanished file" error if some files were specified with + "./" prefixes and others were not. + + - Fixed a bug in --sparse where an extra gap could get inserted after a + partial write. + + - Changed the way --progress overwrites its prior output in order to make + it nearly impossible for the progress to get overwritten by an error. + + - Improved the propagation of abnormal-exit error messages. This should + help the client side to receive errors from the server when it is exiting + abnormally, and should also avoid dying with an "connection unexpectedly + closed" exit when the closed connection is really expected. + + - The sender now checks each file it plans to remove to ensure that it + hasn't changed from the first stat's info. This helps to avoid losing + file data when the user is not using the option in a safe manner. + + - Fixed a data-duplication bug in the compress option that made compression + less efficient. This improves protocol 31 onward, while behaving in a + compatible (buggy) manner with older rsync protocols. + + - When creating a temp-file, rsync is now a bit smarter about it dot-char + choices, which can fix a problem on OS X with names that start with "..". + + - Rsync now sets a cleanup flag for --inplace and --append transfers that + will flush the write buffer if the transfer aborts. This ensures that + more received data gets written out to the disk on an aborted transfer + (which is quite helpful on a slow, flaky connection). + + - The reads that map_ptr() now does are aligned on 1K boundaries. This + helps some filesystems and/or files that don't like unaligned reads. + + - Fix an issue in the msleep() function if time jumps backwards. + + - Fix daemon-server module-name splitting bug where an arg would get split + even if --protect-args was used. + + ENHANCEMENTS: + + - Added the --remote-option=OPT (-M OPT) command-line option that is useful + for things like sending a remote --log-file=FILE or --fake-super option. + + - Added the --info=FLAGS and --debug=FLAGS options to allow finer-grained + control over what is output. Added an extra type of --progress output + using --info=progress2. + + - The --msgs2stderr option can help with debugging rsync by allowing the + debug messages to get output to stderr rather than travel via the socket + protocol. + + - Added the --delete-missing-args and --ignore-missing-args options to + either delete or ignore user-specified files on the receiver that are + missing on the sender (normally the absence of user-specified files + generates an error). + + - Added a "T" (terabyte) category to the --human-readable size suffixes. + + - Added the --usermap/--groupmap/--chown options for manipulating file + ownership during the copy. + + - Added the "%C" escape to the log-output handling, which will output the + MD5 checksum of any transferred file, or all files if --checksum was + specified (when protocol 30 or above is in effect). + + - Added the "reverse lookup" parameter to the rsync daemon config file to + allow reverse-DNS lookups to be disabled. + + - Added a forward-DNS lookup for the daemon's hosts allow/deny config. Can + be disabled via "forward lookup" parameter (defaults to enabled). + + - Added a way for more than one group to be specified in the daemon's + config file, including a way to specify that you want all of the + specified user's groups without having to name them. Also changed the + daemon to complain about an inability to set explicitly-specified uid/gid + values, even when not run by a super-user. + + - The daemon now tries to send the user the error messages from the + pre-xfer exec script when it fails. + + - Improved the use of alt-dest options into an existing hierarchy of files: + If a match is found in an alt-dir, it takes precedence over an existing + file. (We'll need to wait for a future version before attribute-changes + on otherwise unchanged files are safe when using an existing hierarchy.) + + - Added per-user authorization options and group-authorization support to + the daemon's "auth users" parameter. + + - Added a way to reference environment variables in a daemon's config file + (using %VAR% references). + + - When replacing a non-dir with a symlink/hard-link/device/special-file, + the update should now be done in an atomic manner. + + - Avoid re-sending xattr info for hard-linked files w/the same xattrs + (protocol 31). + + - The backup code was improved to use better logic maintaining the backup + directory hierarchy. Also, when a file is being backed up, rsync tries + to hard-link it into place so that the upcoming replacement of the + destination file will be atomic (for the normal, non-inplace logic). + + - Added the ability to synchronize nano-second modified times. + + - Added a few more default suffixes for the "dont compress" settings. + + - Added the checking of the RSYNC_PROTECT_ARGS environment variable to allow + the default for the --protect-args command-line option to be overridden. + + - Added the --preallocate command-line option. + + - Allow --password-file=- to read the password from stdin (filename "-"). + + - Rsync now comes packaged with an rsync-ssl helper script that can be + used to contact a remote rsync daemon using a piped-stunnel command. + It also includes an stunnel config file to run the server side to + support ssl daemon connections. See the packaging/lsb/rsync.spec + file for one way to package the resulting files. (Suggestions for + how to make this even easier to install & use are welcomed.) + + - Improved the speed of some --inplace updates when there are lots of + identical checksum blocks that end up being unusable. + + - Added the --outbuf=N|L|B option for choosing the output buffering. + + - Repeating the --fuzzy option now causes the code to look for fuzzy + matches inside alt-dest directories too. + + - The --chmod option now supports numeric modes, e.g. --chmod=644,D755 + + - Added some Solaris xattr code. + + - Made an rsync daemon (the listening process) exit with a 0 status when + it was signaled to die. This helps launchd. + + - Improved the RSYNC_* environment variables for the pre-xfer exec script: + when a daemon is sent multiple request args, they are now joined into a + single return value (separated by spaces) so that the RSYNC_REQUEST + environment variable is accurate for any "pre-xfer exec". The values in + RSYNC_ARG# vars are no longer truncated at the "." arg (prior to the + request dirs/files), so that all the requested values are also listed + (separately) in RSYNC_ARG# variables. + + EXTRAS: + + - Added an "instant-rsyncd" script to the support directory, which makes + it easy to configure a simple rsync daemon in the current directory. + + - Added the "mapfrom" and "mapto" scripts to the support directory, which + makes it easier to do user/group mapping in a local transfer based on + passwd/group files from another machine. + + - There's a new, improved version of the lsh script in the support dir: + it's written in perl and supports -u without resorting to using sudo + (when run as root). The old shell version is now named lsh.sh. + + - There is a helper script named rsync-slash-strip in the support directory + for anyone that wants to change the way rsync handles args with trailing + slashes. (e.g. arg/ would get stripped to arg while arg/. would turn into + arg/). + + INTERNAL: + + - The I/O code was rewritten to be simpler and do bigger buffered reads + over the socket. The I/O between the receiver and the generator was + changed to be standard multiplexed-I/O (like that over the socket). + + - The sender tries to use any dead time while the generator is looking for + files to transfer in order to do sender-side directory scanning in a more + parallel manner. + + - A daemon can now inform a client about a daemon-configured timeout value + so that the client can assist in the keep-alive activity (protocol 31). + + - The filter code received some refactoring to make it more extendible, to + read better, and do better sanity checking. + + - Really big numbers are now output using our own big-num routine rather + than casting them to a double and using a %.0f conversion. + + - The pool_alloc library has received some minor improvements in alignment + handling. + + - Added init_stat_x() function to avoid duplication of acl/xattr init code. + + - The included zlib was upgraded from 1.2.3 to 1.2.8. + + - Rsync can now be compiled to use an unmodified zlib library instead of + the tweaked one that is included with rsync. This will eventually + become the default, at which point we'll start the countdown to removing + the included zlib. Until then, feel free to configure using: + + ./configure --with-included-zlib=no + + DEVELOPER RELATED: + + - Added more conditional debug output. + + - Fixed some build issues for android and minix. + NEWS for rsync 3.0.9 (23 Sep 2011) Protocol: 30 (unchanged) Changes since 3.0.8: @@ -3290,6 +3534,7 @@ Changes since 2.4.6: Partial Protocol History RELEASE DATE VER. DATE OF COMMIT* PROTOCOL + 22 Jun 2014 3.1.1 31 28 Sep 2013 3.1.0 31 Aug 2008 31 23 Sep 2011 3.0.9 30 26 Mar 2011 3.0.8 30 diff --git a/access.c b/access.c index e5a8d72..df89d6d 100644 --- a/access.c +++ b/access.c @@ -2,7 +2,7 @@ * Routines to authenticate access to a daemon (hosts allow/deny). * * Copyright (C) 1998 Andrew Tridgell - * Copyright (C) 2004-2013 Wayne Davison + * Copyright (C) 2004-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/acls.c b/acls.c index 46c5be5..ec1afc4 100644 --- a/acls.c +++ b/acls.c @@ -3,7 +3,7 @@ * * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras - * Copyright (C) 2006-2013 Wayne Davison + * Copyright (C) 2006-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -423,7 +423,7 @@ static BOOL pack_smb_acl(SMB_ACL_T *smb_acl, const rsync_acl *racl) #ifdef ACLS_NEED_MASK mask_bits = racl->mask_obj == NO_ENTRY ? racl->group_obj & ~NO_ENTRY : racl->mask_obj; COE( sys_acl_create_entry,(smb_acl, &entry) ); - COE( sys_acl_set_info,(entry, SMB_ACL_MASK, mask_bits, NULL) ); + COE( sys_acl_set_info,(entry, SMB_ACL_MASK, mask_bits, 0) ); #else if (racl->mask_obj != NO_ENTRY) { COE( sys_acl_create_entry,(smb_acl, &entry) ); @@ -560,7 +560,8 @@ int get_acl(const char *fname, stat_x *sxp) if (!preserve_devices) #endif return 0; - } + } else if (IS_MISSING_FILE(sxp->st)) + return 0; if (get_rsync_acl(fname, sxp->acc_acl, SMB_ACL_TYPE_ACCESS, sxp->st.st_mode) < 0) { diff --git a/authenticate.c b/authenticate.c index 84d78c5..c92746c 100644 --- a/authenticate.c +++ b/authenticate.c @@ -2,7 +2,7 @@ * Support rsync daemon authentication. * * Copyright (C) 1998-2000 Andrew Tridgell - * Copyright (C) 2002-2013 Wayne Davison + * Copyright (C) 2002-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -102,15 +102,16 @@ static const char *check_secret(int module, const char *user, const char *group, char pass2[MAX_DIGEST_LEN*2]; const char *fname = lp_secrets_file(module); STRUCT_STAT st; - int fd, ok = 1; + int ok = 1; int user_len = strlen(user); int group_len = group ? strlen(group) : 0; char *err; + FILE *fh; - if (!fname || !*fname || (fd = open(fname, O_RDONLY)) < 0) + if (!fname || !*fname || (fh = fopen(fname, "r")) == NULL) return "no secrets file"; - if (do_fstat(fd, &st) == -1) { + if (do_fstat(fileno(fh), &st) == -1) { rsyserr(FLOG, errno, "fstat(%s)", fname); ok = 0; } else if (lp_strict_modes(module)) { @@ -123,29 +124,30 @@ static const char *check_secret(int module, const char *user, const char *group, } } if (!ok) { - close(fd); + fclose(fh); return "ignoring secrets file"; } if (*user == '#') { /* Reject attempt to match a comment. */ - close(fd); + fclose(fh); return "invalid username"; } /* Try to find a line that starts with the user (or @group) name and a ':'. */ err = "secret not found"; - while ((user || group) && read_line_old(fd, line, sizeof line, 1)) { - const char **ptr, *s; + while ((user || group) && fgets(line, sizeof line, fh) != NULL) { + const char **ptr, *s = strtok(line, "\n\r"); int len; - if (*line == '@') { + if (!s) + continue; + if (*s == '@') { ptr = &group; len = group_len; - s = line+1; + s++; } else { ptr = &user; len = user_len; - s = line; } if (!*ptr || strncmp(s, *ptr, len) != 0 || s[len] != ':') continue; @@ -158,7 +160,7 @@ static const char *check_secret(int module, const char *user, const char *group, *ptr = NULL; /* Don't look for name again. */ } - close(fd); + fclose(fh); memset(line, 0, sizeof line); memset(pass2, 0, sizeof pass2); diff --git a/backup.c b/backup.c index 55e1067..8987723 100644 --- a/backup.c +++ b/backup.c @@ -2,7 +2,7 @@ * Backup handling code. * * Copyright (C) 1999 Andrew Tridgell - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/batch.c b/batch.c index afda1f9..5b329ba 100644 --- a/batch.c +++ b/batch.c @@ -3,7 +3,7 @@ * * Copyright (C) 1999 Weiss * Copyright (C) 2004 Chris Shoemaker - * Copyright (C) 2004-2013 Wayne Davison + * Copyright (C) 2004-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/byteorder.h b/byteorder.h index 579145d..49c358b 100644 --- a/byteorder.h +++ b/byteorder.h @@ -2,7 +2,7 @@ * Simple byteorder handling. * * Copyright (C) 1992-1995 Andrew Tridgell - * Copyright (C) 2007-2013 Wayne Davison + * Copyright (C) 2007-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -23,7 +23,7 @@ /* We know that the x86 can handle misalignment and has the same * byte order (LSB-first) as the 32-bit numbers we transmit. */ -#ifdef __i386__ +#if defined __i386__ || defined __i486__ || defined __i586__ || defined __i686__ || __amd64 #define CAREFUL_ALIGNMENT 0 #endif @@ -38,9 +38,11 @@ #define PVAL(buf,pos) (UVAL(buf,pos)|UVAL(buf,(pos)+1)<<8) #define IVAL(buf,pos) (PVAL(buf,pos)|PVAL(buf,(pos)+2)<<16) +#define IVAL64(buf,pos) (IVAL(buf,pos)|(int64)IVAL(buf,(pos)+4)<<32) #define SSVALX(buf,pos,val) (CVAL(buf,pos)=(val)&0xFF,CVAL(buf,pos+1)=(val)>>8) #define SIVALX(buf,pos,val) (SSVALX(buf,pos,val&0xFFFF),SSVALX(buf,pos+2,val>>16)) -#define SIVAL(buf,pos,val) SIVALX((buf),(pos),((uint32)(val))) +#define SIVAL(buf,pos,val) SIVALX(buf,pos,(uint32)(val)) +#define SIVAL64(buf,pos,val) (SIVAL(buf,pos,val),SIVAL(buf,(pos)+4,(val)>>32)) #define IVALu(buf,pos) IVAL(buf,pos) #define SIVALu(buf,pos,val) SIVAL(buf,pos,val) @@ -95,6 +97,28 @@ SIVAL(char *buf, int pos, uint32 val) SIVALu((uchar*)buf, pos, val); } +static inline int64 +IVAL64(const char *buf, int pos) +{ + union { + const char *b; + const int64 *num; + } u; + u.b = buf + pos; + return *u.num; +} + +static inline void +SIVAL64(char *buf, int pos, int64 val) +{ + union { + char *b; + int64 *num; + } u; + u.b = buf + pos; + *u.num = val; +} + # endif /* !AVOID_BYTEORDER_INLINE */ #endif /* !CAREFUL_ALIGNMENT */ diff --git a/case_N.h b/case_N.h index fb8a0fd..72451a9 100644 --- a/case_N.h +++ b/case_N.h @@ -1,7 +1,7 @@ /* * Allow an arbitrary sequence of case labels. * - * Copyright (C) 2006-2013 Wayne Davison + * Copyright (C) 2006-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/checksum.c b/checksum.c index 9e071a4..a1c2aa2 100644 --- a/checksum.c +++ b/checksum.c @@ -3,7 +3,7 @@ * * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras - * Copyright (C) 2004-2013 Wayne Davison + * Copyright (C) 2004-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -98,10 +98,10 @@ void get_checksum2(char *buf, int32 len, char *sum) } } -void file_checksum(char *fname, char *sum, OFF_T size) +void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum) { struct map_struct *buf; - OFF_T i, len = size; + OFF_T i, len = st_p->st_size; md_context m; int32 remainder; int fd; @@ -112,7 +112,7 @@ void file_checksum(char *fname, char *sum, OFF_T size) if (fd == -1) return; - buf = map_file(fd, size, MAX_MAP_SIZE, CSUM_CHUNK); + buf = map_file(fd, len, MAX_MAP_SIZE, CSUM_CHUNK); if (protocol_version >= 30) { md5_begin(&m); diff --git a/chmod.c b/chmod.c index 882f83e..1f73958 100644 --- a/chmod.c +++ b/chmod.c @@ -2,7 +2,7 @@ * Implement the core of the --chmod option. * * Copyright (C) 2002 Scott Howard - * Copyright (C) 2005-2013 Wayne Davison + * Copyright (C) 2005-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/cleanup.c b/cleanup.c index c750ba6..cd023aa 100644 --- a/cleanup.c +++ b/cleanup.c @@ -4,7 +4,7 @@ * Copyright (C) 1996-2000 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -156,8 +156,6 @@ NORETURN void _exit_cleanup(int code, const char *file, int line) switch_step++; if (cleanup_got_literal && (cleanup_fname || cleanup_fd_w != -1)) { - const char *fname = cleanup_fname; - cleanup_fname = NULL; if (cleanup_fd_r != -1) { close(cleanup_fd_r); cleanup_fd_r = -1; @@ -167,9 +165,11 @@ NORETURN void _exit_cleanup(int code, const char *file, int line) close(cleanup_fd_w); cleanup_fd_w = -1; } - if (fname && cleanup_new_fname && keep_partial + if (cleanup_fname && cleanup_new_fname && keep_partial && handle_partial_dir(cleanup_new_fname, PDIR_CREATE)) { int tweak_modtime = 0; + const char *fname = cleanup_fname; + cleanup_fname = NULL; if (!partial_dir) { /* We don't want to leave a partial file with a modern time or it * could be skipped via --update. Setting the time to something diff --git a/clientname.c b/clientname.c index 935d2d6..ccd43d7 100644 --- a/clientname.c +++ b/clientname.c @@ -3,7 +3,7 @@ * * Copyright (C) 1992-2001 Andrew Tridgell * Copyright (C) 2001, 2002 Martin Pool - * Copyright (C) 2002-2013 Wayne Davison + * Copyright (C) 2002-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/clientserver.c b/clientserver.c index 3b83e50..d9b5bb0 100644 --- a/clientserver.c +++ b/clientserver.c @@ -3,7 +3,7 @@ * * Copyright (C) 1998-2001 Andrew Tridgell * Copyright (C) 2001-2002 Martin Pool - * Copyright (C) 2002-2013 Wayne Davison + * Copyright (C) 2002-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/compat.c b/compat.c index fb978bd..2454937 100644 --- a/compat.c +++ b/compat.c @@ -3,7 +3,7 @@ * * Copyright (C) Andrew Tridgell 1996 * Copyright (C) Paul Mackerras 1996 - * Copyright (C) 2004-2013 Wayne Davison + * Copyright (C) 2004-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -26,6 +26,7 @@ int file_extra_cnt = 0; /* count of file-list extras that everyone gets */ int inc_recurse = 0; int compat_flags = 0; int use_safe_inc_flist = 0; +int want_xattr_optim = 0; extern int am_server; extern int am_sender; @@ -76,6 +77,7 @@ int filesfrom_convert = 0; #define CF_SYMLINK_TIMES (1<<1) #define CF_SYMLINK_ICONV (1<<2) #define CF_SAFE_FLIST (1<<3) +#define CF_AVOID_XATTR_OPTIM (1<<4) static const char *client_info; @@ -267,11 +269,14 @@ void setup_protocol(int f_out,int f_in) #endif if (local_server || strchr(client_info, 'f') != NULL) compat_flags |= CF_SAFE_FLIST; + if (local_server || strchr(client_info, 'x') != NULL) + compat_flags |= CF_AVOID_XATTR_OPTIM; write_byte(f_out, compat_flags); } else compat_flags = read_byte(f_in); /* The inc_recurse var MUST be set to 0 or 1. */ inc_recurse = compat_flags & CF_INC_RECURSE ? 1 : 0; + want_xattr_optim = protocol_version >= 31 && !(compat_flags & CF_AVOID_XATTR_OPTIM); if (am_sender) { receiver_symlink_times = am_server ? strchr(client_info, 'L') != NULL diff --git a/configure.ac b/configure.ac index c7507de..cf588ce 100644 --- a/configure.ac +++ b/configure.ac @@ -3,9 +3,9 @@ dnl Process this file with autoconf to produce a configure script. AC_INIT() AC_CONFIG_SRCDIR([byteorder.h]) AC_CONFIG_HEADER(config.h) -AC_PREREQ(2.59) +AC_PREREQ(2.60) -RSYNC_VERSION=3.1.0 +RSYNC_VERSION=3.1.1 AC_SUBST(RSYNC_VERSION) AC_MSG_NOTICE([Configuring rsync $RSYNC_VERSION]) @@ -705,6 +705,7 @@ AC_CACHE_CHECK([whether link() can hard-link symlinks],rsync_cv_can_hardlink_sym main() { unlink(FILENAME); if (symlink("conftest.no-such", FILENAME) < 0) abort(); + unlink(FILENAME "2"); if (link(FILENAME, FILENAME "2") < 0) exit(1); exit(0); }], @@ -724,6 +725,7 @@ AC_CACHE_CHECK([whether link() can hard-link special files],rsync_cv_can_hardlin main() { unlink(FILENAME); if (mkfifo(FILENAME, 0777) < 0) abort(); + unlink(FILENAME "2"); if (link(FILENAME, FILENAME "2") < 0) exit(1); exit(0); }], @@ -739,11 +741,7 @@ AC_TRY_RUN([ main() { int fd[2]; -#ifdef __CYGWIN__ - exit(1); -#else exit((socketpair(AF_UNIX, SOCK_STREAM, 0, fd) != -1) ? 0 : 1); -#endif }], rsync_cv_HAVE_SOCKETPAIR=yes,rsync_cv_HAVE_SOCKETPAIR=no,rsync_cv_HAVE_SOCKETPAIR=cross)]) if test x"$rsync_cv_HAVE_SOCKETPAIR" = x"yes"; then @@ -769,7 +767,7 @@ AC_MSG_CHECKING([whether to use included libpopt]) if test x"$with_included_popt" = x"yes"; then AC_MSG_RESULT($srcdir/popt) BUILD_POPT='$(popt_OBJS)' - CFLAGS="$CFLAGS -I$srcdir/popt" + CFLAGS="-I$srcdir/popt $CFLAGS" if test x"$ALLOCA" != x then # this can be removed when/if we add an included alloca.c; @@ -794,8 +792,9 @@ AC_MSG_CHECKING([whether to use included zlib]) if test x"$with_included_zlib" = x"yes"; then AC_MSG_RESULT($srcdir/zlib) BUILD_ZLIB='$(zlib_OBJS)' - CFLAGS="$CFLAGS -I$srcdir/zlib" + CFLAGS="-I$srcdir/zlib $CFLAGS" else + AC_DEFINE(EXTERNAL_ZLIB, 1, [Define to 1 if using external zlib]) AC_MSG_RESULT(no) fi @@ -1010,7 +1009,7 @@ else *) AC_MSG_RESULT(running tests:) AC_CHECK_LIB(acl,acl_get_file) - AC_CACHE_CHECK([for ACL support],samba_cv_HAVE_POSIX_ACLS,[ + AC_CACHE_CHECK([for ACL support],samba_cv_HAVE_POSIX_ACLS,[ AC_TRY_LINK([#include #include ], [ acl_t acl; int entry_id; acl_entry_t *entry_p; return acl_get_entry( acl, entry_id, entry_p);], @@ -1060,6 +1059,7 @@ else AC_DEFINE(HAVE_LINUX_XATTRS, 1, [True if you have Linux xattrs]) AC_DEFINE(SUPPORT_XATTRS, 1) AC_DEFINE(NO_SYMLINK_USER_XATTRS, 1, [True if symlinks do not support user xattrs]) + AC_CHECK_LIB(attr,getxattr) ;; darwin*) AC_MSG_RESULT(Using OS X xattrs) @@ -1112,3 +1112,8 @@ AC_OUTPUT AC_MSG_RESULT() AC_MSG_RESULT([ rsync ${RSYNC_VERSION} configuration successful]) AC_MSG_RESULT() +if test x$HAVE_YODL2MAN != x1; then + AC_MSG_RESULT([ Note that yodl2man was not found, so pre-existing manpage files will be]) + AC_MSG_RESULT([ used w/o change (if available) -- no .yo file changes will be used.]) + AC_MSG_RESULT() +fi diff --git a/delete.c b/delete.c index 13358dd..2927a93 100644 --- a/delete.c +++ b/delete.c @@ -4,7 +4,7 @@ * Copyright (C) 1996-2000 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -199,7 +199,7 @@ enum delret delete_item(char *fbuf, uint16 mode, uint16 flags) fbuf); ret = DR_NOT_EMPTY; } else if (errno != ENOENT) { - rsyserr(FERROR, errno, "delete_file: %s(%s) failed", + rsyserr(FERROR_XFER, errno, "delete_file: %s(%s) failed", what, fbuf); ret = DR_FAILURE; } else diff --git a/errcode.h b/errcode.h index aaa4d46..a428b89 100644 --- a/errcode.h +++ b/errcode.h @@ -2,7 +2,7 @@ * Error codes returned by rsync. * * Copyright (C) 1998-2000 Andrew Tridgell - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/exclude.c b/exclude.c index a7331e2..efa5b48 100644 --- a/exclude.c +++ b/exclude.c @@ -4,7 +4,7 @@ * Copyright (C) 1996-2001 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -356,7 +356,7 @@ static char *parse_merge_name(const char *merge_file, unsigned int *len_ptr, fn_len = clean_fname(fn, CFN_COLLAPSE_DOT_DOT_DIRS); } - /* If the name isn't in buf yet, it's wasn't absolute. */ + /* If the name isn't in buf yet, it wasn't absolute. */ if (fn != buf) { int d_len = dirbuf_len - prefix_skip; if (d_len + fn_len >= MAXPATHLEN) { diff --git a/fileio.c b/fileio.c index 78decee..abef46d 100644 --- a/fileio.c +++ b/fileio.c @@ -3,7 +3,7 @@ * * Copyright (C) 1998 Andrew Tridgell * Copyright (C) 2002 Martin Pool - * Copyright (C) 2004-2013 Wayne Davison + * Copyright (C) 2004-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/flist.c b/flist.c index bf8d124..c24672e 100644 --- a/flist.c +++ b/flist.c @@ -4,7 +4,7 @@ * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2001, 2002 Martin Pool - * Copyright (C) 2002-2013 Wayne Davison + * Copyright (C) 2002-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -736,8 +736,11 @@ static struct file_struct *recv_file_entry(int f, struct file_list *flist, int x } #endif - if (*thisname) - clean_fname(thisname, 0); + if (*thisname + && (clean_fname(thisname, CFN_REFUSE_DOT_DOT_DIRS) < 0 || (!relative_paths && *thisname == '/'))) { + rprintf(FERROR, "ABORTING due to unsafe pathname from sender: %s\n", thisname); + exit_cleanup(RERR_PROTOCOL); + } if (sanitize_paths) sanitize_path(thisname, thisname, "", 0, SP_DEFAULT); @@ -945,7 +948,14 @@ static struct file_struct *recv_file_entry(int f, struct file_list *flist, int x memcpy(bp, basename, basename_len); #ifdef SUPPORT_HARD_LINKS - if (xflags & XMIT_HLINKED) + if (xflags & XMIT_HLINKED +#ifndef CAN_HARDLINK_SYMLINK + && !S_ISLNK(mode) +#endif +#ifndef CAN_HARDLINK_SPECIAL + && !IS_SPECIAL(mode) && !IS_DEVICE(mode) +#endif + ) file->flags |= FLAG_HLINKED; #endif file->modtime = (time_t)modtime; @@ -1156,7 +1166,7 @@ struct file_struct *make_file(const char *fname, struct file_list *flist, if (sanitize_paths) sanitize_path(thisname, thisname, "", 0, SP_DEFAULT); - if (stp && (S_ISDIR(stp->st_mode) || stp->st_mode == 0)) { + if (stp && (S_ISDIR(stp->st_mode) || IS_MISSING_FILE(*stp))) { /* This is needed to handle a "symlink/." with a --relative * dir, or a request to delete a specific file. */ st = *stp; @@ -1200,7 +1210,7 @@ struct file_struct *make_file(const char *fname, struct file_list *flist, full_fname(thisname)); } return NULL; - } else if (st.st_mode == 0) { + } else if (IS_MISSING_FILE(st)) { io_error |= IOERR_GENERAL; rprintf(FINFO, "skipping file with bogus (zero) st_mode: %s\n", full_fname(thisname)); @@ -1303,7 +1313,7 @@ struct file_struct *make_file(const char *fname, struct file_list *flist, #endif if (always_checksum && am_sender && S_ISREG(st.st_mode)) { - file_checksum(thisname, tmp_sum, st.st_size); + file_checksum(thisname, &st, tmp_sum); if (sender_keeps_checksum) extra_len += SUM_EXTRA_CNT * EXTRA_LEN; } @@ -2290,7 +2300,7 @@ struct file_list *send_file_list(int f, int argc, char *argv[]) } else fn = p; send_implied_dirs(f, flist, fbuf, fbuf, p, flags, - st.st_mode == 0 ? MISSING_NAME : name_type); + IS_MISSING_FILE(st) ? MISSING_NAME : name_type); if (fn == p) continue; } @@ -2553,6 +2563,9 @@ struct file_list *recv_file_list(int f) rprintf(FINFO, "[%s] flist_eof=1\n", who_am_i()); } + /* The --relative option sends paths with a leading slash, so we need + * to specify the strip_root option here. We rejected leading slashes + * for a non-relative transfer in recv_file_entry(). */ flist_sort_and_clean(flist, relative_paths); if (protocol_version < 30) { diff --git a/generator.c b/generator.c index b243b07..91009a5 100644 --- a/generator.c +++ b/generator.c @@ -4,7 +4,7 @@ * Copyright (C) 1996-2000 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -57,6 +57,7 @@ extern int update_only; extern int human_readable; extern int ignore_existing; extern int ignore_non_existing; +extern int want_xattr_optim; extern int inplace; extern int append_mode; extern int make_backups; @@ -356,6 +357,9 @@ static void do_delete_pass(void) for (j = 0; j < cur_flist->used; j++) { struct file_struct *file = cur_flist->sorted[j]; + if (!F_IS_ACTIVE(file)) + continue; + f_name(file, fbuf); if (!(file->flags & FLAG_CONTENT_DIR)) { @@ -551,7 +555,7 @@ void itemize(const char *fnamecmp, struct file_struct *file, int ndx, int statre if (preserve_xattrs && do_xfers && iflags & (ITEM_REPORT_XATTR|ITEM_TRANSFER)) { int fd = iflags & ITEM_REPORT_XATTR - && (protocol_version < 31 || !BITS_SET(iflags, ITEM_XNAME_FOLLOWS|ITEM_LOCAL_CHANGE)) + && !(want_xattr_optim && BITS_SET(iflags, ITEM_XNAME_FOLLOWS|ITEM_LOCAL_CHANGE)) ? sock_f_out : -1; send_xattr_request(NULL, file, fd); } @@ -574,7 +578,7 @@ int unchanged_file(char *fn, struct file_struct *file, STRUCT_STAT *st) of the file time to determine whether to sync */ if (always_checksum > 0 && S_ISREG(st->st_mode)) { char sum[MAX_DIGEST_LEN]; - file_checksum(fn, sum, st->st_size); + file_checksum(fn, st, sum); return memcmp(sum, F_SUM(file), checksum_len) == 0; } @@ -755,6 +759,9 @@ static struct file_struct *find_fuzzy(struct file_struct *file, struct file_list for (j = 0; j < dirlist->used; j++) { struct file_struct *fp = dirlist->files[j]; + if (!F_IS_ACTIVE(fp)) + continue; + if (!S_ISREG(fp->mode) || !F_LENGTH(fp) || fp->flags & FLAG_FILE_SENT) continue; @@ -780,6 +787,9 @@ static struct file_struct *find_fuzzy(struct file_struct *file, struct file_list int len, suf_len; uint32 dist; + if (!F_IS_ACTIVE(fp)) + continue; + if (!S_ISREG(fp->mode) || !F_LENGTH(fp) || fp->flags & FLAG_FILE_SENT) continue; @@ -1346,6 +1356,8 @@ static void recv_generator(char *fname, struct file_struct *file, int ndx, } else added_perms = 0; if (is_dir < 0) { + if (!(preserve_times & PRESERVE_DIR_TIMES)) + return; /* In inc_recurse mode we want to make sure any missing * directories get created while we're still processing * the parent dir (which allows us to touch the parent @@ -1499,7 +1511,7 @@ static void recv_generator(char *fname, struct file_struct *file, int ndx, set_file_attrs(fname, file, &sx, NULL, maybe_ATTRS_REPORT); if (itemizing) itemize(fname, file, ndx, 0, &sx, 0, 0, NULL); -#if defined SUPPORT_HARD_LINKS && defined CAN_HARDLINK_SYMLINK +#ifdef SUPPORT_HARD_LINKS if (preserve_hard_links && F_IS_HLINKED(file)) finish_hard_link(file, fname, ndx, &sx.st, itemizing, code, -1); #endif @@ -1520,15 +1532,17 @@ static void recv_generator(char *fname, struct file_struct *file, int ndx, goto cleanup; itemizing = 0; code = FNONE; - } else if (j >= 0) + } else if (j >= 0) { statret = 1; + fnamecmp = fnamecmpbuf; + } } - if (atomic_create(file, fname, sl, MAKEDEV(0, 0), &sx, statret == 0 ? DEL_FOR_SYMLINK : 0)) { + if (atomic_create(file, fname, sl, NULL, MAKEDEV(0, 0), &sx, statret == 0 ? DEL_FOR_SYMLINK : 0)) { set_file_attrs(fname, file, NULL, NULL, 0); if (itemizing) { if (statret == 0 && !S_ISLNK(sx.st.st_mode)) statret = -1; - itemize(fname, file, ndx, statret, &sx, + itemize(fnamecmp, file, ndx, statret, &sx, ITEM_LOCAL_CHANGE|ITEM_REPORT_CHANGE, 0, NULL); } if (code != FNONE && INFO_GTE(NAME, 1)) @@ -1594,18 +1608,20 @@ static void recv_generator(char *fname, struct file_struct *file, int ndx, goto cleanup; itemizing = 0; code = FNONE; - } else if (j >= 0) + } else if (j >= 0) { statret = 1; + fnamecmp = fnamecmpbuf; + } } if (DEBUG_GTE(GENR, 1)) { rprintf(FINFO, "mknod(%s, 0%o, [%ld,%ld])\n", fname, (int)file->mode, (long)major(rdev), (long)minor(rdev)); } - if (atomic_create(file, fname, NULL, rdev, &sx, del_for_flag)) { + if (atomic_create(file, fname, NULL, NULL, rdev, &sx, del_for_flag)) { set_file_attrs(fname, file, NULL, NULL, 0); if (itemizing) { - itemize(fname, file, ndx, statret, &sx, + itemize(fnamecmp, file, ndx, statret, &sx, ITEM_LOCAL_CHANGE|ITEM_REPORT_CHANGE, 0, NULL); } if (code != FNONE && INFO_GTE(NAME, 1)) @@ -1907,11 +1923,11 @@ static void recv_generator(char *fname, struct file_struct *file, int ndx, } /* If we are replacing an existing hard link, symlink, device, or special file, - * create a temp-name item and rename it into place. Only a symlink or hard - * link puts a non-NULL value into the lnk arg. Only a device puts a non-0 - * value into the rdev arg. Specify 0 for the del_for_flag if there is not a - * file to replace. This returns 1 on success and 0 on failure. */ -int atomic_create(struct file_struct *file, char *fname, const char *lnk, + * create a temp-name item and rename it into place. A symlimk specifies slnk, + * a hard link specifies hlnk, otherwise we create a device based on rdev. + * Specify 0 for the del_for_flag if there is not a file to replace. This + * returns 1 on success and 0 on failure. */ +int atomic_create(struct file_struct *file, char *fname, const char *slnk, const char *hlnk, dev_t rdev, stat_x *sxp, int del_for_flag) { char tmpname[MAXPATHLEN]; @@ -1936,23 +1952,22 @@ int atomic_create(struct file_struct *file, char *fname, const char *lnk, create_name = skip_atomic ? fname : tmpname; - if (lnk) { + if (slnk) { #ifdef SUPPORT_LINKS - if (S_ISLNK(file->mode) -#ifdef SUPPORT_HARD_LINKS /* The first symlink in a hard-linked cluster is always created. */ - && (!F_IS_HLINKED(file) || file->flags & FLAG_HLINK_FIRST) -#endif - ) { - if (do_symlink(lnk, create_name) < 0) { - rsyserr(FERROR_XFER, errno, "symlink %s -> \"%s\" failed", - full_fname(create_name), lnk); - return 0; - } - } else + if (do_symlink(slnk, create_name) < 0) { + rsyserr(FERROR_XFER, errno, "symlink %s -> \"%s\" failed", + full_fname(create_name), slnk); + return 0; + } +#else + return 0; #endif + } else if (hlnk) { #ifdef SUPPORT_HARD_LINKS - if (!hard_link_one(file, create_name, lnk, 0)) + if (!hard_link_one(file, create_name, hlnk, 0)) return 0; +#else + return 0; #endif } else { if (do_mknod(create_name, file->mode, rdev) < 0) { @@ -2014,6 +2029,8 @@ static void touch_up_dirs(struct file_list *flist, int ndx) * transfer and/or re-set any tweaked modified-time values. */ for (i = start; i <= end; i++, counter++) { file = flist->files[i]; + if (!F_IS_ACTIVE(file)) + continue; if (!S_ISDIR(file->mode) || (!implied_dirs && file->flags & FLAG_IMPLIED_DIR)) continue; @@ -2024,8 +2041,7 @@ static void touch_up_dirs(struct file_list *flist, int ndx) } /* Be sure not to retouch permissions with --fake-super. */ fix_dir_perms = !am_root && !(file->mode & S_IWUSR); - if (!F_IS_ACTIVE(file) || file->flags & FLAG_MISSING_DIR - || !(need_retouch_dir_times || fix_dir_perms)) + if (file->flags & FLAG_MISSING_DIR || !(need_retouch_dir_times || fix_dir_perms)) continue; fname = f_name(file, NULL); if (fix_dir_perms) diff --git a/getgroups.c b/getgroups.c index 236c193..bacb21b 100644 --- a/getgroups.c +++ b/getgroups.c @@ -3,7 +3,7 @@ * `id -G` on Linux, but it's too hard to find a portable equivalent. * * Copyright (C) 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License version 3 as diff --git a/hashtable.c b/hashtable.c index 9fc555d..5cdcf61 100644 --- a/hashtable.c +++ b/hashtable.c @@ -1,7 +1,7 @@ /* * Routines to provide a memory-efficient hashtable. * - * Copyright (C) 2007-2013 Wayne Davison + * Copyright (C) 2007-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/hlink.c b/hlink.c index 937c3c6..3b57898 100644 --- a/hlink.c +++ b/hlink.c @@ -4,7 +4,7 @@ * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2002 Martin Pool - * Copyright (C) 2004-2013 Wayne Davison + * Copyright (C) 2004-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -231,7 +231,7 @@ static int maybe_hard_link(struct file_struct *file, int ndx, } } - if (atomic_create(file, fname, oldname, MAKEDEV(0, 0), sxp, statret == 0 ? DEL_FOR_FILE : 0)) { + if (atomic_create(file, fname, NULL, oldname, MAKEDEV(0, 0), sxp, statret == 0 ? DEL_FOR_FILE : 0)) { if (itemizing) { itemize(fname, file, ndx, statret, sxp, ITEM_LOCAL_CHANGE | ITEM_XNAME_FOLLOWS, 0, diff --git a/ifuncs.h b/ifuncs.h index c0e45bb..6b119aa 100644 --- a/ifuncs.h +++ b/ifuncs.h @@ -1,6 +1,6 @@ /* Inline functions for rsync. * - * Copyright (C) 2007-2013 Wayne Davison + * Copyright (C) 2007-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/inums.h b/inums.h index 62eea04..3980b7a 100644 --- a/inums.h +++ b/inums.h @@ -1,6 +1,6 @@ /* Inline functions for rsync. * - * Copyright (C) 2008 Wayne Davison + * Copyright (C) 2008-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/io.c b/io.c index 3036903..b9a9bd0 100644 --- a/io.c +++ b/io.c @@ -4,7 +4,7 @@ * Copyright (C) 1996-2001 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2001, 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -155,7 +155,7 @@ static void read_a_msg(void); static void drain_multiplex_messages(void); static void sleep_for_bwlimit(int bytes_written); -static void check_timeout(BOOL allow_keepalive) +static void check_timeout(BOOL allow_keepalive, int keepalive_flags) { time_t t, chk; @@ -177,7 +177,7 @@ static void check_timeout(BOOL allow_keepalive) if (allow_keepalive) { /* This may put data into iobuf.msg w/o flushing. */ - maybe_send_keepalive(t, 0); + maybe_send_keepalive(t, keepalive_flags); } if (!last_io_in) @@ -232,28 +232,10 @@ static NORETURN void whine_about_eof(BOOL allow_kluge) * the socket except very early in the transfer. */ static size_t safe_read(int fd, char *buf, size_t len) { - size_t got; - int n; + size_t got = 0; assert(fd != iobuf.in_fd); - n = read(fd, buf, len); - if ((size_t)n == len || n == 0) { - if (DEBUG_GTE(IO, 2)) - rprintf(FINFO, "[%s] safe_read(%d)=%ld\n", who_am_i(), fd, (long)n); - return n; - } - if (n < 0) { - if (errno != EINTR && errno != EWOULDBLOCK && errno != EAGAIN) { - read_failed: - rsyserr(FERROR, errno, "safe_read failed to read %ld bytes [%s]", - (long)len, who_am_i()); - exit_cleanup(RERR_STREAMIO); - } - got = 0; - } else - got = n; - while (1) { struct timeval tv; fd_set r_fds, e_fds; @@ -273,8 +255,7 @@ static size_t safe_read(int fd, char *buf, size_t len) who_am_i()); exit_cleanup(RERR_FILEIO); } - if (io_timeout) - maybe_send_keepalive(time(NULL), MSK_ALLOW_FLUSH); + check_timeout(1, MSK_ALLOW_FLUSH); continue; } @@ -282,7 +263,7 @@ static size_t safe_read(int fd, char *buf, size_t len) rprintf(FINFO, "select exception on fd %d\n", fd); */ if (FD_ISSET(fd, &r_fds)) { - n = read(fd, buf + got, len - got); + int n = read(fd, buf + got, len - got); if (DEBUG_GTE(IO, 2)) rprintf(FINFO, "[%s] safe_read(%d)=%ld\n", who_am_i(), fd, (long)n); if (n == 0) @@ -290,7 +271,9 @@ static size_t safe_read(int fd, char *buf, size_t len) if (n < 0) { if (errno == EINTR) continue; - goto read_failed; + rsyserr(FERROR, errno, "safe_read failed to read %ld bytes [%s]", + (long)len, who_am_i()); + exit_cleanup(RERR_STREAMIO); } if ((got += (size_t)n) == len) break; @@ -768,7 +751,7 @@ static char *perform_io(size_t needed, int flags) send_extra_file_list(sock_f_out, -1); extra_flist_sending_enabled = !flist_eof; } else - check_timeout((flags & PIO_NEED_INPUT) != 0); + check_timeout((flags & PIO_NEED_INPUT) != 0, 0); FD_ZERO(&r_fds); /* Just in case... */ FD_ZERO(&w_fds); } @@ -1388,6 +1371,14 @@ void maybe_send_keepalive(time_t now, int flags) if (flags & MSK_ACTIVE_RECEIVER) last_io_in = now; /* Fudge things when we're working hard on the files. */ + /* Early in the transfer (before the receiver forks) the receiving side doesn't + * care if it hasn't sent data in a while as long as it is receiving data (in + * fact, a pre-3.1.0 rsync would die if we tried to send it a keep alive during + * this time). So, if we're an early-receiving proc, just return and let the + * incoming data determine if we timeout. */ + if (!am_sender && !am_receiver && !am_generator) + return; + if (now - last_io_out >= allowed_lull) { /* The receiver is special: it only sends keep-alive messages if it is * actively receiving data. Otherwise, it lets the generator timeout. */ @@ -1794,7 +1785,7 @@ int64 read_varlong(int f, uchar min_bytes) #if SIZEOF_INT64 < 8 u.x = IVAL(u.b,0); #elif CAREFUL_ALIGNMENT - u.x = IVAL(u.b,0) | (((int64)IVAL(u.b,4))<<32); + u.x = IVAL64(u.b,0); #endif return u.x; } @@ -2046,10 +2037,10 @@ void write_varlong(int f, int64 x, uchar min_bytes) uchar bit; int cnt = 8; - SIVAL(b, 1, x); #if SIZEOF_INT64 >= 8 - SIVAL(b, 5, x >> 32); + SIVAL64(b, 1, x); #else + SIVAL(b, 1, x); if (x <= 0x7FFFFFFF && x >= 0) memset(b + 5, 0, 4); else { @@ -2096,6 +2087,19 @@ void write_longint(int f, int64 x) #endif } +void write_bigbuf(int f, const char *buf, size_t len) +{ + size_t half_max = (iobuf.out.size - iobuf.out_empty_len) / 2; + + while (len > half_max + 1024) { + write_buf(f, buf, half_max); + buf += half_max; + len -= half_max; + } + + write_buf(f, buf, len); +} + void write_buf(int f, const char *buf, size_t len) { size_t pos, siz; diff --git a/io.h b/io.h index 7f6505c..14c8489 100644 --- a/io.h +++ b/io.h @@ -1,5 +1,5 @@ /* - * Copyright (C) 2007-2013 Wayne Davison + * Copyright (C) 2007-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/itypes.h b/itypes.h index eb2b891..3949087 100644 --- a/itypes.h +++ b/itypes.h @@ -1,6 +1,6 @@ /* Inline functions for rsync. * - * Copyright (C) 2007-2013 Wayne Davison + * Copyright (C) 2007-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/lib/compat.c b/lib/compat.c index b89ef02..dfe963c 100644 --- a/lib/compat.c +++ b/lib/compat.c @@ -3,7 +3,7 @@ * * Copyright (C) 1998 Andrew Tridgell * Copyright (C) 2002 Martin Pool - * Copyright (C) 2004, 2005, 2006 Wayne Davison + * Copyright (C) 2004-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/lib/mdfour.c b/lib/mdfour.c index 18308b7..16b2358 100644 --- a/lib/mdfour.c +++ b/lib/mdfour.c @@ -4,7 +4,7 @@ * An implementation of MD4 designed for use in the SMB authentication protocol. * * Copyright (C) 1997-1998 Andrew Tridgell - * Copyright (C) 2005-2008 Wayne Davison + * Copyright (C) 2005-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/lib/permstring.c b/lib/permstring.c index 2ce1e7f..7b2414d 100644 --- a/lib/permstring.c +++ b/lib/permstring.c @@ -4,7 +4,7 @@ * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2001 Martin Pool - * Copyright (C) 2003, 2006 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/lib/sysacls.c b/lib/sysacls.c index 52314bc..6ccfe43 100644 --- a/lib/sysacls.c +++ b/lib/sysacls.c @@ -2,7 +2,7 @@ * Unix SMB/CIFS implementation. * Based on the Samba ACL support code. * Copyright (C) Jeremy Allison 2000. - * Copyright (C) 2007-2008 Wayne Davison + * Copyright (C) 2007-2014 Wayne Davison * * The permission functions have been changed to get/set all bits via * one call. Some functions that rsync doesn't need were also removed. @@ -873,6 +873,10 @@ int sys_acl_get_tag_type(SMB_ACL_ENTRY_T entry_d, SMB_ACL_TAG_T *type_p) #define INITIAL_ACL_SIZE 16 +#ifndef NACLENTRIES +#define NACLENTRIES 0 +#endif + SMB_ACL_T sys_acl_get_file(const char *path_p, SMB_ACL_TYPE_T type) { SMB_ACL_T acl_d; @@ -909,7 +913,7 @@ SMB_ACL_T sys_acl_get_file(const char *path_p, SMB_ACL_TYPE_T type) sys_acl_free_acl(acl_d); - if ((count = acl(path_p, ACL_CNT, 0, NULL)) < 0) { + if ((count = acl(path_p, ACL_CNT, NACLENTRIES, NULL)) < 0) { return NULL; } diff --git a/lib/sysacls.h b/lib/sysacls.h index fd75bb6..31c4909 100644 --- a/lib/sysacls.h +++ b/lib/sysacls.h @@ -3,7 +3,7 @@ * Version 2.2.x * Portable SMB ACL interface * Copyright (C) Jeremy Allison 2000 - * Copyright (C) 2007-2008 Wayne Davison + * Copyright (C) 2007-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/lib/sysxattrs.c b/lib/sysxattrs.c index d55ee0c..f02802a 100644 --- a/lib/sysxattrs.c +++ b/lib/sysxattrs.c @@ -2,7 +2,7 @@ * Extended attribute support for rsync. * * Copyright (C) 2004 Red Hat, Inc. - * Copyright (C) 2003-2008 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * Written by Jay Fenlason. * * This program is free software; you can redistribute it and/or modify @@ -24,6 +24,10 @@ #ifdef SUPPORT_XATTRS +#ifdef HAVE_OSX_XATTRS +#define GETXATTR_FETCH_LIMIT (64*1024*1024) +#endif + #if defined HAVE_LINUX_XATTRS ssize_t sys_lgetxattr(const char *path, const char *name, void *value, size_t size) @@ -55,7 +59,24 @@ ssize_t sys_llistxattr(const char *path, char *list, size_t size) ssize_t sys_lgetxattr(const char *path, const char *name, void *value, size_t size) { - return getxattr(path, name, value, size, 0, XATTR_NOFOLLOW); + ssize_t len = getxattr(path, name, value, size, 0, XATTR_NOFOLLOW); + + /* If we're retrieving data, handle resource forks > 64MB specially */ + if (value != NULL && len == GETXATTR_FETCH_LIMIT && (size_t)len < size) { + /* getxattr will only return 64MB of data at a time, need to call again with a new offset */ + u_int32_t offset = len; + size_t data_retrieved = len; + while (data_retrieved < size) { + len = getxattr(path, name, value + offset, size - data_retrieved, offset, XATTR_NOFOLLOW); + if (len <= 0) + break; + data_retrieved += len; + offset += (u_int32_t)len; + } + len = data_retrieved; + } + + return len; } ssize_t sys_fgetxattr(int filedes, const char *name, void *value, size_t size) diff --git a/loadparm.c b/loadparm.c index be411fc..a37bbae 100644 --- a/loadparm.c +++ b/loadparm.c @@ -17,7 +17,7 @@ * and Karl Auer. Some of the changes are: * * Copyright (C) 2001, 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison */ /* Load parameters. diff --git a/log.c b/log.c index 34a013b..6e87a5b 100644 --- a/log.c +++ b/log.c @@ -3,7 +3,7 @@ * * Copyright (C) 1998-2001 Andrew Tridgell * Copyright (C) 2000-2001 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -846,13 +846,13 @@ void log_delete(const char *fname, int mode) x.file.mode = mode; - if (!INFO_GTE(DEL, 1) && !stdout_format) - ; - else if (am_server && protocol_version >= 29 && len < MAXPATHLEN) { + if (am_server && protocol_version >= 29 && len < MAXPATHLEN) { if (S_ISDIR(mode)) len++; /* directories include trailing null */ send_msg(MSG_DELETED, fname, len, am_generator); - } else { + } else if (!INFO_GTE(DEL, 1) && !stdout_format) + ; + else { fmt = stdout_format_has_o_or_i ? stdout_format : "deleting %n"; log_formatted(FCLIENT, fmt, "del.", &x.file, fname, ITEM_DELETED, NULL); } @@ -874,9 +874,9 @@ void log_exit(int code, const char *file, int line) { if (code == 0) { rprintf(FLOG,"sent %s bytes received %s bytes total size %s\n", - comma_num(stats.total_written), - comma_num(stats.total_read), - comma_num(stats.total_size)); + big_num(stats.total_written), + big_num(stats.total_read), + big_num(stats.total_size)); } else if (am_server != 2) { const char *name; diff --git a/main.c b/main.c index 726d0aa..e7a13f7 100644 --- a/main.c +++ b/main.c @@ -4,7 +4,7 @@ * Copyright (C) 1996-2001 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2001, 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/match.c b/match.c index a8bd1f3..39d15ed 100644 --- a/match.c +++ b/match.c @@ -3,7 +3,7 @@ * * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/options.c b/options.c index cebf256..62dfe4f 100644 --- a/options.c +++ b/options.c @@ -3,7 +3,7 @@ * * Copyright (C) 1998-2001 Andrew Tridgell * Copyright (C) 2000, 2001, 2002 Martin Pool - * Copyright (C) 2002-2013 Wayne Davison + * Copyright (C) 2002-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -32,6 +32,8 @@ extern unsigned int module_dirlen; extern filter_rule_list filter_list; extern filter_rule_list daemon_filter_list; +#define NOT_SPECIFIED (-42) + int make_backups = 0; /** @@ -75,7 +77,7 @@ int protocol_version = PROTOCOL_VERSION; int sparse_files = 0; int preallocate_files = 0; int do_compression = 0; -int def_compress_level = Z_DEFAULT_COMPRESSION; +int def_compress_level = NOT_SPECIFIED; int am_root = 0; /* 0 = normal, 1 = root, 2 = --super, -1 = --fake-super */ int am_server = 0; int am_sender = 0; @@ -611,7 +613,7 @@ static void print_rsync_version(enum logcode f) rprintf(f, "%s version %s protocol version %d%s\n", RSYNC_NAME, RSYNC_VERSION, PROTOCOL_VERSION, subprotocol); - rprintf(f, "Copyright (C) 1996-2013 by Andrew Tridgell, Wayne Davison, and others.\n"); + rprintf(f, "Copyright (C) 1996-2014 by Andrew Tridgell, Wayne Davison, and others.\n"); rprintf(f, "Web site: http://rsync.samba.org/\n"); rprintf(f, "Capabilities:\n"); rprintf(f, " %d-bit files, %d-bit inums, %d-bit timestamps, %d-bit long ints,\n", @@ -965,10 +967,12 @@ static struct poptOption long_options[] = { {"no-fuzzy", 0, POPT_ARG_VAL, &fuzzy_basis, 0, 0, 0 }, {"no-y", 0, POPT_ARG_VAL, &fuzzy_basis, 0, 0, 0 }, {"compress", 'z', POPT_ARG_NONE, 0, 'z', 0, 0 }, + {"old-compress", 0, POPT_ARG_VAL, &do_compression, 1, 0, 0 }, + {"new-compress", 0, POPT_ARG_VAL, &do_compression, 2, 0, 0 }, {"no-compress", 0, POPT_ARG_VAL, &do_compression, 0, 0, 0 }, {"no-z", 0, POPT_ARG_VAL, &do_compression, 0, 0, 0 }, {"skip-compress", 0, POPT_ARG_STRING, &skip_compress, 0, 0, 0 }, - {"compress-level", 0, POPT_ARG_INT, &def_compress_level, 'z', 0, 0 }, + {"compress-level", 0, POPT_ARG_INT, &def_compress_level, 0, 0, 0 }, {0, 'P', POPT_ARG_NONE, 0, 'P', 0, 0 }, {"progress", 0, POPT_ARG_VAL, &do_progress, 1, 0, 0 }, {"no-progress", 0, POPT_ARG_VAL, &do_progress, 0, 0, 0 }, @@ -1543,18 +1547,7 @@ int parse_arguments(int *argc_p, const char ***argv_p) break; case 'z': - if (def_compress_level < Z_DEFAULT_COMPRESSION - || def_compress_level > Z_BEST_COMPRESSION) { - snprintf(err_buf, sizeof err_buf, - "--compress-level value is invalid: %d\n", - def_compress_level); - return 0; - } - do_compression = def_compress_level != Z_NO_COMPRESSION; - if (do_compression && refused_compress) { - create_refuse_error(refused_compress); - return 0; - } + do_compression++; break; case 'M': @@ -1829,6 +1822,33 @@ int parse_arguments(int *argc_p, const char ***argv_p) exit_cleanup(0); } + if (do_compression || def_compress_level != NOT_SPECIFIED) { + if (def_compress_level == NOT_SPECIFIED) + def_compress_level = Z_DEFAULT_COMPRESSION; + else if (def_compress_level < Z_DEFAULT_COMPRESSION || def_compress_level > Z_BEST_COMPRESSION) { + snprintf(err_buf, sizeof err_buf, "--compress-level value is invalid: %d\n", + def_compress_level); + return 0; + } else if (def_compress_level == Z_NO_COMPRESSION) + do_compression = 0; + else if (!do_compression) + do_compression = 1; + if (do_compression && refused_compress) { + create_refuse_error(refused_compress); + return 0; + } +#ifdef EXTERNAL_ZLIB + if (do_compression == 1) { + snprintf(err_buf, sizeof err_buf, + "This rsync lacks old-style --compress due to its external zlib. Try -zz.\n"); + if (am_server) + return 0; + fprintf(stderr, "%s" "Continuing without compression.\n\n", err_buf); + do_compression = 0; + } +#endif + } + #ifdef HAVE_SETVBUF if (outbuf_mode && !am_server) { int mode = *(uchar *)outbuf_mode; @@ -2451,12 +2471,14 @@ void server_options(char **args, int *argc_p) } if (sparse_files) argstr[x++] = 'S'; - if (do_compression) + if (do_compression == 1) argstr[x++] = 'z'; set_allow_inc_recurse(); - /* Checking the pre-negotiated value allows --protocol=29 override. */ + /* We don't really know the actual protocol_version at this point, + * but checking the pre-negotiated value allows the user to use a + * --protocol=29 override to avoid the use of this -eFLAGS opt. */ if (protocol_version >= 30) { /* We make use of the -e option to let the server know about * any pre-release protocol version && some behavior flags. */ @@ -2472,11 +2494,13 @@ void server_options(char **args, int *argc_p) if (allow_inc_recurse) argstr[x++] = 'i'; #ifdef CAN_SET_SYMLINK_TIMES - argstr[x++] = 'L'; + argstr[x++] = 'L'; /* symlink time-setting support */ #endif #ifdef ICONV_OPTION - argstr[x++] = 's'; + argstr[x++] = 's'; /* symlink iconv translation support */ #endif + argstr[x++] = 'f'; /* flist I/O-error safety support */ + argstr[x++] = 'x'; /* xattr hardlink optimization not desired */ } if (x >= (int)sizeof argstr) { /* Not possible... */ @@ -2744,6 +2768,9 @@ void server_options(char **args, int *argc_p) exit_cleanup(RERR_MALLOC); } + if (do_compression > 1) + args[ac++] = "--new-compress"; + if (remote_option_cnt) { int j; if (ac + remote_option_cnt > MAX_SERVER_ARGS) { diff --git a/packaging/cull_options b/packaging/cull_options index add2b98..44b48d1 100755 --- a/packaging/cull_options +++ b/packaging/cull_options @@ -8,7 +8,9 @@ our %short_no_arg; our %short_with_num; our %long_opt = ( 'daemon' => -1, + 'debug' => 1, 'fake-super' => 0, + 'info' => 1, 'log-file' => 3, ); our $last_long_opt; diff --git a/packaging/lsb/rsync.spec b/packaging/lsb/rsync.spec index dd223a6..d9c3196 100644 --- a/packaging/lsb/rsync.spec +++ b/packaging/lsb/rsync.spec @@ -1,25 +1,27 @@ Summary: A fast, versatile, remote (and local) file-copying tool Name: rsync -Version: 3.1.0 +Version: 3.1.1 %define fullversion %{version} Release: 1 %define srcdir src Group: Applications/Internet +License: GPL Source0: http://rsync.samba.org/ftp/rsync/%{srcdir}/rsync-%{fullversion}.tar.gz #Source1: http://rsync.samba.org/ftp/rsync/%{srcdir}/rsync-patches-%{fullversion}.tar.gz URL: http://rsync.samba.org/ Prefix: %{_prefix} BuildRoot: /var/tmp/%{name}-root -License: GPL %package ssl-client Summary: Provides rsync-ssl -Requires: stunnel >= 4 +Group: Applications/Internet +Requires: rsync, stunnel >= 4 %package ssl-daemon Summary: An stunnel config file to support ssl rsync daemon connections. -Requires: stunnel >= 4 +Group: Applications/Internet +Requires: rsync, stunnel >= 4 %description Rsync is a fast and extraordinarily versatile file copying tool. It can @@ -89,8 +91,8 @@ rm -rf $RPM_BUILD_ROOT %dir /etc/rsync-ssl/certs %changelog -* Sat Sep 28 2013 Wayne Davison -Released 3.1.0. +* Sun Jun 22 2014 Wayne Davison +Released 3.1.1. * Fri Mar 21 2008 Wayne Davison Added installation of /etc/xinetd.d/rsync file and some commented-out diff --git a/packaging/patch-update b/packaging/patch-update index 4ac01eb..a1cb42c 100755 --- a/packaging/patch-update +++ b/packaging/patch-update @@ -45,20 +45,8 @@ while () { close PIPE; die "Unable to determine commit hash for master branch: $master_branch\n" unless defined $master_commit; -my @extra_files; -open(IN, '<', 'Makefile.in') or die "Couldn't open Makefile.in: $!\n"; -while () { - if (s/^GENFILES=//) { - while (s/\\$//) { - $_ .= ; - } - @extra_files = split(' ', $_); - last; - } -} -close IN; - if ($incl_generated_files) { + my @extra_files = get_extra_files(); die "'$tmp_dir' must not exist in the current directory.\n" if -e $tmp_dir; mkdir($tmp_dir, 0700) or die "Unable to mkdir($tmp_dir): $!\n"; system "$make_gen_cmd && rsync -a @extra_files $tmp_dir/master/" and exit 1; @@ -171,7 +159,9 @@ sub update_patch open(OUT, '>', "$patches_dir/$patch.diff") or die $!; print OUT $description{$patch}, "\nbased-on: $based_on\n"; + my @extra_files; if ($incl_generated_files) { + @extra_files = get_extra_files(); system "$make_gen_cmd && rsync -a @extra_files $tmp_dir/$patch/" and exit 1; } $last_touch = time; @@ -196,14 +186,15 @@ sub update_patch } else { ($parent_dir) = $parent =~ m{([^/]+)$}; } - open(PIPE, '-|', 'diff', '-up', "$tmp_dir/$parent_dir", "$tmp_dir/$patch") or die $!; + open(PIPE, '-|', 'diff', '-Nurp', "$tmp_dir/$parent_dir", "$tmp_dir/$patch") or die $!; while () { - s#^(diff -up) $tmp_dir/[^/]+/(.*?) $tmp_dir/[^/]+/(.*)#$1 a/$2 b/$3#o; + s#^(diff -Nurp) $tmp_dir/[^/]+/(.*?) $tmp_dir/[^/]+/(.*)#$1 a/$2 b/$3#o; s#^\Q---\E $tmp_dir/[^/]+/([^\t]+)\t.*#--- a/$1#o; s#^\Q+++\E $tmp_dir/[^/]+/([^\t]+)\t.*#+++ b/$1#o; print OUT $_; } close PIPE; + unlink @extra_files; } close OUT; @@ -213,6 +204,25 @@ sub update_patch exit; +sub get_extra_files +{ + my @extras; + + open(IN, '<', 'Makefile.in') or die "Couldn't open Makefile.in: $!\n"; + while () { + if (s/^GENFILES=//) { + while (s/\\$//) { + $_ .= ; + } + @extras = split(' ', $_); + last; + } + } + close IN; + + return @extras; +} + sub usage { die < - * Copyright (C) 2004-2013 Wayne Davison + * Copyright (C) 2004-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/progress.c b/progress.c index 89ecff1..318a77f 100644 --- a/progress.c +++ b/progress.c @@ -4,7 +4,7 @@ * Copyright (C) 1996-2000 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2001, 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -177,6 +177,11 @@ void show_progress(OFF_T ofs, OFF_T size) gettimeofday(&now, NULL); + if (INFO_GTE(PROGRESS, 2)) { + ofs = stats.total_transferred_size - size + ofs; + size = stats.total_size; + } + if (!ph_start.time.tv_sec) { int i; @@ -212,9 +217,5 @@ void show_progress(OFF_T ofs, OFF_T size) return; #endif - if (INFO_GTE(PROGRESS, 2)) { - rprint_progress(stats.total_transferred_size, - stats.total_size, &now, False); - } else - rprint_progress(ofs, size, &now, False); + rprint_progress(ofs, size, &now, False); } diff --git a/receiver.c b/receiver.c index 1e064d9..571b7da 100644 --- a/receiver.c +++ b/receiver.c @@ -3,7 +3,7 @@ * * Copyright (C) 1996-2000 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -30,6 +30,7 @@ extern int inc_recurse; extern int log_before_transfer; extern int stdout_format_has_i; extern int logfile_format_has_i; +extern int want_xattr_optim; extern int csum_length; extern int read_batch; extern int write_batch; @@ -113,9 +114,12 @@ int get_tmpname(char *fnametmp, const char *fname, BOOL make_unique) } } else f = fname; - if (*f == '.') /* avoid an extra leading dot for OS X's sake */ - f++; - fnametmp[length++] = '.'; + + if (!tmpdir) { /* using a tmpdir avoids the leading dot on our temp names */ + if (*f == '.') /* avoid an extra leading dot for OS X's sake */ + f++; + fnametmp[length++] = '.'; + } /* The maxname value is bufsize, and includes space for the '\0'. * NAME_MAX needs an extra -1 for the name's leading dot. */ @@ -581,7 +585,7 @@ int recv_files(int f_in, int f_out, char *local_name) #ifdef SUPPORT_XATTRS if (preserve_xattrs && iflags & ITEM_REPORT_XATTR && do_xfers - && (protocol_version < 31 || !BITS_SET(iflags, ITEM_XNAME_FOLLOWS|ITEM_LOCAL_CHANGE))) + && !(want_xattr_optim && BITS_SET(iflags, ITEM_XNAME_FOLLOWS|ITEM_LOCAL_CHANGE))) recv_xattr_request(file, f_in); #endif diff --git a/rounding.c b/rounding.c index 7e2f207..ea9604f 100644 --- a/rounding.c +++ b/rounding.c @@ -1,7 +1,7 @@ /* * A pre-compilation helper program to aid in the creation of rounding.h. * - * Copyright (C) 2007-2013 Wayne Davison + * Copyright (C) 2007-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/rsync.c b/rsync.c index 1f9b029..c498c44 100644 --- a/rsync.c +++ b/rsync.c @@ -3,7 +3,7 @@ * * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -489,31 +489,6 @@ int set_file_attrs(const char *fname, struct file_struct *file, stat_x *sxp, get_acl(fname, sxp); #endif -#ifdef SUPPORT_XATTRS - if (am_root < 0) - set_stat_xattr(fname, file, new_mode); - if (preserve_xattrs && fnamecmp) - set_xattr(fname, file, fnamecmp, sxp); -#endif - - if (!preserve_times - || (!(preserve_times & PRESERVE_DIR_TIMES) && S_ISDIR(sxp->st.st_mode)) - || (!(preserve_times & PRESERVE_LINK_TIMES) && S_ISLNK(sxp->st.st_mode))) - flags |= ATTRS_SKIP_MTIME; - if (!(flags & ATTRS_SKIP_MTIME) - && cmp_time(sxp->st.st_mtime, file->modtime) != 0) { - int ret = set_modtime(fname, file->modtime, F_MOD_NSEC(file), sxp->st.st_mode); - if (ret < 0) { - rsyserr(FERROR_XFER, errno, "failed to set times on %s", - full_fname(fname)); - goto cleanup; - } - if (ret == 0) /* ret == 1 if symlink could not be set */ - updated = 1; - else - file->flags |= FLAG_TIME_FAILED; - } - change_uid = am_root && uid_ndx && sxp->st.st_uid != (uid_t)F_OWNER(file); change_gid = gid_ndx && !(file->flags & FLAG_SKIP_GROUP) && sxp->st.st_gid != (gid_t)F_GROUP(file); @@ -561,6 +536,31 @@ int set_file_attrs(const char *fname, struct file_struct *file, stat_x *sxp, updated = 1; } +#ifdef SUPPORT_XATTRS + if (am_root < 0) + set_stat_xattr(fname, file, new_mode); + if (preserve_xattrs && fnamecmp) + set_xattr(fname, file, fnamecmp, sxp); +#endif + + if (!preserve_times + || (!(preserve_times & PRESERVE_DIR_TIMES) && S_ISDIR(sxp->st.st_mode)) + || (!(preserve_times & PRESERVE_LINK_TIMES) && S_ISLNK(sxp->st.st_mode))) + flags |= ATTRS_SKIP_MTIME; + if (!(flags & ATTRS_SKIP_MTIME) + && cmp_time(sxp->st.st_mtime, file->modtime) != 0) { + int ret = set_modtime(fname, file->modtime, F_MOD_NSEC(file), sxp->st.st_mode); + if (ret < 0) { + rsyserr(FERROR_XFER, errno, "failed to set times on %s", + full_fname(fname)); + goto cleanup; + } + if (ret == 0) /* ret == 1 if symlink could not be set */ + updated = 1; + else + file->flags |= FLAG_TIME_FAILED; + } + #ifdef SUPPORT_ACLS /* It's OK to call set_acl() now, even for a dir, as the generator * will enable owner-writability using chmod, if necessary. diff --git a/rsync.h b/rsync.h index fcb4c26..4fef882 100644 --- a/rsync.h +++ b/rsync.h @@ -2,7 +2,7 @@ * Copyright (C) 1996, 2000 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2001, 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -208,6 +208,7 @@ #define CFN_KEEP_TRAILING_SLASH (1<<1) #define CFN_DROP_TRAILING_DOT_DIR (1<<2) #define CFN_COLLAPSE_DOT_DOT_DIRS (1<<3) +#define CFN_REFUSE_DOT_DOT_DIRS (1<<4) #define SP_DEFAULT 0 #define SP_KEEP_DOT_DIRS (1<<0) @@ -787,6 +788,8 @@ extern int xattrs_ndx; #define DIR_FIRST_CHILD(a) (a)[1] #define DIR_NEXT_SIBLING(a) (a)[2] +#define IS_MISSING_FILE(statbuf) ((statbuf).st_mode == 0) + /* * Start the flist array at FLIST_START entries and grow it * by doubling until FLIST_LINEAR then grow by FLIST_LINEAR diff --git a/rsync.yo b/rsync.yo index a84b7bf..87028ca 100644 --- a/rsync.yo +++ b/rsync.yo @@ -1,5 +1,5 @@ mailto(rsync-bugs@samba.org) -manpage(rsync)(1)(28 Sep 2013)()() +manpage(rsync)(1)(22 Jun 2014)()() manpagename(rsync)(a fast, versatile, remote (and local) file-copying tool) manpagesynopsis() @@ -103,7 +103,10 @@ This would transfer all files matching the pattern *.c from the current directory to the directory src on the machine foo. If any of the files already exist on the remote system then the rsync remote-update protocol is used to update the file by sending only the -differences. See the tech report for details. +differences in the data. Note that the expansion of wildcards on the +commandline (*.c) into a list of files is handled by the shell before +it runs rsync and not by rsync itself (exactly the same as all other +posix-style programs). quote(tt(rsync -avz foo:src/bar /data/tmp)) @@ -513,6 +516,12 @@ fine-grained settings override the implied settings of bf(-v). Both bf(--info) and bf(--debug) have a way to ask for help that tells you exactly what flags are set for each increase in verbosity. +However, do keep in mind that a daemon's "max verbosity" setting will limit how +high of a level the various individual flags can be set on the daemon side. +For instance, if the max is 2, then any info and/or debug flag that is set to +a higher value than what would be set by bf(-vv) will be downgraded to the +bf(-vv) level in the daemon's logging. + dit(bf(--info=FLAGS)) This option lets you have fine-grained control over the information @@ -534,6 +543,7 @@ information on what is output and when. This option was added to 3.1.0, so an older rsync on the server side might reject your attempts at fine-grained control (if one or more flags needed to be send to the server and the server was too old to understand them). +See also the "max verbosity" caveat above when dealing with a daemon. dit(bf(--debug=FLAGS)) This option lets you have fine-grained control over the debug @@ -554,6 +564,7 @@ specified, especially those pertaining to I/O and buffer debugging. This option was added to 3.1.0, so an older rsync on the server side might reject your attempts at fine-grained control (if one or more flags needed to be send to the server and the server was too old to understand them). +See also the "max verbosity" caveat above when dealing with a daemon. dit(bf(--msgs2stderr)) This option changes rsync to send all its output directly to stderr rather than to send messages to the client side via the @@ -789,7 +800,7 @@ the destination and have a modified time that is newer than the source file. (If an existing destination file has a modification time equal to the source file's, it will be updated if the sizes are different.) -Note that this does not affect the copying of symlinks or other special +Note that this does not affect the copying of dirs, symlinks, or other special files. Also, a difference of file format between the sender and receiver is always considered to be important enough for an update, no matter what date is on the objects. In other words, if the source has a directory @@ -1167,6 +1178,17 @@ it is preserving modification times (see bf(--times)). If NFS is sharing the directories on the receiving side, it is a good idea to use bf(-O). This option is inferred if you use bf(--backup) without bf(--backup-dir). +This option also has the side-effect of avoiding early creation of directories +in incremental recursion copies. The default bf(--inc-recursive) copying +normally does an early-create pass of all the sub-directories in a parent +directory in order for it to be able to then set the modify time of the parent +directory right away (without having to delay that until a bunch of recursive +copying has finished). This early-create idiom is not necessary if directory +modify times are not being preserved, so it is skipped. Since early-create +directories don't have accurate mode, mtime, or ownership, the use of this +option can help when someone wants to avoid these partially-finished +directories. + dit(bf(-J, --omit-link-times)) This tells rsync to omit symlinks when it is preserving modification times (see bf(--times)). @@ -1726,6 +1748,9 @@ dit(bf(-T, --temp-dir=DIR)) This option instructs rsync to use DIR as a scratch directory when creating temporary copies of the files transferred on the receiving side. The default behavior is to create each temporary file in the same directory as the associated destination file. +Beginning with rsync 3.1.1, the temp-file names inside the specified DIR will +not be prefixed with an extra dot (though they will still have a random suffix +added). This option is most often used when the receiving disk partition does not have enough free space to hold a copy of the largest file in the transfer. @@ -1856,7 +1881,20 @@ being transmitted -- something that is useful over a slow connection. Note that this option typically achieves better compression ratios than can be achieved by using a compressing remote shell or a compressing transport because it takes advantage of the implicit information in the matching data -blocks that are not explicitly sent over the connection. +blocks that are not explicitly sent over the connection. This matching-data +compression comes at a cost of CPU, though, and can be disabled by repeating +the bf(-z) option, but only if both sides are at least version 3.1.1. + +Note that if your version of rsync was compiled with an external zlib (instead +of the zlib that comes packaged with rsync) then it will not support the +old-style compression, only the new-style (repeated-option) compression. In +the future this new-style compression will likely become the default. + +The client rsync requests new-style compression on the server via the +bf(--new-compress) option, so if you see that option rejected it means that +the server is not new enough to support bf(-zz). Rsync also accepts the +bf(--old-compress) option for a future time when new-style compression +becomes the default. See the bf(--skip-compress) option for the default list of file suffixes that will not be compressed. @@ -2743,6 +2781,10 @@ itemization( it() a '[' introduces a character class, such as [a-z] or [[:alpha:]]. it() in a wildcard pattern, a backslash can be used to escape a wildcard character, but it is matched literally when no wildcards are present. + This means that there is an extra level of backslash removal when a + pattern contains wildcard characters compared to a pattern that has none. + e.g. if you add a wildcard to "foo\bar" (which matches the backslash) you + would need to use "foo\\bar*" to avoid the "\b" becoming just "b". it() if the pattern contains a / (not counting a trailing /) or a "**", then it is matched against the full pathname, including any leading directories. If the pattern doesn't contain a / or a "**", then it is @@ -3361,7 +3403,7 @@ url(http://rsync.samba.org/)(http://rsync.samba.org/) manpagesection(VERSION) -This man page is current for version 3.1.0 of rsync. +This man page is current for version 3.1.1 of rsync. manpagesection(INTERNAL OPTIONS) diff --git a/rsyncd.conf.yo b/rsyncd.conf.yo index 203e456..9f9f6ac 100644 --- a/rsyncd.conf.yo +++ b/rsyncd.conf.yo @@ -1,5 +1,5 @@ mailto(rsync-bugs@samba.org) -manpage(rsyncd.conf)(5)(28 Sep 2013)()() +manpage(rsyncd.conf)(5)(22 Jun 2014)()() manpagename(rsyncd.conf)(configuration file for rsync in daemon mode) manpagesynopsis() @@ -318,6 +318,13 @@ the maximum amount of verbose information that you'll allow the daemon to generate (since the information goes into the log file). The default is 1, which allows the client to request one level of verbosity. +This also affects the user's ability to request higher levels of bf(--info) and +bf(--debug) logging. If the max value is 2, then no info and/or debug value +that is higher than what would be set by bf(-vv) will be honored by the daemon +in its logging. To see how high of a verbosity level you need to accept for a +particular info/debug level, refer to "rsync --info=help" and "rsync --debug=help". +For instance, it takes max-verbosity 4 to be able to output debug TIME2 and FLIST3. + dit(bf(lock file)) This parameter specifies the file to use to support the "max connections" parameter. The rsync daemon uses record locking on this file to ensure that the max connections limit is not @@ -894,7 +901,7 @@ url(http://rsync.samba.org/)(http://rsync.samba.org/) manpagesection(VERSION) -This man page is current for version 3.1.0 of rsync. +This man page is current for version 3.1.1 of rsync. manpagesection(CREDITS) diff --git a/sender.c b/sender.c index 12242ab..5adc2fd 100644 --- a/sender.c +++ b/sender.c @@ -3,7 +3,7 @@ * * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -29,6 +29,7 @@ extern int inc_recurse; extern int log_before_transfer; extern int stdout_format_has_i; extern int logfile_format_has_i; +extern int want_xattr_optim; extern int csum_length; extern int append_mode; extern int io_error; @@ -147,7 +148,7 @@ void successful_send(int ndx) || (NSEC_BUMP(file) && (uint32)st.ST_MTIME_NSEC != F_MOD_NSEC(file)) #endif ) { - rprintf(FERROR, "ERROR: Skipping sender remove for changed file: %s\n", fname); + rprintf(FERROR_XFER, "ERROR: Skipping sender remove for changed file: %s\n", fname); return; } @@ -157,7 +158,7 @@ void successful_send(int ndx) if (errno == ENOENT) rprintf(FINFO, "sender file already removed: %s\n", fname); else - rsyserr(FERROR, errno, "sender failed to %s %s", failed_op, fname); + rsyserr(FERROR_XFER, errno, "sender failed to %s %s", failed_op, fname); } else { if (INFO_GTE(REMOVE, 1)) rprintf(FINFO, "sender removed %s\n", fname); @@ -178,7 +179,7 @@ static void write_ndx_and_attrs(int f_out, int ndx, int iflags, write_vstring(f_out, buf, len); #ifdef SUPPORT_XATTRS if (preserve_xattrs && iflags & ITEM_REPORT_XATTR && do_xfers - && (protocol_version < 31 || !BITS_SET(iflags, ITEM_XNAME_FOLLOWS|ITEM_LOCAL_CHANGE))) + && !(want_xattr_optim && BITS_SET(iflags, ITEM_XNAME_FOLLOWS|ITEM_LOCAL_CHANGE))) send_xattr_request(fname, file, f_out); #endif } @@ -260,7 +261,7 @@ void send_files(int f_in, int f_out) #ifdef SUPPORT_XATTRS if (preserve_xattrs && iflags & ITEM_REPORT_XATTR && do_xfers - && (protocol_version < 31 || !BITS_SET(iflags, ITEM_XNAME_FOLLOWS|ITEM_LOCAL_CHANGE))) + && !(want_xattr_optim && BITS_SET(iflags, ITEM_XNAME_FOLLOWS|ITEM_LOCAL_CHANGE))) recv_xattr_request(file, f_in); #endif diff --git a/socket.c b/socket.c index 6f4da2a..3f5786b 100644 --- a/socket.c +++ b/socket.c @@ -3,7 +3,7 @@ * * Copyright (C) 1992-2001 Andrew Tridgell * Copyright (C) 2001, 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/support/rrsync b/support/rrsync index e3c5d41..e95f992 100644 --- a/support/rrsync +++ b/support/rrsync @@ -51,7 +51,7 @@ die "$0 -ro: sending to read-only server not allowed\n" if $ro && !$am_sender; # To disable a short-named option, add its letter to this string: our $short_disabled = 's'; -our $short_no_arg = 'ACDEHIKLORSWXbcdgklmnoprstuvxz'; # DO NOT REMOVE ANY +our $short_no_arg = 'ACDEHIJKLORSWXbcdgklmnoprstuvxyz'; # DO NOT REMOVE ANY our $short_with_num = 'B'; # DO NOT REMOVE ANY # To disable a long-named option, change its value to a -1. The values mean: @@ -67,6 +67,7 @@ our %long_opt = ( 'copy-dest' => 2, 'copy-unsafe-links' => 0, 'daemon' => -1, + 'debug' => 1, 'delay-updates' => 0, 'delete' => 0, 'delete-after' => 0, @@ -86,6 +87,7 @@ our %long_opt = ( 'ignore-errors' => 0, 'ignore-existing' => 0, 'ignore-missing-args' => 0, + 'info' => 1, 'inplace' => 0, 'link-dest' => 2, 'list-only' => 0, @@ -95,6 +97,7 @@ our %long_opt = ( 'max-size' => 1, 'min-size' => 1, 'modify-window' => 1, + 'new-compress' => 0, 'no-implied-dirs' => 0, 'no-r' => 0, 'no-relative' => 0, @@ -103,6 +106,7 @@ our %long_opt = ( 'only-write-batch' => 1, 'partial' => 0, 'partial-dir' => 2, + 'preallocate' => 0, 'remove-sent-files' => $ro ? -1 : 0, 'remove-source-files' => $ro ? -1 : 0, 'safe-links' => 0, diff --git a/support/rsync-no-vanished b/support/rsync-no-vanished new file mode 100755 index 0000000..1cce75c --- /dev/null +++ b/support/rsync-no-vanished @@ -0,0 +1,15 @@ +#!/bin/bash + +IGNOREEXIT=24 +IGNOREOUT='^(file has vanished: |rsync warning: some files vanished before they could be transferred)' + +set -o pipefail + +rsync "${@}" 2>&1 | (egrep -v "$IGNOREOUT" || true) +ret=$? + +if [[ $ret == $IGNOREEXIT ]]; then + ret=0 +fi + +exit $ret diff --git a/syscall.c b/syscall.c index fd23d15..8f1d2fe 100644 --- a/syscall.c +++ b/syscall.c @@ -4,7 +4,7 @@ * * Copyright (C) 1998 Andrew Tridgell * Copyright (C) 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -147,13 +147,13 @@ int do_mknod(const char *pathname, mode_t mode, dev_t dev) if (S_ISSOCK(mode)) { int sock; struct sockaddr_un saddr; + unsigned int len = strlcpy(saddr.sun_path, pathname, sizeof saddr.sun_path); + if (len >= sizeof saddr.sun_path) { + errno = ENAMETOOLONG; + return -1; + } #ifdef HAVE_SOCKADDR_UN_LEN - unsigned int len = -#endif - strlcpy(saddr.sun_path, pathname, sizeof saddr.sun_path); -#ifdef HAVE_SOCKADDR_UN_LEN - saddr.sun_len = len >= sizeof saddr.sun_path - ? sizeof saddr.sun_path : len + 1; + saddr.sun_len = len + 1; #endif saddr.sun_family = AF_UNIX; diff --git a/t_stub.c b/t_stub.c index e5e723e..c230396 100644 --- a/t_stub.c +++ b/t_stub.c @@ -3,7 +3,7 @@ * functions, so that module test harnesses can run standalone. * * Copyright (C) 2001, 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/t_unsafe.c b/t_unsafe.c index 73bd549..72339d0 100644 --- a/t_unsafe.c +++ b/t_unsafe.c @@ -2,7 +2,7 @@ * Test harness for unsafe_symlink(). Not linked into rsync itself. * * Copyright (C) 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/testsuite/daemon-gzip-download.test b/testsuite/daemon-gzip-download.test index 89a112f..57dd820 100644 --- a/testsuite/daemon-gzip-download.test +++ b/testsuite/daemon-gzip-download.test @@ -31,7 +31,7 @@ hands_setup # Build chkdir with a normal rsync and an --exclude. $RSYNC -av --exclude=foobar.baz "$fromdir/" "$chkdir/" -checkit "$RSYNC -avvvvz localhost::test-from/ '$todir/'" "$chkdir" "$todir" +checkit "$RSYNC -avvvvzz localhost::test-from/ '$todir/'" "$chkdir" "$todir" # The script would have aborted on error, so getting here means we've won. exit 0 diff --git a/testsuite/daemon-gzip-upload.test b/testsuite/daemon-gzip-upload.test index 11c52ba..b2110ea 100644 --- a/testsuite/daemon-gzip-upload.test +++ b/testsuite/daemon-gzip-upload.test @@ -25,7 +25,7 @@ hands_setup # Build chkdir with a normal rsync and an --exclude. $RSYNC -av --exclude=foobar.baz "$fromdir/" "$chkdir/" -checkit "'$ignore23' $RSYNC -avvvvz '$fromdir/' localhost::test-to/" "$chkdir" "$todir" +checkit "'$ignore23' $RSYNC -avvvvzz '$fromdir/' localhost::test-to/" "$chkdir" "$todir" # The script would have aborted on error, so getting here means we've won. exit 0 diff --git a/tls.c b/tls.c index ca3ec7c..2197d16 100644 --- a/tls.c +++ b/tls.c @@ -2,7 +2,7 @@ * Trivial ls for comparing two directories after running an rsync. * * Copyright (C) 2001, 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/token.c b/token.c index 953db78..8cc5532 100644 --- a/token.c +++ b/token.c @@ -3,7 +3,7 @@ * * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -23,10 +23,6 @@ #include "itypes.h" #include -#ifndef Z_INSERT_ONLY -#define Z_INSERT_ONLY Z_SYNC_FLUSH -#endif - extern int do_compression; extern int protocol_version; extern int module_id; @@ -406,9 +402,10 @@ send_deflated_token(int f, int32 token, struct map_struct *buf, OFF_T offset, if (token == -1) { /* end of file - clean up */ write_byte(f, END_FLAG); - } else if (token != -2) { + } else if (token != -2 && do_compression == 1) { /* Add the data in the current block to the compressor's * history and hash table. */ +#ifndef EXTERNAL_ZLIB do { /* Break up long sections in the same way that * see_deflate_token() does. */ @@ -418,17 +415,20 @@ send_deflated_token(int f, int32 token, struct map_struct *buf, OFF_T offset, tx_strm.avail_in = n1; if (protocol_version >= 31) /* Newer protocols avoid a data-duplicating bug */ offset += n1; - do { - tx_strm.next_out = (Bytef *) obuf; - tx_strm.avail_out = AVAIL_OUT_SIZE(CHUNK_SIZE); - r = deflate(&tx_strm, Z_INSERT_ONLY); - if (r != Z_OK) { - rprintf(FERROR, "deflate on token returned %d (%d bytes left)\n", - r, tx_strm.avail_in); - exit_cleanup(RERR_STREAMIO); - } - } while (tx_strm.avail_in != 0); + tx_strm.next_out = (Bytef *) obuf; + tx_strm.avail_out = AVAIL_OUT_SIZE(CHUNK_SIZE); + r = deflate(&tx_strm, Z_INSERT_ONLY); + if (r != Z_OK || tx_strm.avail_in != 0) { + rprintf(FERROR, "deflate on token returned %d (%d bytes left)\n", + r, tx_strm.avail_in); + exit_cleanup(RERR_STREAMIO); + } } while (toklen > 0); +#else + toklen++; + rprintf(FERROR, "Impossible error in external-zlib code (1).\n"); + exit_cleanup(RERR_STREAMIO); +#endif } } @@ -579,6 +579,7 @@ static int32 recv_deflated_token(int f, char **data) */ static void see_deflate_token(char *buf, int32 len) { +#ifndef EXTERNAL_ZLIB int r; int32 blklen; unsigned char hdr[5]; @@ -616,6 +617,11 @@ static void see_deflate_token(char *buf, int32 len) exit_cleanup(RERR_STREAMIO); } } while (len || rx_strm.avail_out == 0); +#else + buf++; len++; + rprintf(FERROR, "Impossible error in external-zlib code (2).\n"); + exit_cleanup(RERR_STREAMIO); +#endif } /** @@ -655,6 +661,6 @@ int32 recv_token(int f, char **data) */ void see_token(char *data, int32 toklen) { - if (do_compression) + if (do_compression == 1) see_deflate_token(data, toklen); } diff --git a/trimslash.c b/trimslash.c index 7694eb1..8ee4766 100644 --- a/trimslash.c +++ b/trimslash.c @@ -2,7 +2,7 @@ * Simple utility used only by the test harness. * * Copyright (C) 2002 Martin Pool - * Copyright (C) 2003 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/uidlist.c b/uidlist.c index a4bc8ed..641f147 100644 --- a/uidlist.c +++ b/uidlist.c @@ -3,7 +3,7 @@ * * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras - * Copyright (C) 2004-2013 Wayne Davison + * Copyright (C) 2004-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -473,12 +473,15 @@ void parse_name_map(char *map, BOOL usernames) usernames ? "user" : "group", cp); exit_cleanup(RERR_SYNTAX); } - if (dash) + if (dash) { + *dash = '\0'; noiu.max_id = id_parse(dash+1); - else + } else noiu.max_id = 0; flags = 0; id1 = id_parse(cp); + if (dash) + *dash = '-'; } else if (strpbrk(cp, "*[?")) { flags = NFLAGS_WILD_NAME_MATCH; noiu.name = cp; diff --git a/util.c b/util.c index c943ce0..05aa86a 100644 --- a/util.c +++ b/util.c @@ -4,7 +4,7 @@ * Copyright (C) 1996-2000 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2001, 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -25,6 +25,7 @@ #include "itypes.h" #include "inums.h" +extern int dry_run; extern int module_id; extern int protect_args; extern int modify_window; @@ -197,7 +198,15 @@ int make_path(char *fname, int flags) /* Try to find an existing dir, starting from the deepest dir. */ for (p = end; ; ) { - if (do_mkdir(fname, ACCESSPERMS) == 0) { + if (dry_run) { + STRUCT_STAT st; + if (do_stat(fname, &st) == 0) { + if (S_ISDIR(st.st_mode)) + errno = EEXIST; + else + errno = ENOTDIR; + } + } else if (do_mkdir(fname, ACCESSPERMS) == 0) { ret++; break; } @@ -208,12 +217,14 @@ int make_path(char *fname, int flags) } while (1) { if (p == fname) { - ret = -ret - 1; + /* We got a relative path that doesn't exist, so assume that '.' + * is there and just break out and create the whole thing. */ + p = NULL; goto double_break; } if (*--p == '/') { if (p == fname) { - ret = -ret - 1; /* impossible... */ + /* We reached the "/" dir, which we assume is there. */ goto double_break; } *p = '\0'; @@ -225,7 +236,10 @@ int make_path(char *fname, int flags) /* Make all the dirs that we didn't find on the way here. */ while (p != end) { - *p = '/'; + if (p) + *p = '/'; + else + p = fname; p += strlen(p); if (ret < 0) /* Skip mkdir on error, but keep restoring the path. */ continue; @@ -858,7 +872,7 @@ int count_dir_elements(const char *p) * CFN_KEEP_TRAILING_SLASH is flagged, and will also collapse ".." elements * (except at the start) if CFN_COLLAPSE_DOT_DOT_DIRS is flagged. If the * resulting name would be empty, returns ".". */ -unsigned int clean_fname(char *name, int flags) +int clean_fname(char *name, int flags) { char *limit = name - 1, *t = name, *f = name; int anchored; @@ -866,6 +880,8 @@ unsigned int clean_fname(char *name, int flags) if (!name) return 0; +#define DOT_IS_DOT_DOT_DIR(bp) (bp[1] == '.' && (bp[2] == '/' || !bp[2])) + if ((anchored = *f == '/') != 0) { *t++ = *f++; #ifdef __CYGWIN__ @@ -878,7 +894,8 @@ unsigned int clean_fname(char *name, int flags) } else if (flags & CFN_KEEP_DOT_DIRS && *f == '.' && f[1] == '/') { *t++ = *f++; *t++ = *f++; - } + } else if (flags & CFN_REFUSE_DOT_DOT_DIRS && *f == '.' && DOT_IS_DOT_DOT_DIR(f)) + return -1; while (*f) { /* discard extra slashes */ if (*f == '/') { @@ -894,9 +911,10 @@ unsigned int clean_fname(char *name, int flags) if (f[1] == '\0' && flags & CFN_DROP_TRAILING_DOT_DIR) break; /* collapse ".." dirs */ - if (flags & CFN_COLLAPSE_DOT_DOT_DIRS - && f[1] == '.' && (f[2] == '/' || !f[2])) { + if (flags & (CFN_COLLAPSE_DOT_DOT_DIRS|CFN_REFUSE_DOT_DOT_DIRS) && DOT_IS_DOT_DOT_DIR(f)) { char *s = t - 1; + if (flags & CFN_REFUSE_DOT_DOT_DIRS) + return -1; if (s == name && anchored) { f += 2; continue; @@ -919,6 +937,8 @@ unsigned int clean_fname(char *name, int flags) *t++ = '.'; *t = '\0'; +#undef DOT_IS_DOT_DOT_DIR + return t - name; } @@ -1305,16 +1325,17 @@ char *timestring(time_t t) int cmp_time(time_t file1, time_t file2) { if (file2 > file1) { - if (file2 - file1 <= modify_window) - return 0; - return -1; + /* The final comparison makes sure that modify_window doesn't overflow a + * time_t, which would mean that file2 must be in the equality window. */ + if (!modify_window || (file2 > file1 + modify_window && file1 + modify_window > file1)) + return -1; + } else if (file1 > file2) { + if (!modify_window || (file1 > file2 + modify_window && file2 + modify_window > file2)) + return 1; } - if (file1 - file2 <= modify_window) - return 0; - return 1; + return 0; } - #ifdef __INSURE__XX #include diff --git a/util2.c b/util2.c index 85b1903..6ffbcec 100644 --- a/util2.c +++ b/util2.c @@ -4,7 +4,7 @@ * Copyright (C) 1996-2000 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras * Copyright (C) 2001, 2002 Martin Pool - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/wildtest.c b/wildtest.c index 2621c15..19809b6 100644 --- a/wildtest.c +++ b/wildtest.c @@ -1,7 +1,7 @@ /* * Test suite for the wildmatch code. * - * Copyright (C) 2003-2013 Wayne Davison + * Copyright (C) 2003-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by diff --git a/xattrs.c b/xattrs.c index df2ea82..57d40e1 100644 --- a/xattrs.c +++ b/xattrs.c @@ -3,7 +3,7 @@ * Written by Jay Fenlason, vaguely based on the ACLs patch. * * Copyright (C) 2004 Red Hat, Inc. - * Copyright (C) 2006-2013 Wayne Davison + * Copyright (C) 2006-2014 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -308,7 +308,8 @@ int get_xattr(const char *fname, stat_x *sxp) if (!preserve_devices) #endif return 0; - } + } else if (IS_MISSING_FILE(sxp->st)) + return 0; if (rsync_xal_get(fname, sxp->xattr) < 0) { free_xattr(sxp); @@ -451,7 +452,7 @@ int send_xattr(int f, stat_x *sxp) if (rxa->datum_len > MAX_FULL_DATUM) write_buf(f, rxa->datum + 1, MAX_DIGEST_LEN); else - write_buf(f, rxa->datum, rxa->datum_len); + write_bigbuf(f, rxa->datum, rxa->datum_len); } ndx = rsync_xal_l.count; /* pre-incremented count */ rsync_xal_store(sxp->xattr); /* adds item to rsync_xal_l */ @@ -579,7 +580,7 @@ void send_xattr_request(const char *fname, struct file_struct *file, int f_out) } write_varint(f_out, len); /* length might have changed! */ - write_buf(f_out, ptr, len); + write_bigbuf(f_out, ptr, len); free(ptr); } } @@ -610,9 +611,10 @@ int recv_xattr_request(struct file_struct *file, int f_in) num = 0; while ((rel_pos = read_varint(f_in)) != 0) { num += rel_pos; - while (cnt && rxa->num < num) { - rxa++; - cnt--; + /* Note that the sender-related num values may not be in order on the receiver! */ + while (cnt && (am_sender ? rxa->num < num : rxa->num != num)) { + rxa++; + cnt--; } if (!cnt || rxa->num != num) { rprintf(FERROR, "[%s] could not find xattr #%d for %s\n", diff -upN a/config.h.in b/config.h.in --- a/config.h.in +++ b/config.h.in @@ -23,6 +23,9 @@ /* Define to 1 if using `alloca.c'. */ #undef C_ALLOCA +/* Define to 1 if using external zlib */ +#undef EXTERNAL_ZLIB + /* Used to make "checker" understand that FD_ZERO() clears memory. */ #undef FORCE_FD_ZERO_MEMSET @@ -195,6 +198,9 @@ /* Define to 1 if you have the `acl' library (-lacl). */ #undef HAVE_LIBACL +/* Define to 1 if you have the `attr' library (-lattr). */ +#undef HAVE_LIBATTR + /* Define to 1 if you have the header file. */ #undef HAVE_LIBCHARSET_H diff -upN a/configure.sh b/configure.sh --- a/configure.sh +++ b/configure.sh @@ -2429,7 +2429,7 @@ ac_config_headers="$ac_config_headers co -RSYNC_VERSION=3.1.0 +RSYNC_VERSION=3.1.1 { $as_echo "$as_me:${as_lineno-$LINENO}: Configuring rsync $RSYNC_VERSION" >&5 $as_echo "$as_me: Configuring rsync $RSYNC_VERSION" >&6;} @@ -7915,6 +7915,7 @@ else main() { unlink(FILENAME); if (symlink("conftest.no-such", FILENAME) < 0) abort(); + unlink(FILENAME "2"); if (link(FILENAME, FILENAME "2") < 0) exit(1); exit(0); } @@ -7958,6 +7959,7 @@ else main() { unlink(FILENAME); if (mkfifo(FILENAME, 0777) < 0) abort(); + unlink(FILENAME "2"); if (link(FILENAME, FILENAME "2") < 0) exit(1); exit(0); } @@ -7997,11 +7999,7 @@ else main() { int fd[2]; -#ifdef __CYGWIN__ - exit(1); -#else exit((socketpair(AF_UNIX, SOCK_STREAM, 0, fd) != -1) ? 0 : 1); -#endif } _ACEOF if ac_fn_c_try_run "$LINENO"; then : @@ -8106,7 +8104,7 @@ if test x"$with_included_popt" = x"yes"; { $as_echo "$as_me:${as_lineno-$LINENO}: result: $srcdir/popt" >&5 $as_echo "$srcdir/popt" >&6; } BUILD_POPT='$(popt_OBJS)' - CFLAGS="$CFLAGS -I$srcdir/popt" + CFLAGS="-I$srcdir/popt $CFLAGS" if test x"$ALLOCA" != x then # this can be removed when/if we add an included alloca.c; @@ -8181,8 +8179,11 @@ if test x"$with_included_zlib" = x"yes"; { $as_echo "$as_me:${as_lineno-$LINENO}: result: $srcdir/zlib" >&5 $as_echo "$srcdir/zlib" >&6; } BUILD_ZLIB='$(zlib_OBJS)' - CFLAGS="$CFLAGS -I$srcdir/zlib" + CFLAGS="-I$srcdir/zlib $CFLAGS" else + +$as_echo "#define EXTERNAL_ZLIB 1" >>confdefs.h + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi @@ -8764,7 +8765,7 @@ _ACEOF fi - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ACL support" >&5 + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ACL support" >&5 $as_echo_n "checking for ACL support... " >&6; } if ${samba_cv_HAVE_POSIX_ACLS+:} false; then : $as_echo_n "(cached) " >&6 @@ -8878,6 +8879,51 @@ $as_echo "#define HAVE_LINUX_XATTRS 1" > $as_echo "#define NO_SYMLINK_USER_XATTRS 1" >>confdefs.h + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for getxattr in -lattr" >&5 +$as_echo_n "checking for getxattr in -lattr... " >&6; } +if ${ac_cv_lib_attr_getxattr+:} false; then : + $as_echo_n "(cached) " >&6 +else + ac_check_lib_save_LIBS=$LIBS +LIBS="-lattr $LIBS" +cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +/* Override any GCC internal prototype to avoid an error. + Use char because int might match the return type of a GCC + builtin and then its argument prototype would still apply. */ +#ifdef __cplusplus +extern "C" +#endif +char getxattr (); +int +main () +{ +return getxattr (); + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + ac_cv_lib_attr_getxattr=yes +else + ac_cv_lib_attr_getxattr=no +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext +LIBS=$ac_check_lib_save_LIBS +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_attr_getxattr" >&5 +$as_echo "$ac_cv_lib_attr_getxattr" >&6; } +if test "x$ac_cv_lib_attr_getxattr" = xyes; then : + cat >>confdefs.h <<_ACEOF +#define HAVE_LIBATTR 1 +_ACEOF + + LIBS="-lattr $LIBS" + +fi + ;; darwin*) { $as_echo "$as_me:${as_lineno-$LINENO}: result: Using OS X xattrs" >&5 @@ -10275,3 +10321,11 @@ $as_echo "" >&6; } $as_echo " rsync ${RSYNC_VERSION} configuration successful" >&6; } { $as_echo "$as_me:${as_lineno-$LINENO}: result: " >&5 $as_echo "" >&6; } +if test x$HAVE_YODL2MAN != x1; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: Note that yodl2man was not found, so pre-existing manpage files will be" >&5 +$as_echo " Note that yodl2man was not found, so pre-existing manpage files will be" >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: result: used w/o change (if available) -- no .yo file changes will be used." >&5 +$as_echo " used w/o change (if available) -- no .yo file changes will be used." >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: result: " >&5 +$as_echo "" >&6; } +fi diff -upN a/proto.h b/proto.h --- a/proto.h +++ b/proto.h @@ -22,7 +22,7 @@ void check_batch_flags(void); void write_batch_shell_file(int argc, char *argv[], int file_arg_cnt); uint32 get_checksum1(char *buf1, int32 len); void get_checksum2(char *buf, int32 len, char *sum); -void file_checksum(char *fname, char *sum, OFF_T size); +void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum); void sum_init(int seed); void sum_update(const char *p, int32 len); int sum_end(char *sum); @@ -106,7 +106,7 @@ void itemize(const char *fnamecmp, struc stat_x *sxp, int32 iflags, uchar fnamecmp_type, const char *xname); int unchanged_file(char *fn, struct file_struct *file, STRUCT_STAT *st); -int atomic_create(struct file_struct *file, char *fname, const char *lnk, +int atomic_create(struct file_struct *file, char *fname, const char *slnk, const char *hlnk, dev_t rdev, stat_x *sxp, int del_for_flag); void check_for_finished_files(int itemizing, enum logcode code, int check_redo); void generate_files(int f_out, const char *local_name); @@ -166,6 +166,7 @@ void write_int(int f, int32 x); void write_varint(int f, int32 x); void write_varlong(int f, int64 x, uchar min_bytes); void write_longint(int f, int64 x); +void write_bigbuf(int f, const char *buf, size_t len); void write_buf(int f, const char *buf, size_t len); void write_sbuf(int f, const char *buf); void write_byte(int f, uchar c); @@ -368,7 +369,7 @@ void strlower(char *s); size_t pathjoin(char *dest, size_t destsize, const char *p1, const char *p2); size_t stringjoin(char *dest, size_t destsize, ...); int count_dir_elements(const char *p); -unsigned int clean_fname(char *name, int flags); +int clean_fname(char *name, int flags); char *sanitize_path(char *dest, const char *p, const char *rootdir, int depth, int flags); int change_dir(const char *dir, int set_path_only); diff -upN a/rsync.1 b/rsync.1 --- a/rsync.1 +++ b/rsync.1 @@ -1,4 +1,4 @@ -.TH "rsync" "1" "28 Sep 2013" "" "" +.TH "rsync" "1" "22 Jun 2014" "" "" .SH "NAME" rsync \- a fast, versatile, remote (and local) file\-copying tool .SH "SYNOPSIS" @@ -120,7 +120,10 @@ This would transfer all files matching t current directory to the directory src on the machine foo. If any of the files already exist on the remote system then the rsync remote\-update protocol is used to update the file by sending only the -differences. See the tech report for details. +differences in the data. Note that the expansion of wildcards on the +commandline (*.c) into a list of files is handled by the shell before +it runs rsync and not by rsync itself (exactly the same as all other +posix\-style programs). .PP .RS \f(CWrsync \-avz foo:src/bar /data/tmp\fP @@ -598,6 +601,12 @@ fine\-grained settings override the impl \fB\-\-info\fP and \fB\-\-debug\fP have a way to ask for help that tells you exactly what flags are set for each increase in verbosity. .IP +However, do keep in mind that a daemon\(cq\&s \(dq\&max verbosity\(dq\& setting will limit how +high of a level the various individual flags can be set on the daemon side. +For instance, if the max is 2, then any info and/or debug flag that is set to +a higher value than what would be set by \fB\-vv\fP will be downgraded to the +\fB\-vv\fP level in the daemon\(cq\&s logging. +.IP .IP "\fB\-\-info=FLAGS\fP" This option lets you have fine\-grained control over the information @@ -622,6 +631,7 @@ information on what is output and when. This option was added to 3.1.0, so an older rsync on the server side might reject your attempts at fine\-grained control (if one or more flags needed to be send to the server and the server was too old to understand them). +See also the \(dq\&max verbosity\(dq\& caveat above when dealing with a daemon. .IP .IP "\fB\-\-debug=FLAGS\fP" This option lets you have fine\-grained control over the debug @@ -645,6 +655,7 @@ specified, especially those pertaining t This option was added to 3.1.0, so an older rsync on the server side might reject your attempts at fine\-grained control (if one or more flags needed to be send to the server and the server was too old to understand them). +See also the \(dq\&max verbosity\(dq\& caveat above when dealing with a daemon. .IP .IP "\fB\-\-msgs2stderr\fP" This option changes rsync to send all its output @@ -910,7 +921,7 @@ the destination and have a modified time file. (If an existing destination file has a modification time equal to the source file\(cq\&s, it will be updated if the sizes are different.) .IP -Note that this does not affect the copying of symlinks or other special +Note that this does not affect the copying of dirs, symlinks, or other special files. Also, a difference of file format between the sender and receiver is always considered to be important enough for an update, no matter what date is on the objects. In other words, if the source has a directory @@ -1342,6 +1353,17 @@ it is preserving modification times (see the directories on the receiving side, it is a good idea to use \fB\-O\fP. This option is inferred if you use \fB\-\-backup\fP without \fB\-\-backup\-dir\fP. .IP +This option also has the side\-effect of avoiding early creation of directories +in incremental recursion copies. The default \fB\-\-inc\-recursive\fP copying +normally does an early\-create pass of all the sub\-directories in a parent +directory in order for it to be able to then set the modify time of the parent +directory right away (without having to delay that until a bunch of recursive +copying has finished). This early\-create idiom is not necessary if directory +modify times are not being preserved, so it is skipped. Since early\-create +directories don\(cq\&t have accurate mode, mtime, or ownership, the use of this +option can help when someone wants to avoid these partially\-finished +directories. +.IP .IP "\fB\-J, \-\-omit\-link\-times\fP" This tells rsync to omit symlinks when it is preserving modification times (see \fB\-\-times\fP). @@ -1977,6 +1999,9 @@ This option instructs rsync to use DIR a scratch directory when creating temporary copies of the files transferred on the receiving side. The default behavior is to create each temporary file in the same directory as the associated destination file. +Beginning with rsync 3.1.1, the temp\-file names inside the specified DIR will +not be prefixed with an extra dot (though they will still have a random suffix +added). .IP This option is most often used when the receiving disk partition does not have enough free space to hold a copy of the largest file in the transfer. @@ -2115,7 +2140,20 @@ being transmitted \-\- something that is Note that this option typically achieves better compression ratios than can be achieved by using a compressing remote shell or a compressing transport because it takes advantage of the implicit information in the matching data -blocks that are not explicitly sent over the connection. +blocks that are not explicitly sent over the connection. This matching\-data +compression comes at a cost of CPU, though, and can be disabled by repeating +the \fB\-z\fP option, but only if both sides are at least version 3.1.1. +.IP +Note that if your version of rsync was compiled with an external zlib (instead +of the zlib that comes packaged with rsync) then it will not support the +old\-style compression, only the new\-style (repeated\-option) compression. In +the future this new\-style compression will likely become the default. +.IP +The client rsync requests new\-style compression on the server via the +\fB\-\-new\-compress\fP option, so if you see that option rejected it means that +the server is not new enough to support \fB\-zz\fP. Rsync also accepts the +\fB\-\-old\-compress\fP option for a future time when new\-style compression +becomes the default. .IP See the \fB\-\-skip\-compress\fP option for the default list of file suffixes that will not be compressed. @@ -3138,6 +3176,10 @@ a \(cq\&[\(cq\& introduces a character c .IP o in a wildcard pattern, a backslash can be used to escape a wildcard character, but it is matched literally when no wildcards are present. +This means that there is an extra level of backslash removal when a +pattern contains wildcard characters compared to a pattern that has none. +e.g. if you add a wildcard to \(dq\&foo\ebar\(dq\& (which matches the backslash) you +would need to use \(dq\&foo\e\ebar*\(dq\& to avoid the \(dq\&\eb\(dq\& becoming just \(dq\&b\(dq\&. .IP o if the pattern contains a / (not counting a trailing /) or a \(dq\&**\(dq\&, then it is matched against the full pathname, including any leading @@ -3904,7 +3946,7 @@ http://rsync.samba.org/ .SH "VERSION" .PP -This man page is current for version 3.1.0 of rsync. +This man page is current for version 3.1.1 of rsync. .PP .SH "INTERNAL OPTIONS" diff -upN a/rsyncd.conf.5 b/rsyncd.conf.5 --- a/rsyncd.conf.5 +++ b/rsyncd.conf.5 @@ -1,4 +1,4 @@ -.TH "rsyncd.conf" "5" "28 Sep 2013" "" "" +.TH "rsyncd.conf" "5" "22 Jun 2014" "" "" .SH "NAME" rsyncd.conf \- configuration file for rsync in daemon mode .SH "SYNOPSIS" @@ -357,6 +357,13 @@ the maximum amount of verbose informatio generate (since the information goes into the log file). The default is 1, which allows the client to request one level of verbosity. .IP +This also affects the user\(cq\&s ability to request higher levels of \fB\-\-info\fP and +\fB\-\-debug\fP logging. If the max value is 2, then no info and/or debug value +that is higher than what would be set by \fB\-vv\fP will be honored by the daemon +in its logging. To see how high of a verbosity level you need to accept for a +particular info/debug level, refer to \(dq\&rsync \-\-info=help\(dq\& and \(dq\&rsync \-\-debug=help\(dq\&. +For instance, it takes max\-verbosity 4 to be able to output debug TIME2 and FLIST3. +.IP .IP "\fBlock file\fP" This parameter specifies the file to use to support the \(dq\&max connections\(dq\& parameter. The rsync daemon uses record @@ -1039,7 +1046,7 @@ http://rsync.samba.org/ .SH "VERSION" .PP -This man page is current for version 3.1.0 of rsync. +This man page is current for version 3.1.1 of rsync. .PP .SH "CREDITS"