diff --git a/.cirrus.yml b/.cirrus.yml index 16489f8f..33e2685e 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -1,7 +1,7 @@ freebsd_task: name: FreeBSD freebsd_instance: - image_family: freebsd-13-0 + image_family: freebsd-13-1 env: PATH: /usr/local/bin:$PATH prep_script: diff --git a/Makefile.in b/Makefile.in index 3cde9557..a1253e5d 100644 --- a/Makefile.in +++ b/Makefile.in @@ -70,6 +70,8 @@ CHECK_OBJS=tls.o testrun.o getgroups.o getfsdev.o t_stub.o t_unsafe.o trimslash. $(CC) -I. -I$(srcdir) $(CFLAGS) $(CPPFLAGS) -c $< @CC_SHOBJ_FLAG@ @OBJ_RESTORE@ +# NOTE: consider running "packaging/smart-make" instead of "make" to auto-handle +# any changes to configure.sh and the main Makefile prior to a "make all". all: Makefile rsync$(EXEEXT) stunnel-rsyncd.conf @MAKE_RRSYNC@ @MAKE_MAN@ .PHONY: all diff --git a/NEWS.md b/NEWS.md index 02cc2fc7..fb656288 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,3 +1,103 @@ +# NEWS for rsync 3.2.7 (20 Oct 2022) + +## Changes in this version: + +### BUG FIXES: + +- Fixed the client-side validating of the remote sender's filtering behavior. + +- More fixes for the "unrequested file-list name" name, including a copy of + "/" with `--relative` enabled and a copy with a lot of related paths with + `--relative` enabled (often derived from a `--files-from` list). + +- When rsync gets an unpack error on an ACL, mention the filename. + +- Avoid over-setting sanitize_paths when a daemon is serving "/" (even if + "use chroot" is false). + +### ENHANCEMENTS: + +- Added negotiated daemon-auth support that allows a stronger checksum digest + to be used to validate a user's login to the daemon. Added SHA512, SHA256, + and SHA1 digests to MD5 & MD4. These new digests are at the highest priority + in the new daemon-auth negotiation list. + +- Added support for the SHA1 digest in file checksums. While this tends to be + overkill, it is available if someone really needs it. This overly-long + checksum is at the lowest priority in the normal checksum negotiation list. + See [`--checksum-choice`](rsync.1#opt) (`--cc`) and the `RSYNC_CHECKSUM_LIST` + environment var for how to customize this. + +- Improved the xattr hash table to use a 64-bit key without slowing down the + key's computation. This should make extra sure that a hash collision doesn't + happen. + +- If the `--version` option is repeated (e.g. `-VV`) then the information is + output in a (still readable) JSON format. Client side only. + +- The script `support/json-rsync-version` is available to get the JSON style + version output from any rsync. The script accepts either text on stdin + **or** an arg that specifies an rsync executable to run with a doubled + `--version` option. If the text we get isn't already in JSON format, it is + converted. Newer rsync versions will provide more complete json info than + older rsync versions. Various tweaks are made to keep the flag names + consistent across versions. + +- The [`use chroot`](rsyncd.conf.5#) daemon parameter now defaults to "unset" + so that rsync can use chroot when it works and a sanitized copy when chroot + is not supported (e.g., for a non-root daemon). Explicitly setting the + parameter to true or false (on or off) behaves the same way as before. + +- The `--fuzzy` option was optimized a bit to try to cut down on the amount of + computations when considering a big pool of files. The simple heuristic from + Kenneth Finnegan resuled in about a 2x speedup. + +- If rsync is forced to use protocol 29 or before (perhaps due to talking to an + rsync before 3.0.0), the modify time of a file is limited to 4-bytes. Rsync + now interprets this value as an unsigned integer so that a current year past + 2038 can continue to be represented. This does mean that years prior to 1970 + cannot be represented in an older protocol, but this trade-off seems like the + right choice given that (1) 2038 is very rapidly approaching, and (2) newer + protocols support a much wider range of old and new dates. + +- The rsync client now treats an empty destination arg as an error, just like + it does for an empty source arg. This doesn't affect a `host:` arg (which is + treated the same as `host:.`) since the arg is not completely empty. The use + of [`--old-args`](rsync.1#opt) (including via `RSYNC_OLD_ARGS`) allows the + prior behavior of treating an empty destination arg as a ".". + +### PACKAGING RELATED: + +- The checksum code now uses openssl's EVP methods, which gets rid of various + deprecation warnings and makes it easy to support more digest methods. On + newer systems, the MD4 digest is marked as legacy in the openssl code, which + makes openssl refuse to support it via EVP. You can choose to ignore this + and allow rsync's MD4 code to be used for older rsync connections (when + talking to an rsync prior to 3.0.0) or you can choose to configure rsync to + tell openssl to enable legacy algorithms (see below). + +- A simple openssl config file is supplied that can be installed for rsync to + use. If you install packaging/openssl-rsync.cnf to a public spot (such as + `/etc/ssl/openssl-rsync.cnf`) and then run configure with the option + `--with-openssl-conf=/path/name.cnf`, this will cause rsync to export the + configured path in the OPENSSL_CONF environment variable (when the variable + is not already set). This will enable openssl's MD4 code for rsync to use. + +- The packager may wish to include an explicit "use chroot = true" in the top + section of their supplied /etc/rsyncd.conf file if the daemon is being + installed to run as the root user (though rsync should behave the same even + with the value unset, a little extra paranoia doesn't hurt). + +- I've noticed that some packagers haven't installed support/nameconvert for + users to use in their chrooted rsync configs. Even if it is not installed + as an executable script (to avoid a python3 dependency) it would be good to + install it with the other rsync-related support scripts. + +- It would be good to add support/json-rsync-version to the list of installed + support scripts. + +------------------------------------------------------------------------------ + # NEWS for rsync 3.2.6 (9 Sep 2022) ## Changes in this version: @@ -4592,6 +4692,7 @@ | RELEASE DATE | VER. | DATE OF COMMIT\* | PROTOCOL | |--------------|--------|------------------|-------------| +| 20 Oct 2022 | 3.2.7 | | 31 | | 09 Sep 2022 | 3.2.6 | | 31 | | 14 Aug 2022 | 3.2.5 | | 31 | | 15 Apr 2022 | 3.2.4 | | 31 | diff --git a/acls.c b/acls.c index 12e662cf..3cf12eeb 100644 --- a/acls.c +++ b/acls.c @@ -519,6 +519,7 @@ static int get_rsync_acl(const char *fname, rsync_acl *racl, sys_acl_free_acl(sacl); if (!ok) { + rsyserr(FERROR_XFER, errno, "get_acl: unpack_smb_acl(%s)", fname); return -1; } } else if (no_acl_syscall_error(errno)) { diff --git a/authenticate.c b/authenticate.c index 4306d167..b7f6ead9 100644 --- a/authenticate.c +++ b/authenticate.c @@ -2,7 +2,7 @@ * Support rsync daemon authentication. * * Copyright (C) 1998-2000 Andrew Tridgell - * Copyright (C) 2002-2020 Wayne Davison + * Copyright (C) 2002-2022 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -24,6 +24,7 @@ extern int read_only; extern char *password_file; +extern struct name_num_obj valid_auth_checksums; /*************************************************************************** encode a buffer using base64 - simple and slow algorithm. null terminates @@ -72,9 +73,9 @@ static void gen_challenge(const char *addr, char *challenge) SIVAL(input, 20, tv.tv_usec); SIVAL(input, 24, getpid()); - sum_init(-1, 0); + len = sum_init(valid_auth_checksums.negotiated_nni, 0); sum_update(input, sizeof input); - len = sum_end(digest); + sum_end(digest); base64_encode(digest, len, challenge, 0); } @@ -86,10 +87,10 @@ static void generate_hash(const char *in, const char *challenge, char *out) char buf[MAX_DIGEST_LEN]; int len; - sum_init(-1, 0); + len = sum_init(valid_auth_checksums.negotiated_nni, 0); sum_update(in, strlen(in)); sum_update(challenge, strlen(challenge)); - len = sum_end(buf); + sum_end(buf); base64_encode(buf, len, out, 0); } @@ -238,6 +239,7 @@ char *auth_server(int f_in, int f_out, int module, const char *host, if (!users || !*users) return ""; + negotiate_daemon_auth(f_out, 0); gen_challenge(addr, challenge); io_printf(f_out, "%s%s\n", leader, challenge); @@ -350,6 +352,7 @@ void auth_client(int fd, const char *user, const char *challenge) if (!user || !*user) user = "nobody"; + negotiate_daemon_auth(-1, 1); if (!(pass = getpassf(password_file)) && !(pass = getenv("RSYNC_PASSWORD"))) { diff --git a/checksum.c b/checksum.c index 7eb50f17..60de3655 100644 --- a/checksum.c +++ b/checksum.c @@ -42,50 +42,96 @@ extern int protocol_version; extern int proper_seed_order; extern const char *checksum_choice; +#define NNI_BUILTIN (1<<0) +#define NNI_EVP (1<<1) +#define NNI_EVP_OK (1<<2) + struct name_num_item valid_checksums_items[] = { #ifdef SUPPORT_XXH3 - { CSUM_XXH3_128, "xxh128", NULL }, - { CSUM_XXH3_64, "xxh3", NULL }, + { CSUM_XXH3_128, 0, "xxh128", NULL }, + { CSUM_XXH3_64, 0, "xxh3", NULL }, #endif #ifdef SUPPORT_XXHASH - { CSUM_XXH64, "xxh64", NULL }, - { CSUM_XXH64, "xxhash", NULL }, + { CSUM_XXH64, 0, "xxh64", NULL }, + { CSUM_XXH64, 0, "xxhash", NULL }, #endif - { CSUM_MD5, "md5", NULL }, - { CSUM_MD4, "md4", NULL }, - { CSUM_NONE, "none", NULL }, - { 0, NULL, NULL } + { CSUM_MD5, NNI_BUILTIN|NNI_EVP, "md5", NULL }, + { CSUM_MD4, NNI_BUILTIN|NNI_EVP, "md4", NULL }, +#ifdef SHA_DIGEST_LENGTH + { CSUM_SHA1, NNI_EVP, "sha1", NULL }, +#endif + { CSUM_NONE, 0, "none", NULL }, + { 0, 0, NULL, NULL } }; struct name_num_obj valid_checksums = { - "checksum", NULL, NULL, 0, 0, valid_checksums_items + "checksum", NULL, 0, 0, valid_checksums_items }; -int xfersum_type = 0; /* used for the file transfer checksums */ -int checksum_type = 0; /* used for the pre-transfer (--checksum) checksums */ +struct name_num_item valid_auth_checksums_items[] = { +#ifdef SHA512_DIGEST_LENGTH + { CSUM_SHA512, NNI_EVP, "sha512", NULL }, +#endif +#ifdef SHA256_DIGEST_LENGTH + { CSUM_SHA256, NNI_EVP, "sha256", NULL }, +#endif +#ifdef SHA_DIGEST_LENGTH + { CSUM_SHA1, NNI_EVP, "sha1", NULL }, +#endif + { CSUM_MD5, NNI_BUILTIN|NNI_EVP, "md5", NULL }, + { CSUM_MD4, NNI_BUILTIN|NNI_EVP, "md4", NULL }, + { 0, 0, NULL, NULL } +}; + +struct name_num_obj valid_auth_checksums = { + "daemon auth checksum", NULL, 0, 0, valid_auth_checksums_items +}; + +/* These cannot make use of openssl, so they're marked just as built-in */ +struct name_num_item implied_checksum_md4 = + { CSUM_MD4, NNI_BUILTIN, "md4", NULL }; +struct name_num_item implied_checksum_md5 = + { CSUM_MD5, NNI_BUILTIN, "md5", NULL }; + +struct name_num_item *xfer_sum_nni; /* used for the transfer checksum2 computations */ +int xfer_sum_len; +struct name_num_item *file_sum_nni; /* used for the pre-transfer --checksum computations */ +int file_sum_len, file_sum_extra_cnt; + +#ifdef USE_OPENSSL +const EVP_MD *xfer_sum_evp_md; +const EVP_MD *file_sum_evp_md; +EVP_MD_CTX *ctx_evp = NULL; +#endif static int initialized_choices = 0; -int parse_csum_name(const char *name, int len) +struct name_num_item *parse_csum_name(const char *name, int len) { struct name_num_item *nni; if (len < 0 && name) len = strlen(name); + init_checksum_choices(); + if (!name || (len == 4 && strncasecmp(name, "auto", 4) == 0)) { - if (protocol_version >= 30) - return CSUM_MD5; - if (protocol_version >= 27) - return CSUM_MD4_OLD; - if (protocol_version >= 21) - return CSUM_MD4_BUSTED; - return CSUM_MD4_ARCHAIC; + if (protocol_version >= 30) { + if (!proper_seed_order) + return &implied_checksum_md5; + name = "md5"; + len = 3; + } else { + if (protocol_version >= 27) + implied_checksum_md4.num = CSUM_MD4_OLD; + else if (protocol_version >= 21) + implied_checksum_md4.num = CSUM_MD4_BUSTED; + else + implied_checksum_md4.num = CSUM_MD4_ARCHAIC; + return &implied_checksum_md4; + } } - if (!initialized_choices) - init_checksum_choices(); - nni = get_nni_by_name(&valid_checksums, name, len); if (!nni) { @@ -93,44 +139,74 @@ int parse_csum_name(const char *name, int len) exit_cleanup(RERR_UNSUPPORTED); } - return nni->num; + return nni; } -static const char *checksum_name(int num) +#ifdef USE_OPENSSL +static const EVP_MD *csum_evp_md(struct name_num_item *nni) { - struct name_num_item *nni = get_nni_by_num(&valid_checksums, num); + const EVP_MD *emd; + if (!(nni->flags & NNI_EVP)) + return NULL; - return nni ? nni->name : num < CSUM_MD4 ? "md4" : "UNKNOWN"; +#ifdef USE_MD5_ASM + if (nni->num == CSUM_MD5) + emd = NULL; + else +#endif + emd = EVP_get_digestbyname(nni->name); + if (emd && !(nni->flags & NNI_EVP_OK)) { /* Make sure it works before we advertise it */ + if (!ctx_evp && !(ctx_evp = EVP_MD_CTX_create())) + out_of_memory("csum_evp_md"); + /* Some routines are marked as legacy and are not enabled in the openssl.cnf file. + * If we can't init the emd, we'll fall back to our built-in code. */ + if (EVP_DigestInit_ex(ctx_evp, emd, NULL) == 0) + emd = NULL; + else + nni->flags = (nni->flags & ~NNI_BUILTIN) | NNI_EVP_OK; + } + if (!emd) + nni->flags &= ~NNI_EVP; + return emd; } +#endif void parse_checksum_choice(int final_call) { - if (valid_checksums.negotiated_name) - xfersum_type = checksum_type = valid_checksums.negotiated_num; + if (valid_checksums.negotiated_nni) + xfer_sum_nni = file_sum_nni = valid_checksums.negotiated_nni; else { char *cp = checksum_choice ? strchr(checksum_choice, ',') : NULL; if (cp) { - xfersum_type = parse_csum_name(checksum_choice, cp - checksum_choice); - checksum_type = parse_csum_name(cp+1, -1); + xfer_sum_nni = parse_csum_name(checksum_choice, cp - checksum_choice); + file_sum_nni = parse_csum_name(cp+1, -1); } else - xfersum_type = checksum_type = parse_csum_name(checksum_choice, -1); + xfer_sum_nni = file_sum_nni = parse_csum_name(checksum_choice, -1); if (am_server && checksum_choice) - validate_choice_vs_env(NSTR_CHECKSUM, xfersum_type, checksum_type); + validate_choice_vs_env(NSTR_CHECKSUM, xfer_sum_nni->num, file_sum_nni->num); } + xfer_sum_len = csum_len_for_type(xfer_sum_nni->num, 0); + file_sum_len = csum_len_for_type(file_sum_nni->num, 0); +#ifdef USE_OPENSSL + xfer_sum_evp_md = csum_evp_md(xfer_sum_nni); + file_sum_evp_md = csum_evp_md(file_sum_nni); +#endif - if (xfersum_type == CSUM_NONE) + file_sum_extra_cnt = (file_sum_len + EXTRA_LEN - 1) / EXTRA_LEN; + + if (xfer_sum_nni->num == CSUM_NONE) whole_file = 1; /* Snag the checksum name for both write_batch's option output & the following debug output. */ - if (valid_checksums.negotiated_name) - checksum_choice = valid_checksums.negotiated_name; + if (valid_checksums.negotiated_nni) + checksum_choice = valid_checksums.negotiated_nni->name; else if (checksum_choice == NULL) - checksum_choice = checksum_name(xfersum_type); + checksum_choice = xfer_sum_nni->name; if (final_call && DEBUG_GTE(NSTR, am_server ? 3 : 1)) { rprintf(FINFO, "%s%s checksum: %s\n", am_server ? "Server" : "Client", - valid_checksums.negotiated_name ? " negotiated" : "", + valid_checksums.negotiated_nni ? " negotiated" : "", checksum_choice); } } @@ -150,6 +226,18 @@ int csum_len_for_type(int cst, BOOL flist_csum) return MD4_DIGEST_LEN; case CSUM_MD5: return MD5_DIGEST_LEN; +#ifdef SHA_DIGEST_LENGTH + case CSUM_SHA1: + return SHA_DIGEST_LENGTH; +#endif +#ifdef SHA256_DIGEST_LENGTH + case CSUM_SHA256: + return SHA256_DIGEST_LENGTH; +#endif +#ifdef SHA512_DIGEST_LENGTH + case CSUM_SHA512: + return SHA512_DIGEST_LENGTH; +#endif case CSUM_XXH64: case CSUM_XXH3_64: return 64/8; @@ -175,6 +263,9 @@ int canonical_checksum(int csum_type) break; case CSUM_MD4: case CSUM_MD5: + case CSUM_SHA1: + case CSUM_SHA256: + case CSUM_SHA512: return -1; case CSUM_XXH64: case CSUM_XXH3_64: @@ -211,7 +302,22 @@ uint32 get_checksum1(char *buf1, int32 len) void get_checksum2(char *buf, int32 len, char *sum) { - switch (xfersum_type) { +#ifdef USE_OPENSSL + if (xfer_sum_evp_md) { + static EVP_MD_CTX *evp = NULL; + uchar seedbuf[4]; + if (!evp && !(evp = EVP_MD_CTX_create())) + out_of_memory("get_checksum2"); + EVP_DigestInit_ex(evp, xfer_sum_evp_md, NULL); + if (checksum_seed) { + SIVALu(seedbuf, 0, checksum_seed); + EVP_DigestUpdate(evp, seedbuf, 4); + } + EVP_DigestUpdate(evp, (uchar *)buf, len); + EVP_DigestFinal_ex(evp, (uchar *)sum, NULL); + } else +#endif + switch (xfer_sum_nni->num) { #ifdef SUPPORT_XXHASH case CSUM_XXH64: SIVAL64(sum, 0, XXH64(buf, len, checksum_seed)); @@ -229,7 +335,7 @@ void get_checksum2(char *buf, int32 len, char *sum) } #endif case CSUM_MD5: { - md5_context m5; + md_context m5; uchar seedbuf[4]; md5_begin(&m5); if (proper_seed_order) { @@ -249,20 +355,6 @@ void get_checksum2(char *buf, int32 len, char *sum) break; } case CSUM_MD4: -#ifdef USE_OPENSSL - { - MD4_CTX m4; - MD4_Init(&m4); - MD4_Update(&m4, (uchar *)buf, len); - if (checksum_seed) { - uchar seedbuf[4]; - SIVALu(seedbuf, 0, checksum_seed); - MD4_Update(&m4, seedbuf, 4); - } - MD4_Final((uchar *)sum, &m4); - break; - } -#endif case CSUM_MD4_OLD: case CSUM_MD4_BUSTED: case CSUM_MD4_ARCHAIC: { @@ -295,7 +387,7 @@ void get_checksum2(char *buf, int32 len, char *sum) * are multiples of 64. This is fixed by calling mdfour_update() * even when there are no more bytes. */ - if (len - i > 0 || xfersum_type > CSUM_MD4_BUSTED) + if (len - i > 0 || xfer_sum_nni->num > CSUM_MD4_BUSTED) mdfour_update(&m, (uchar *)(buf1+i), len-i); mdfour_result(&m, (uchar *)sum); @@ -313,15 +405,33 @@ void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum) int32 remainder; int fd; - memset(sum, 0, MAX_DIGEST_LEN); - fd = do_open(fname, O_RDONLY, 0); - if (fd == -1) + if (fd == -1) { + memset(sum, 0, file_sum_len); return; + } buf = map_file(fd, len, MAX_MAP_SIZE, CHUNK_SIZE); - switch (checksum_type) { +#ifdef USE_OPENSSL + if (file_sum_evp_md) { + static EVP_MD_CTX *evp = NULL; + if (!evp && !(evp = EVP_MD_CTX_create())) + out_of_memory("file_checksum"); + + EVP_DigestInit_ex(evp, file_sum_evp_md, NULL); + + for (i = 0; i + CHUNK_SIZE <= len; i += CHUNK_SIZE) + EVP_DigestUpdate(evp, (uchar *)map_ptr(buf, i, CHUNK_SIZE), CHUNK_SIZE); + + remainder = (int32)(len - i); + if (remainder > 0) + EVP_DigestUpdate(evp, (uchar *)map_ptr(buf, i, remainder), remainder); + + EVP_DigestFinal_ex(evp, (uchar *)sum, NULL); + } else +#endif + switch (file_sum_nni->num) { #ifdef SUPPORT_XXHASH case CSUM_XXH64: { static XXH64_state_t* state = NULL; @@ -381,7 +491,7 @@ void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum) } #endif case CSUM_MD5: { - md5_context m5; + md_context m5; md5_begin(&m5); @@ -396,23 +506,6 @@ void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum) break; } case CSUM_MD4: -#ifdef USE_OPENSSL - { - MD4_CTX m4; - - MD4_Init(&m4); - - for (i = 0; i + CHUNK_SIZE <= len; i += CHUNK_SIZE) - MD4_Update(&m4, (uchar *)map_ptr(buf, i, CHUNK_SIZE), CHUNK_SIZE); - - remainder = (int32)(len - i); - if (remainder > 0) - MD4_Update(&m4, (uchar *)map_ptr(buf, i, remainder), remainder); - - MD4_Final((uchar *)sum, &m4); - break; - } -#endif case CSUM_MD4_OLD: case CSUM_MD4_BUSTED: case CSUM_MD4_ARCHAIC: { @@ -428,7 +521,7 @@ void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum) * are multiples of 64. This is fixed by calling mdfour_update() * even when there are no more bytes. */ remainder = (int32)(len - i); - if (remainder > 0 || checksum_type > CSUM_MD4_BUSTED) + if (remainder > 0 || file_sum_nni->num > CSUM_MD4_BUSTED) mdfour_update(&m, (uchar *)map_ptr(buf, i, remainder), remainder); mdfour_result(&m, (uchar *)sum); @@ -436,7 +529,7 @@ void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum) } default: rprintf(FERROR, "Invalid checksum-choice for --checksum: %s (%d)\n", - checksum_name(checksum_type), checksum_type); + file_sum_nni->name, file_sum_nni->num); exit_cleanup(RERR_UNSUPPORTED); } @@ -445,30 +538,43 @@ void file_checksum(const char *fname, const STRUCT_STAT *st_p, char *sum) } static int32 sumresidue; -static union { - md_context md; -#ifdef USE_OPENSSL - MD4_CTX m4; -#endif - md5_context m5; -} ctx; +static md_context ctx_md; #ifdef SUPPORT_XXHASH static XXH64_state_t* xxh64_state; #endif #ifdef SUPPORT_XXH3 static XXH3_state_t* xxh3_state; #endif -static int cursum_type; +static struct name_num_item *cur_sum_nni; +int cur_sum_len; -void sum_init(int csum_type, int seed) +#ifdef USE_OPENSSL +static const EVP_MD *cur_sum_evp_md; +#endif + +/* Initialize a hash digest accumulator. Data is supplied via + * sum_update() and the resulting binary digest is retrieved via + * sum_end(). This only supports one active sum at a time. */ +int sum_init(struct name_num_item *nni, int seed) { char s[4]; - if (csum_type < 0) - csum_type = parse_csum_name(NULL, 0); - cursum_type = csum_type; + if (!nni) + nni = parse_csum_name(NULL, 0); + cur_sum_nni = nni; + cur_sum_len = csum_len_for_type(nni->num, 0); +#ifdef USE_OPENSSL + cur_sum_evp_md = csum_evp_md(nni); +#endif - switch (csum_type) { +#ifdef USE_OPENSSL + if (cur_sum_evp_md) { + if (!ctx_evp && !(ctx_evp = EVP_MD_CTX_create())) + out_of_memory("file_checksum"); + EVP_DigestInit_ex(ctx_evp, cur_sum_evp_md, NULL); + } else +#endif + switch (cur_sum_nni->num) { #ifdef SUPPORT_XXHASH case CSUM_XXH64: if (!xxh64_state && !(xxh64_state = XXH64_createState())) @@ -489,20 +595,16 @@ void sum_init(int csum_type, int seed) break; #endif case CSUM_MD5: - md5_begin(&ctx.m5); + md5_begin(&ctx_md); break; case CSUM_MD4: -#ifdef USE_OPENSSL - MD4_Init(&ctx.m4); -#else - mdfour_begin(&ctx.md); + mdfour_begin(&ctx_md); sumresidue = 0; -#endif break; case CSUM_MD4_OLD: case CSUM_MD4_BUSTED: case CSUM_MD4_ARCHAIC: - mdfour_begin(&ctx.md); + mdfour_begin(&ctx_md); sumresidue = 0; SIVAL(s, 0, seed); sum_update(s, 4); @@ -512,19 +614,19 @@ void sum_init(int csum_type, int seed) default: /* paranoia to prevent missing case values */ exit_cleanup(RERR_UNSUPPORTED); } + + return cur_sum_len; } -/** - * Feed data into an MD4 accumulator, md. The results may be - * retrieved using sum_end(). md is used for different purposes at - * different points during execution. - * - * @todo Perhaps get rid of md and just pass in the address each time. - * Very slightly clearer and slower. - **/ +/* Feed data into a hash digest accumulator. */ void sum_update(const char *p, int32 len) { - switch (cursum_type) { +#ifdef USE_OPENSSL + if (cur_sum_evp_md) { + EVP_DigestUpdate(ctx_evp, (uchar *)p, len); + } else +#endif + switch (cur_sum_nni->num) { #ifdef SUPPORT_XXHASH case CSUM_XXH64: XXH64_update(xxh64_state, p, len); @@ -539,39 +641,35 @@ void sum_update(const char *p, int32 len) break; #endif case CSUM_MD5: - md5_update(&ctx.m5, (uchar *)p, len); + md5_update(&ctx_md, (uchar *)p, len); break; case CSUM_MD4: -#ifdef USE_OPENSSL - MD4_Update(&ctx.m4, (uchar *)p, len); - break; -#endif case CSUM_MD4_OLD: case CSUM_MD4_BUSTED: case CSUM_MD4_ARCHAIC: if (len + sumresidue < CSUM_CHUNK) { - memcpy(ctx.md.buffer + sumresidue, p, len); + memcpy(ctx_md.buffer + sumresidue, p, len); sumresidue += len; break; } if (sumresidue) { int32 i = CSUM_CHUNK - sumresidue; - memcpy(ctx.md.buffer + sumresidue, p, i); - mdfour_update(&ctx.md, (uchar *)ctx.md.buffer, CSUM_CHUNK); + memcpy(ctx_md.buffer + sumresidue, p, i); + mdfour_update(&ctx_md, (uchar *)ctx_md.buffer, CSUM_CHUNK); len -= i; p += i; } while (len >= CSUM_CHUNK) { - mdfour_update(&ctx.md, (uchar *)p, CSUM_CHUNK); + mdfour_update(&ctx_md, (uchar *)p, CSUM_CHUNK); len -= CSUM_CHUNK; p += CSUM_CHUNK; } sumresidue = len; if (sumresidue) - memcpy(ctx.md.buffer, p, sumresidue); + memcpy(ctx_md.buffer, p, sumresidue); break; case CSUM_NONE: break; @@ -580,13 +678,18 @@ void sum_update(const char *p, int32 len) } } -/* NOTE: all the callers of sum_end() pass in a pointer to a buffer that is - * MAX_DIGEST_LEN in size, so even if the csum-len is shorter than that (i.e. - * CSUM_MD4_ARCHAIC), we don't have to worry about limiting the data we write - * into the "sum" buffer. */ -int sum_end(char *sum) +/* The sum buffer only needs to be as long as the current checksum's digest + * len, not MAX_DIGEST_LEN. Note that for CSUM_MD4_ARCHAIC that is the full + * MD4_DIGEST_LEN even if the file-list code is going to ignore all but the + * first 2 bytes of it. */ +void sum_end(char *sum) { - switch (cursum_type) { +#ifdef USE_OPENSSL + if (cur_sum_evp_md) { + EVP_DigestFinal_ex(ctx_evp, (uchar *)sum, NULL); + } else +#endif + switch (cur_sum_nni->num) { #ifdef SUPPORT_XXHASH case CSUM_XXH64: SIVAL64(sum, 0, XXH64_digest(xxh64_state)); @@ -604,22 +707,18 @@ int sum_end(char *sum) } #endif case CSUM_MD5: - md5_result(&ctx.m5, (uchar *)sum); + md5_result(&ctx_md, (uchar *)sum); break; case CSUM_MD4: -#ifdef USE_OPENSSL - MD4_Final((uchar *)sum, &ctx.m4); - break; -#endif case CSUM_MD4_OLD: - mdfour_update(&ctx.md, (uchar *)ctx.md.buffer, sumresidue); - mdfour_result(&ctx.md, (uchar *)sum); + mdfour_update(&ctx_md, (uchar *)ctx_md.buffer, sumresidue); + mdfour_result(&ctx_md, (uchar *)sum); break; case CSUM_MD4_BUSTED: case CSUM_MD4_ARCHAIC: if (sumresidue) - mdfour_update(&ctx.md, (uchar *)ctx.md.buffer, sumresidue); - mdfour_result(&ctx.md, (uchar *)sum); + mdfour_update(&ctx_md, (uchar *)ctx_md.buffer, sumresidue); + mdfour_result(&ctx_md, (uchar *)sum); break; case CSUM_NONE: *sum = '\0'; @@ -627,34 +726,74 @@ int sum_end(char *sum) default: /* paranoia to prevent missing case values */ exit_cleanup(RERR_UNSUPPORTED); } +} + +#if defined SUPPORT_XXH3 || defined USE_OPENSSL +static void verify_digest(struct name_num_item *nni, BOOL check_auth_list) +{ +#ifdef SUPPORT_XXH3 + static int xxh3_result = 0; +#endif +#ifdef USE_OPENSSL + static int prior_num = 0, prior_flags = 0, prior_result = 0; +#endif + +#ifdef SUPPORT_XXH3 + if (nni->num == CSUM_XXH3_64 || nni->num == CSUM_XXH3_128) { + if (!xxh3_result) { + char buf[32816]; + int j; + for (j = 0; j < (int)sizeof buf; j++) + buf[j] = ' ' + (j % 96); + sum_init(nni, 0); + sum_update(buf, 32816); + sum_update(buf, 31152); + sum_update(buf, 32474); + sum_update(buf, 9322); + xxh3_result = XXH3_64bits_digest(xxh3_state) != 0xadbcf16d4678d1de ? -1 : 1; + } + if (xxh3_result < 0) + nni->num = CSUM_gone; + return; + } +#endif - return csum_len_for_type(cursum_type, 0); +#ifdef USE_OPENSSL + if (BITS_SETnUNSET(nni->flags, NNI_EVP, NNI_BUILTIN|NNI_EVP_OK)) { + if (nni->num == prior_num && nni->flags == prior_flags) { + nni->flags = prior_result; + if (!(nni->flags & NNI_EVP)) + nni->num = CSUM_gone; + } else { + prior_num = nni->num; + prior_flags = nni->flags; + if (!csum_evp_md(nni)) + nni->num = CSUM_gone; + prior_result = nni->flags; + if (check_auth_list && (nni = get_nni_by_num(&valid_auth_checksums, prior_num)) != NULL) + verify_digest(nni, False); + } + } +#endif } +#endif void init_checksum_choices() { -#ifdef SUPPORT_XXH3 - char buf[32816]; - int j; - for (j = 0; j < (int)sizeof buf; j++) { - buf[j] = ' ' + (j % 96); - } - sum_init(CSUM_XXH3_64, 0); - sum_update(buf, 32816); - sum_update(buf, 31152); - sum_update(buf, 32474); - sum_update(buf, 9322); - if (XXH3_64bits_digest(xxh3_state) != 0xadbcf16d4678d1de) { - int t, f; - struct name_num_item *nni = valid_checksums.list; - for (t = f = 0; nni[f].name; f++) { - if (nni[f].num == CSUM_XXH3_64 || nni[f].num == CSUM_XXH3_128) - continue; - if (t != f) - nni[t++] = nni[f]; - } - nni[t].name = NULL; - } +#if defined SUPPORT_XXH3 || defined USE_OPENSSL + struct name_num_item *nni; #endif + + if (initialized_choices) + return; + +#if defined SUPPORT_XXH3 || defined USE_OPENSSL + for (nni = valid_checksums.list; nni->name; nni++) + verify_digest(nni, True); + + for (nni = valid_auth_checksums.list; nni->name; nni++) + verify_digest(nni, False); +#endif + initialized_choices = 1; } diff --git a/clientserver.c b/clientserver.c index ca897ff1..7c897abc 100644 --- a/clientserver.c +++ b/clientserver.c @@ -67,6 +67,7 @@ extern uid_t our_uid; extern gid_t our_gid; char *auth_user; +char *daemon_auth_choices; int read_only = 0; int module_id = -1; int pid_file_fd = -1; @@ -149,13 +150,9 @@ int start_socket_client(char *host, int remote_argc, char *remote_argv[], static int exchange_protocols(int f_in, int f_out, char *buf, size_t bufsiz, int am_client) { int remote_sub = -1; -#if SUBPROTOCOL_VERSION != 0 - int our_sub = protocol_version < PROTOCOL_VERSION ? 0 : SUBPROTOCOL_VERSION; -#else - int our_sub = 0; -#endif + int our_sub = get_subprotocol_version(); - io_printf(f_out, "@RSYNCD: %d.%d\n", protocol_version, our_sub); + output_daemon_greeting(f_out, am_client); if (!am_client) { char *motd = lp_motd_file(); if (motd && *motd) { @@ -187,16 +184,30 @@ static int exchange_protocols(int f_in, int f_out, char *buf, size_t bufsiz, int } if (remote_sub < 0) { - if (remote_protocol == 30) { + if (remote_protocol >= 30) { if (am_client) - rprintf(FERROR, "rsync: server is speaking an incompatible beta of protocol 30\n"); + rprintf(FERROR, "rsync: the server omitted the subprotocol value: %s\n", buf); else - io_printf(f_out, "@ERROR: your client is speaking an incompatible beta of protocol 30\n"); + io_printf(f_out, "@ERROR: your client omitted the subprotocol value: %s\n", buf); return -1; } remote_sub = 0; } + daemon_auth_choices = strchr(buf + 9, ' '); + if (daemon_auth_choices) { + char *cp; + daemon_auth_choices = strdup(daemon_auth_choices + 1); + if ((cp = strchr(daemon_auth_choices, '\n')) != NULL) + *cp = '\0'; + } else if (remote_protocol > 31) { + if (am_client) + rprintf(FERROR, "rsync: the server omitted the digest name list: %s\n", buf); + else + io_printf(f_out, "@ERROR: your client omitted the digest name list: %s\n", buf); + return -1; + } + if (protocol_version > remote_protocol) { protocol_version = remote_protocol; if (remote_sub) @@ -429,7 +440,7 @@ static int read_arg_from_pipe(int fd, char *buf, int limit) } #endif -static void set_env_str(const char *var, const char *str) +void set_env_str(const char *var, const char *str) { #ifdef HAVE_SETENV if (setenv(var, str, 1) < 0) @@ -690,7 +701,7 @@ static int rsync_module(int f_in, int f_out, int i, const char *addr, const char int set_uid; char *p, *err_msg = NULL; char *name = lp_name(i); - int use_chroot = lp_use_chroot(i); + int use_chroot = lp_use_chroot(i); /* might be 1 (yes), 0 (no), or -1 (unset) */ int ret, pre_exec_arg_fd = -1, pre_exec_error_fd = -1; int save_munge_symlinks; pid_t pre_exec_pid = 0; @@ -815,6 +826,20 @@ static int rsync_module(int f_in, int f_out, int i, const char *addr, const char io_printf(f_out, "@ERROR: no path setting.\n"); return -1; } + if (use_chroot < 0) { + if (strstr(module_dir, "/./") != NULL) + use_chroot = 1; /* The module is expecting a chroot inner & outer path. */ + else if (chroot("/") < 0) { + rprintf(FLOG, "chroot test failed: %s. " + "Switching 'use chroot' from unset to false.\n", + strerror(errno)); + use_chroot = 0; + } else { + if (chdir("/") < 0) + rsyserr(FLOG, errno, "chdir(\"/\") failed"); + use_chroot = 1; + } + } if (use_chroot) { if ((p = strstr(module_dir, "/./")) != NULL) { *p = '\0'; /* Temporary... */ @@ -951,20 +976,8 @@ static int rsync_module(int f_in, int f_out, int i, const char *addr, const char } if (use_chroot) { - /* - * XXX: The 'use chroot' flag is a fairly reliable - * source of confusion, because it fails under two - * important circumstances: running as non-root, - * running on Win32 (or possibly others). On the - * other hand, if you are running as root, then it - * might be better to always use chroot. - * - * So, perhaps if we can't chroot we should just issue - * a warning, unless a "require chroot" flag is set, - * in which case we fail. - */ if (chroot(module_chdir)) { - rsyserr(FLOG, errno, "chroot %s failed", module_chdir); + rsyserr(FLOG, errno, "chroot(\"%s\") failed", module_chdir); io_printf(f_out, "@ERROR: chroot failed\n"); return -1; } @@ -973,7 +986,7 @@ static int rsync_module(int f_in, int f_out, int i, const char *addr, const char if (!change_dir(module_chdir, CD_NORMAL)) return path_failure(f_out, module_chdir, True); - if (module_dirlen || (!use_chroot && !*lp_daemon_chroot())) + if (module_dirlen) sanitize_paths = 1; if ((munge_symlinks = lp_munge_symlinks(module_id)) < 0) @@ -1288,8 +1301,12 @@ int start_daemon(int f_in, int f_out) p = lp_daemon_chroot(); if (*p) { log_init(0); /* Make use we've initialized syslog before chrooting. */ - if (chroot(p) < 0 || chdir("/") < 0) { - rsyserr(FLOG, errno, "daemon chroot %s failed", p); + if (chroot(p) < 0) { + rsyserr(FLOG, errno, "daemon chroot(\"%s\") failed", p); + return -1; + } + if (chdir("/") < 0) { + rsyserr(FLOG, errno, "daemon chdir(\"/\") failed"); return -1; } } diff --git a/compat.c b/compat.c index 622910eb..a8a6afe8 100644 --- a/compat.c +++ b/compat.c @@ -60,13 +60,16 @@ extern char *files_from; extern char *filesfrom_host; extern const char *checksum_choice; extern const char *compress_choice; +extern char *daemon_auth_choices; extern filter_rule_list filter_list; extern int need_unsorted_flist; #ifdef ICONV_OPTION extern iconv_t ic_send, ic_recv; extern char *iconv_opt; #endif -extern struct name_num_obj valid_checksums; +extern struct name_num_obj valid_checksums, valid_auth_checksums; + +extern struct name_num_item *xfer_sum_nni; int remote_protocol = 0; int file_extra_cnt = 0; /* count of file-list extras that everyone gets */ @@ -79,6 +82,9 @@ int inplace_partial = 0; int do_negotiated_strings = 0; int xmit_id0_names = 0; +struct name_num_item *xattr_sum_nni; +int xattr_sum_len = 0; + /* These index values are for the file-list's extra-attribute array. */ int pathname_ndx, depth_ndx, atimes_ndx, crtimes_ndx, uid_ndx, gid_ndx, acls_ndx, xattrs_ndx, unsort_ndx; @@ -93,19 +99,19 @@ int filesfrom_convert = 0; struct name_num_item valid_compressions_items[] = { #ifdef SUPPORT_ZSTD - { CPRES_ZSTD, "zstd", NULL }, + { CPRES_ZSTD, 0, "zstd", NULL }, #endif #ifdef SUPPORT_LZ4 - { CPRES_LZ4, "lz4", NULL }, + { CPRES_LZ4, 0, "lz4", NULL }, #endif - { CPRES_ZLIBX, "zlibx", NULL }, - { CPRES_ZLIB, "zlib", NULL }, - { CPRES_NONE, "none", NULL }, - { 0, NULL, NULL } + { CPRES_ZLIBX, 0, "zlibx", NULL }, + { CPRES_ZLIB, 0, "zlib", NULL }, + { CPRES_NONE, 0, "none", NULL }, + { 0, 0, NULL, NULL } }; struct name_num_obj valid_compressions = { - "compress", NULL, NULL, 0, 0, valid_compressions_items + "compress", NULL, 0, 0, valid_compressions_items }; #define CF_INC_RECURSE (1<<0) @@ -127,11 +133,7 @@ static void check_sub_protocol(void) { char *dot; int their_protocol, their_sub; -#if SUBPROTOCOL_VERSION != 0 - int our_sub = protocol_version < PROTOCOL_VERSION ? 0 : SUBPROTOCOL_VERSION; -#else - int our_sub = 0; -#endif + int our_sub = get_subprotocol_version(); /* client_info starts with VER.SUB string if client is a pre-release. */ if (!(their_protocol = atoi(client_info)) @@ -178,8 +180,8 @@ void set_allow_inc_recurse(void) void parse_compress_choice(int final_call) { - if (valid_compressions.negotiated_name) - do_compression = valid_compressions.negotiated_num; + if (valid_compressions.negotiated_nni) + do_compression = valid_compressions.negotiated_nni->num; else if (compress_choice) { struct name_num_item *nni = get_nni_by_name(&valid_compressions, compress_choice, -1); if (!nni) { @@ -201,8 +203,8 @@ void parse_compress_choice(int final_call) compress_choice = NULL; /* Snag the compression name for both write_batch's option output & the following debug output. */ - if (valid_compressions.negotiated_name) - compress_choice = valid_compressions.negotiated_name; + if (valid_compressions.negotiated_nni) + compress_choice = valid_compressions.negotiated_nni->name; else if (compress_choice == NULL) { struct name_num_item *nni = get_nni_by_num(&valid_compressions, do_compression); compress_choice = nni ? nni->name : "UNKNOWN"; @@ -212,7 +214,7 @@ void parse_compress_choice(int final_call) && (do_compression != CPRES_NONE || do_compression_level != CLVL_NOT_SPECIFIED)) { rprintf(FINFO, "%s%s compress: %s (level %d)\n", am_server ? "Server" : "Client", - valid_compressions.negotiated_name ? " negotiated" : "", + valid_compressions.negotiated_nni ? " negotiated" : "", compress_choice, do_compression_level); } } @@ -225,6 +227,8 @@ struct name_num_item *get_nni_by_name(struct name_num_obj *nno, const char *name len = strlen(name); for (nni = nno->list; nni->name; nni++) { + if (nni->num == CSUM_gone) + continue; if (strncasecmp(name, nni->name, len) == 0 && nni->name[len] == '\0') return nni; } @@ -259,10 +263,12 @@ static void init_nno_saw(struct name_num_obj *nno, int val) if (!nno->saw) { nno->saw = new_array0(uchar, nno->saw_len); - /* We'll take this opportunity to make sure that the main_name values are set right. */ + /* We'll take this opportunity to set the main_nni values for duplicates. */ for (cnt = 1, nni = nno->list; nni->name; nni++, cnt++) { + if (nni->num == CSUM_gone) + continue; if (nno->saw[nni->num]) - nni->main_name = nno->list[nno->saw[nni->num]-1].name; + nni->main_nni = &nno->list[nno->saw[nni->num]-1]; else nno->saw[nni->num] = cnt; } @@ -288,8 +294,8 @@ static int parse_nni_str(struct name_num_obj *nno, const char *from, char *tobuf struct name_num_item *nni = get_nni_by_name(nno, tok, to - tok); if (nni && !nno->saw[nni->num]) { nno->saw[nni->num] = ++cnt; - if (nni->main_name) { - to = tok + strlcpy(tok, nni->main_name, tobuf_len - (tok - tobuf)); + if (nni->main_nni) { + to = tok + strlcpy(tok, nni->main_nni->name, tobuf_len - (tok - tobuf)); if (to - tobuf >= tobuf_len) { to = tok - 1; break; @@ -323,13 +329,44 @@ static int parse_nni_str(struct name_num_obj *nno, const char *from, char *tobuf return to - tobuf; } +static int parse_negotiate_str(struct name_num_obj *nno, char *tmpbuf) +{ + struct name_num_item *nni, *ret = NULL; + int best = nno->saw_len; /* We want best == 1 from the client list, so start with a big number. */ + char *space, *tok = tmpbuf; + while (tok) { + while (*tok == ' ') tok++; /* Should be unneeded... */ + if (!*tok) + break; + if ((space = strchr(tok, ' ')) != NULL) + *space = '\0'; + nni = get_nni_by_name(nno, tok, -1); + if (space) { + *space = ' '; + tok = space + 1; + } else + tok = NULL; + if (!nni || !nno->saw[nni->num] || best <= nno->saw[nni->num]) + continue; + ret = nni; + best = nno->saw[nni->num]; + if (best == 1 || am_server) /* The server side stops at the first acceptable client choice */ + break; + } + if (ret) { + free(nno->saw); + nno->saw = NULL; + nno->negotiated_nni = ret->main_nni ? ret->main_nni : ret; + return 1; + } + return 0; +} + /* This routine is always called with a tmpbuf of MAX_NSTR_STRLEN length, but the * buffer may be pre-populated with a "len" length string to use OR a len of -1 * to tell us to read a string from the fd. */ static void recv_negotiate_str(int f_in, struct name_num_obj *nno, char *tmpbuf, int len) { - struct name_num_item *ret = NULL; - if (len < 0) len = read_vstring(f_in, tmpbuf, MAX_NSTR_STRLEN); @@ -340,37 +377,8 @@ static void recv_negotiate_str(int f_in, struct name_num_obj *nno, char *tmpbuf, rprintf(FINFO, "Server %s list (on client): %s\n", nno->type, tmpbuf); } - if (len > 0) { - struct name_num_item *nni; - int best = nno->saw_len; /* We want best == 1 from the client list, so start with a big number. */ - char *space, *tok = tmpbuf; - while (tok) { - while (*tok == ' ') tok++; /* Should be unneeded... */ - if (!*tok) - break; - if ((space = strchr(tok, ' ')) != NULL) - *space = '\0'; - nni = get_nni_by_name(nno, tok, -1); - if (space) { - *space = ' '; - tok = space + 1; - } else - tok = NULL; - if (!nni || !nno->saw[nni->num] || best <= nno->saw[nni->num]) - continue; - ret = nni; - best = nno->saw[nni->num]; - if (best == 1 || am_server) /* The server side stops at the first acceptable client choice */ - break; - } - if (ret) { - free(nno->saw); - nno->saw = NULL; - nno->negotiated_name = ret->main_name ? ret->main_name : ret->name; - nno->negotiated_num = ret->num; - return; - } - } + if (len > 0 && parse_negotiate_str(nno, tmpbuf)) + return; if (!am_server || !do_negotiated_strings) { char *cp = tmpbuf; @@ -466,8 +474,10 @@ int get_default_nno_list(struct name_num_obj *nno, char *to_buf, int to_buf_len, init_nno_saw(nno, 0); for (nni = nno->list, len = 0; nni->name; nni++) { - if (nni->main_name) { - if (!dup_markup) + if (nni->num == CSUM_gone) + continue; + if (nni->main_nni) { + if (!dup_markup || nni->main_nni->num == CSUM_gone) continue; delim = dup_markup; } @@ -556,7 +566,7 @@ static void negotiate_the_strings(int f_in, int f_out) /* If the other side is too old to negotiate, the above steps just made sure that * the env didn't disallow the old algorithm. Mark things as non-negotiated. */ if (!do_negotiated_strings) - valid_checksums.negotiated_name = valid_compressions.negotiated_name = NULL; + valid_checksums.negotiated_nni = valid_compressions.negotiated_nni = NULL; } void setup_protocol(int f_out,int f_in) @@ -805,11 +815,73 @@ void setup_protocol(int f_out,int f_in) checksum_seed = read_int(f_in); } - parse_checksum_choice(1); /* Sets checksum_type & xfersum_type */ + parse_checksum_choice(1); /* Sets file_sum_nni & xfer_sum_nni */ parse_compress_choice(1); /* Sets do_compression */ + /* TODO in the future allow this algorithm to be chosen somehow, but it can't get too + * long or the size starts to cause a problem in the xattr abbrev/non-abbrev code. */ + xattr_sum_nni = parse_csum_name(NULL, 0); + xattr_sum_len = csum_len_for_type(xattr_sum_nni->num, 0); + if (write_batch && !am_server) write_batch_shell_file(); init_flist(); } + +void output_daemon_greeting(int f_out, int am_client) +{ + char tmpbuf[MAX_NSTR_STRLEN]; + int our_sub = get_subprotocol_version(); + + get_default_nno_list(&valid_auth_checksums, tmpbuf, MAX_NSTR_STRLEN, '\0'); + + io_printf(f_out, "@RSYNCD: %d.%d %s\n", protocol_version, our_sub, tmpbuf); + + if (am_client && DEBUG_GTE(NSTR, 2)) + rprintf(FINFO, "Client %s list (on client): %s\n", valid_auth_checksums.type, tmpbuf); +} + +void negotiate_daemon_auth(int f_out, int am_client) +{ + char tmpbuf[MAX_NSTR_STRLEN]; + int save_am_server = am_server; + int md4_is_old = 0; + + if (!am_client) + am_server = 1; + + if (daemon_auth_choices) + strlcpy(tmpbuf, daemon_auth_choices, MAX_NSTR_STRLEN); + else { + strlcpy(tmpbuf, protocol_version >= 30 ? "md5" : "md4", MAX_NSTR_STRLEN); + md4_is_old = 1; + } + + if (am_client) { + recv_negotiate_str(-1, &valid_auth_checksums, tmpbuf, strlen(tmpbuf)); + if (DEBUG_GTE(NSTR, 1)) { + rprintf(FINFO, "Client negotiated %s: %s\n", valid_auth_checksums.type, + valid_auth_checksums.negotiated_nni->name); + } + } else { + if (!parse_negotiate_str(&valid_auth_checksums, tmpbuf)) { + get_default_nno_list(&valid_auth_checksums, tmpbuf, MAX_NSTR_STRLEN, '\0'); + io_printf(f_out, "@ERROR: your client does not support one of our daemon-auth checksums: %s\n", + tmpbuf); + exit_cleanup(RERR_UNSUPPORTED); + } + } + am_server = save_am_server; + if (md4_is_old && valid_auth_checksums.negotiated_nni->num == CSUM_MD4) + valid_auth_checksums.negotiated_nni->num = CSUM_MD4_OLD; +} + +int get_subprotocol_version() +{ +#if SUBPROTOCOL_VERSION != 0 + return protocol_version < PROTOCOL_VERSION ? 0 : SUBPROTOCOL_VERSION; +#else + return 0; +#endif +} diff --git a/configure.ac b/configure.ac index b6de4eb3..a2c99558 100644 --- a/configure.ac +++ b/configure.ac @@ -4,7 +4,6 @@ AC_INIT([rsync],[ ],[https://rsync.samba.org/bug-tracking.html]) AC_C_BIGENDIAN AC_HEADER_DIRENT -AC_HEADER_TIME AC_HEADER_SYS_WAIT AC_CHECK_HEADERS(sys/fcntl.h sys/select.h fcntl.h sys/time.h sys/unistd.h \ unistd.h utime.h compat.h sys/param.h ctype.h sys/wait.h sys/stat.h \ @@ -20,7 +19,7 @@ AC_HEADER_MAJOR_FIXED AC_CONFIG_MACRO_DIR([m4]) AC_CONFIG_SRCDIR([byteorder.h]) -AC_CONFIG_HEADER(config.h) +AC_CONFIG_HEADERS([config.h]) AC_PREREQ([2.69]) PACKAGE_VERSION=`sed -n 's/.*RSYNC_VERSION.*"\(.*\)".*/\1/p' <$srcdir/version.h` @@ -61,7 +60,6 @@ AC_PROG_AWK AC_PROG_EGREP AC_PROG_INSTALL AC_PROG_MKDIR_P -AC_PROG_CC_STDC AC_SUBST(SHELL) AC_PATH_PROG([PERL], [perl]) AC_PATH_PROG([PYTHON3], [python3]) @@ -136,6 +134,16 @@ if test x"$GCC" = x"yes"; then CFLAGS="$CFLAGS -Wall -W" fi +AC_ARG_WITH(openssl-conf, + AS_HELP_STRING([--with-openssl-conf=PATH],[set default OPENSSL_CONF path for rsync])) +case "$with_openssl_conf" in + *[^-/a-zA-Z0-9.,=@+_]*) AC_MSG_ERROR([Invalid path given to --with-openssl-conf]) ;; + /*) CFLAGS="$CFLAGS -DSET_OPENSSL_CONF=$with_openssl_conf" ;; + no|'') ;; + yes) AC_MSG_ERROR([No path given to --with-openssl-conf]) ;; + *) AC_MSG_ERROR([Non absolute path given to --with-openssl-conf]) ;; +esac + AC_ARG_WITH(rrsync, AS_HELP_STRING([--with-rrsync],[also install the rrsync script and its manpage])) if test x"$with_rrsync" != x"yes"; then @@ -380,7 +388,7 @@ AS_HELP_STRING([--disable-ipv6],[disable to omit ipv6 support]), ;; esac ], - AC_TRY_RUN([ /* AF_INET6 availability check */ + AC_RUN_IFELSE([AC_LANG_SOURCE([[ /* AF_INET6 availability check */ #include #include #include @@ -391,11 +399,11 @@ main() else exit(0); } -], - AC_MSG_RESULT(yes) - AC_DEFINE(INET6, 1, [true if you have IPv6]), - AC_MSG_RESULT(no), - AC_MSG_RESULT(no) +]])], + [AC_MSG_RESULT(yes) + AC_DEFINE(INET6, 1, true if you have IPv6)], + [AC_MSG_RESULT(no)], + [AC_MSG_RESULT(no)] )) dnl Do you want to disable use of locale functions @@ -518,7 +526,7 @@ fi AC_MSG_CHECKING([whether to enable zstd compression]) AC_ARG_ENABLE([zstd], - AC_HELP_STRING([--disable-zstd], [disable to omit zstd compression])) + AS_HELP_STRING([--disable-zstd], [disable to omit zstd compression])) AH_TEMPLATE([SUPPORT_ZSTD], [Undefine if you do not want zstd compression. By default this is defined.]) if test x"$enable_zstd" != x"no"; then @@ -539,7 +547,7 @@ fi AC_MSG_CHECKING([whether to enable LZ4 compression]) AC_ARG_ENABLE([lz4], - AC_HELP_STRING([--disable-lz4], [disable to omit LZ4 compression])) + AS_HELP_STRING([--disable-lz4], [disable to omit LZ4 compression])) AH_TEMPLATE([SUPPORT_LZ4], [Undefine if you do not want LZ4 compression. By default this is defined.]) if test x"$enable_lz4" != x"no"; then diff --git a/csprotocol.txt b/csprotocol.txt index c8dadd41..7ba09ab0 100644 --- a/csprotocol.txt +++ b/csprotocol.txt @@ -7,39 +7,54 @@ basically a summary of clientserver.c and authenticate.c. This is the protocol used for rsync --daemon; i.e. connections to port 873 rather than invocations over a remote shell. -When the server accepts a connection, it prints a greeting - - @RSYNCD: . - -where is the numeric version (see PROTOCOL_VERSION in rsync.h) -'.' is a literal period, and is the numeric subprotocol -version (see SUBPROTOCOL_VERSION -- it will be 0 for final releases). -Protocols prior to 30 only output alone. The daemon expects -to see a similar greeting back from the client. For protocols prior to -30, an absent "." value is assumed to be 0. For protocol -30, an absent value is a fatal error. The daemon then follows this line -with a free-format text message-of-the-day (if any is defined). +When the server accepts a connection, it prints a newline-terminated +greeting line: + + @RSYNCD: . + +The is the numeric version (see PROTOCOL_VERSION in rsync.h) +The is the numeric subprotocol version (which is 0 for a +final protocol version, as the SUBPROTOCOL_VERSION define discusses). +The names are the authentication digest algorithms that the +daemon supports, listed in order of preference. + +An rsync prior to 3.2.7 omits the digest names. An rsync prior to 3.0.0 +also omits the period and the value. Since a final +protocol has a subprotocol value of 0, a missing subprotocol value is +assumed to be 0 for any protocol prior to 30. It is considered a fatal +error for protocol 30 and above to omit it. It is considered a fatal +error for protocol 32 and above to omit the digest name list (currently +31 is the newest protocol). + +The daemon expects to see a similar greeting line back from the client. +Once received, the daemon follows the opening line with a free-format +text message-of-the-day (if any is defined). The server is now in the connected state. The client can either send -the command +the command: #list -to get a listing of modules, or the name of a module. After this, the +(to get a listing of modules) or the name of a module. After this, the connection is now bound to a particular module. Access per host for this module is now checked, as is per-module connection limits. -If authentication is required to use this module, the server will say +If authentication is required to use this module, the server will say: @RSYNCD: AUTHREQD where is a random string of base64 characters. The client -must respond with +must respond with: -where is the username they claim to be, and is the -base64 form of the MD4 hash of challenge+password. +The is the username they claim to be. The is the +base64 form of the digest hash of the challenge+password string. The +chosen digest method is the most preferred client method that is also in +the server's list. If no digest list was explicitly provided, the side +expecting a list assumes the other side provided either the single name +"md5" (for a negotiated protocol 30 or 31), or the single name "md4" +(for an older protocol). At this point the server applies all remaining constraints before handing control to the client, including switching uid/gid, setting up @@ -76,6 +91,13 @@ stay tuned (or write it yourself!). ------------ Protocol version changes +31 (2013-09-28, 3.1.0) + + Initial release of protocol 31 had no changes. Rsync 3.2.7 + introduced the suffixed list of digest names on the greeting + line. The presence of the list is allowed even if the greeting + indicates an older protocol version number. + 30 (2007-10-04, 3.0.0pre1) The use of a "." number was added to diff --git a/daemon-parm.txt b/daemon-parm.txt index 3b438b02..69034173 100644 --- a/daemon-parm.txt +++ b/daemon-parm.txt @@ -60,9 +60,9 @@ BOOL read_only True BOOL reverse_lookup True BOOL strict_modes True BOOL transfer_logging False -BOOL use_chroot True BOOL write_only False BOOL3 munge_symlinks Unset BOOL3 numeric_ids Unset BOOL3 open_noatime Unset +BOOL3 use_chroot Unset diff --git a/exclude.c b/exclude.c index 5458455b..ffe55b16 100644 --- a/exclude.c +++ b/exclude.c @@ -78,6 +78,10 @@ static filter_rule **mergelist_parents; static int mergelist_cnt = 0; static int mergelist_size = 0; +#define LOCAL_RULE 1 +#define REMOTE_RULE 2 +static uchar cur_elide_value = REMOTE_RULE; + /* Each filter_list_struct describes a singly-linked list by keeping track * of both the head and tail pointers. The list is slightly unusual in that * a parent-dir's content can be appended to the end of the local list in a @@ -220,6 +224,7 @@ static void add_rule(filter_rule_list *listp, const char *pat, unsigned int pat_ slash_cnt++; } } + rule->elide = 0; strlcpy(rule->pattern + pre_len, pat, pat_len + 1); pat_len += pre_len; if (suf_len) { @@ -488,10 +493,11 @@ void add_implied_include(const char *arg, int skip_daemon_module) if (saw_live_open_brkt) maybe_add_literal_brackets_rule(rule, arg_len); if (relative_paths && slash_cnt) { - filter_rule const *ent; - int found = 0; - slash_cnt = 1; - for (p = new_pat + 1; (p = strchr(p, '/')) != NULL; p++) { + int sub_slash_cnt = slash_cnt; + while ((p = strrchr(new_pat, '/')) != NULL && p != new_pat) { + filter_rule const *ent; + filter_rule *R_rule; + int found = 0; *p = '\0'; for (ent = implied_filter_list.head; ent; ent = ent->next) { if (ent != rule && strcmp(ent->pattern, new_pat) == 0) { @@ -499,25 +505,29 @@ void add_implied_include(const char *arg, int skip_daemon_module) break; } } - if (!found) { - filter_rule *R_rule = new0(filter_rule); - R_rule->rflags = FILTRULE_INCLUDE | FILTRULE_DIRECTORY; - /* Check if our sub-path has wildcards or escaped backslashes */ - if (saw_wild && strpbrk(rule->pattern, "*[?\\")) - R_rule->rflags |= FILTRULE_WILD; - R_rule->pattern = strdup(new_pat); - R_rule->u.slash_cnt = slash_cnt; - R_rule->next = implied_filter_list.head; - implied_filter_list.head = R_rule; - if (DEBUG_GTE(FILTER, 3)) { - rprintf(FINFO, "[%s] add_implied_include(%s/)\n", - who_am_i(), R_rule->pattern); - } - if (saw_live_open_brkt) - maybe_add_literal_brackets_rule(R_rule, -1); + if (found) { + *p = '/'; + break; /* We added all parent dirs already */ } + R_rule = new0(filter_rule); + R_rule->rflags = FILTRULE_INCLUDE | FILTRULE_DIRECTORY; + /* Check if our sub-path has wildcards or escaped backslashes */ + if (saw_wild && strpbrk(new_pat, "*[?\\")) + R_rule->rflags |= FILTRULE_WILD; + R_rule->pattern = strdup(new_pat); + R_rule->u.slash_cnt = --sub_slash_cnt; + R_rule->next = implied_filter_list.head; + implied_filter_list.head = R_rule; + if (DEBUG_GTE(FILTER, 3)) { + rprintf(FINFO, "[%s] add_implied_include(%s/)\n", + who_am_i(), R_rule->pattern); + } + if (saw_live_open_brkt) + maybe_add_literal_brackets_rule(R_rule, -1); + } + for (p = new_pat; sub_slash_cnt < slash_cnt; sub_slash_cnt++) { + p += strlen(p); *p = '/'; - slash_cnt++; } } } @@ -545,15 +555,12 @@ void add_implied_include(const char *arg, int skip_daemon_module) p += arg_len; } } - if (p[-1] != '/') { - *p++ = '/'; - slash_cnt++; - } + *p++ = '/'; *p++ = '*'; if (recurse) *p++ = '*'; *p = '\0'; - rule->u.slash_cnt = slash_cnt; + rule->u.slash_cnt = slash_cnt + 1; rule->next = implied_filter_list.head; implied_filter_list.head = rule; if (DEBUG_GTE(FILTER, 3)) @@ -900,7 +907,7 @@ static int rule_matches(const char *fname, filter_rule *ex, int name_flags) const char *strings[16]; /* more than enough */ const char *name = fname + (*fname == '/'); - if (!*name) + if (!*name || ex->elide == cur_elide_value) return 0; if (!(name_flags & NAME_IS_XATTR) ^ !(ex->rflags & FILTRULE_XATTR)) @@ -1016,6 +1023,15 @@ int name_is_excluded(const char *fname, int name_flags, int filter_level) return 0; } +int check_server_filter(filter_rule_list *listp, enum logcode code, const char *name, int name_flags) +{ + int ret; + cur_elide_value = LOCAL_RULE; + ret = check_filter(listp, code, name, name_flags); + cur_elide_value = REMOTE_RULE; + return ret; +} + /* Return -1 if file "name" is defined to be excluded by the specified * exclude list, 1 if it is included, and 0 if it was not matched. */ int check_filter(filter_rule_list *listp, enum logcode code, @@ -1571,7 +1587,7 @@ char *get_rule_prefix(filter_rule *rule, const char *pat, int for_xfer, static void send_rules(int f_out, filter_rule_list *flp) { - filter_rule *ent, *prev = NULL; + filter_rule *ent; for (ent = flp->head; ent; ent = ent->next) { unsigned int len, plen, dlen; @@ -1586,21 +1602,15 @@ static void send_rules(int f_out, filter_rule_list *flp) * merge files as an optimization (since they can only have * include/exclude rules). */ if (ent->rflags & FILTRULE_SENDER_SIDE) - elide = am_sender ? 1 : -1; + elide = am_sender ? LOCAL_RULE : REMOTE_RULE; if (ent->rflags & FILTRULE_RECEIVER_SIDE) - elide = elide ? 0 : am_sender ? -1 : 1; + elide = elide ? 0 : am_sender ? REMOTE_RULE : LOCAL_RULE; else if (delete_excluded && !elide && (!(ent->rflags & FILTRULE_PERDIR_MERGE) || ent->rflags & FILTRULE_NO_PREFIXES)) - elide = am_sender ? 1 : -1; - if (elide < 0) { - if (prev) - prev->next = ent->next; - else - flp->head = ent->next; - } else - prev = ent; - if (elide > 0) + elide = am_sender ? LOCAL_RULE : REMOTE_RULE; + ent->elide = elide; + if (elide == LOCAL_RULE) continue; if (ent->rflags & FILTRULE_CVS_IGNORE && !(ent->rflags & FILTRULE_MERGE_FILE)) { @@ -1628,7 +1638,6 @@ static void send_rules(int f_out, filter_rule_list *flp) if (dlen) write_byte(f_out, '/'); } - flp->tail = prev; } /* This is only called by the client. */ diff --git a/flist.c b/flist.c index 0e6bf782..65b459b1 100644 --- a/flist.c +++ b/flist.c @@ -33,7 +33,6 @@ extern int am_sender; extern int am_generator; extern int inc_recurse; extern int always_checksum; -extern int checksum_type; extern int module_id; extern int ignore_errors; extern int numeric_ids; @@ -80,6 +79,8 @@ extern struct stats stats; extern char *filesfrom_host; extern char *usermap, *groupmap; +extern struct name_num_item *file_sum_nni; + extern char curr_dir[MAXPATHLEN]; extern struct chmod_mode_struct *chmod_modes; @@ -145,7 +146,8 @@ void init_flist(void) rprintf(FINFO, "FILE_STRUCT_LEN=%d, EXTRA_LEN=%d\n", (int)FILE_STRUCT_LEN, (int)EXTRA_LEN); } - flist_csum_len = csum_len_for_type(checksum_type, 1); + /* Note that this isn't identical to file_sum_len in the case of CSUM_MD4_ARCHAIC: */ + flist_csum_len = csum_len_for_type(file_sum_nni->num, 1); show_filelist_progress = INFO_GTE(FLIST, 1) && xfer_dirs && !am_server && !inc_recurse; } @@ -754,7 +756,7 @@ static struct file_struct *recv_file_entry(int f, struct file_list *flist, int x if (*thisname && (clean_fname(thisname, CFN_REFUSE_DOT_DOT_DIRS) < 0 || (!relative_paths && *thisname == '/'))) { rprintf(FERROR, "ABORTING due to unsafe pathname from sender: %s\n", thisname); - exit_cleanup(RERR_PROTOCOL); + exit_cleanup(RERR_UNSUPPORTED); } if (sanitize_paths) @@ -834,7 +836,7 @@ static struct file_struct *recv_file_entry(int f, struct file_list *flist, int x } #endif } else - modtime = read_int(f); + modtime = read_uint(f); } if (xflags & XMIT_MOD_NSEC) #ifndef CAN_SET_NSEC @@ -986,16 +988,16 @@ static struct file_struct *recv_file_entry(int f, struct file_list *flist, int x exit_cleanup(RERR_UNSUPPORTED); } - if (*thisname != '.' || thisname[1] != '\0') { + if (*thisname == '/' ? thisname[1] != '.' || thisname[2] != '\0' : *thisname != '.' || thisname[1] != '\0') { int filt_flags = S_ISDIR(mode) ? NAME_IS_DIR : NAME_IS_FILE; if (!trust_sender_filter /* a per-dir filter rule means we must trust the sender's filtering */ - && filter_list.head && check_filter(&filter_list, FINFO, thisname, filt_flags) < 0) { + && filter_list.head && check_server_filter(&filter_list, FINFO, thisname, filt_flags) < 0) { rprintf(FERROR, "ERROR: rejecting excluded file-list name: %s\n", thisname); - exit_cleanup(RERR_PROTOCOL); + exit_cleanup(RERR_UNSUPPORTED); } if (implied_filter_list.head && check_filter(&implied_filter_list, FINFO, thisname, filt_flags) <= 0) { rprintf(FERROR, "ERROR: rejecting unrequested file-list name: %s\n", thisname); - exit_cleanup(RERR_PROTOCOL); + exit_cleanup(RERR_UNSUPPORTED); } } @@ -2640,7 +2642,7 @@ struct file_list *recv_file_list(int f, int dir_ndx) rprintf(FERROR, "ABORTING due to invalid path from sender: %s/%s\n", cur_dir, file->basename); - exit_cleanup(RERR_PROTOCOL); + exit_cleanup(RERR_UNSUPPORTED); } good_dirname = cur_dir; } diff --git a/generator.c b/generator.c index 935c84f9..21c4a595 100644 --- a/generator.c +++ b/generator.c @@ -875,9 +875,12 @@ static struct file_struct *find_fuzzy(struct file_struct *file, struct file_list len = strlen(name); suf = find_filename_suffix(name, len, &suf_len); - dist = fuzzy_distance(name, len, fname, fname_len); - /* Add some extra weight to how well the suffixes match. */ - dist += fuzzy_distance(suf, suf_len, fname_suf, fname_suf_len) * 10; + dist = fuzzy_distance(name, len, fname, fname_len, lowest_dist); + /* Add some extra weight to how well the suffixes match unless we've already disqualified + * this file based on a heuristic. */ + if (dist < 0xFFFF0000U) { + dist += fuzzy_distance(suf, suf_len, fname_suf, fname_suf_len, 0xFFFF0000U) * 10; + } if (DEBUG_GTE(FUZZY, 2)) { rprintf(FINFO, "fuzzy distance for %s = %d.%05d\n", f_name(fp, NULL), (int)(dist>>16), (int)(dist&0xFFFF)); diff --git a/hashtable.c b/hashtable.c index e272f439..2cc4e550 100644 --- a/hashtable.c +++ b/hashtable.c @@ -1,7 +1,7 @@ /* * Routines to provide a memory-efficient hashtable. * - * Copyright (C) 2007-2020 Wayne Davison + * Copyright (C) 2007-2022 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -350,6 +350,9 @@ void *hashtable_find(struct hashtable *tbl, int64 key, void *data_when_new) ------------------------------------------------------------------------------- */ +#define NON_ZERO_32(x) ((x) ? (x) : (uint32_t)1) +#define NON_ZERO_64(x, y) ((x) || (y) ? (y) | (int64)(x) << 32 | (y) : (int64)1) + uint32_t hashlittle(const void *key, size_t length) { uint32_t a,b,c; /* internal state */ @@ -390,7 +393,7 @@ uint32_t hashlittle(const void *key, size_t length) case 3 : a+=((uint32_t)k8[2])<<16; /* fall through */ case 2 : a+=((uint32_t)k8[1])<<8; /* fall through */ case 1 : a+=k8[0]; break; - case 0 : return c; + case 0 : return NON_ZERO_32(c); } } else if (HASH_LITTLE_ENDIAN && ((u.i & 0x1) == 0)) { const uint16_t *k = (const uint16_t *)key; /* read 16-bit chunks */ @@ -436,7 +439,7 @@ uint32_t hashlittle(const void *key, size_t length) break; case 1 : a+=k8[0]; break; - case 0 : return c; /* zero length requires no mixing */ + case 0 : return NON_ZERO_32(c); /* zero length requires no mixing */ } } else { /* need to read the key one byte at a time */ @@ -489,10 +492,171 @@ uint32_t hashlittle(const void *key, size_t length) /* FALLTHROUGH */ case 1 : a+=k[0]; break; - case 0 : return c; + case 0 : return NON_ZERO_32(c); } } final(a,b,c); - return c; + return NON_ZERO_32(c); } + +#if SIZEOF_INT64 >= 8 +/* + * hashlittle2: return 2 32-bit hash values joined into an int64. + * + * This is identical to hashlittle(), except it returns two 32-bit hash + * values instead of just one. This is good enough for hash table + * lookup with 2^^64 buckets, or if you want a second hash if you're not + * happy with the first, or if you want a probably-unique 64-bit ID for + * the key. *pc is better mixed than *pb, so use *pc first. If you want + * a 64-bit value do something like "*pc + (((uint64_t)*pb)<<32)". + */ +int64 hashlittle2(const void *key, size_t length) +{ + uint32_t a,b,c; /* internal state */ + union { const void *ptr; size_t i; } u; /* needed for Mac Powerbook G4 */ + + /* Set up the internal state */ + a = b = c = 0xdeadbeef + ((uint32_t)length); + + u.ptr = key; + if (HASH_LITTLE_ENDIAN && ((u.i & 0x3) == 0)) { + const uint32_t *k = (const uint32_t *)key; /* read 32-bit chunks */ + const uint8_t *k8; + + /*------ all but last block: aligned reads and affect 32 bits of (a,b,c) */ + while (length > 12) + { + a += k[0]; + b += k[1]; + c += k[2]; + mix(a,b,c); + length -= 12; + k += 3; + } + + /*----------------------------- handle the last (probably partial) block */ + k8 = (const uint8_t *)k; + switch(length) + { + case 12: c+=k[2]; b+=k[1]; a+=k[0]; break; + case 11: c+=((uint32_t)k8[10])<<16; /* fall through */ + case 10: c+=((uint32_t)k8[9])<<8; /* fall through */ + case 9 : c+=k8[8]; /* fall through */ + case 8 : b+=k[1]; a+=k[0]; break; + case 7 : b+=((uint32_t)k8[6])<<16; /* fall through */ + case 6 : b+=((uint32_t)k8[5])<<8; /* fall through */ + case 5 : b+=k8[4]; /* fall through */ + case 4 : a+=k[0]; break; + case 3 : a+=((uint32_t)k8[2])<<16; /* fall through */ + case 2 : a+=((uint32_t)k8[1])<<8; /* fall through */ + case 1 : a+=k8[0]; break; + case 0 : return NON_ZERO_64(b, c); + } + } else if (HASH_LITTLE_ENDIAN && ((u.i & 0x1) == 0)) { + const uint16_t *k = (const uint16_t *)key; /* read 16-bit chunks */ + const uint8_t *k8; + + /*--------------- all but last block: aligned reads and different mixing */ + while (length > 12) + { + a += k[0] + (((uint32_t)k[1])<<16); + b += k[2] + (((uint32_t)k[3])<<16); + c += k[4] + (((uint32_t)k[5])<<16); + mix(a,b,c); + length -= 12; + k += 6; + } + + /*----------------------------- handle the last (probably partial) block */ + k8 = (const uint8_t *)k; + switch(length) + { + case 12: c+=k[4]+(((uint32_t)k[5])<<16); + b+=k[2]+(((uint32_t)k[3])<<16); + a+=k[0]+(((uint32_t)k[1])<<16); + break; + case 11: c+=((uint32_t)k8[10])<<16; /* fall through */ + case 10: c+=k[4]; + b+=k[2]+(((uint32_t)k[3])<<16); + a+=k[0]+(((uint32_t)k[1])<<16); + break; + case 9 : c+=k8[8]; /* fall through */ + case 8 : b+=k[2]+(((uint32_t)k[3])<<16); + a+=k[0]+(((uint32_t)k[1])<<16); + break; + case 7 : b+=((uint32_t)k8[6])<<16; /* fall through */ + case 6 : b+=k[2]; + a+=k[0]+(((uint32_t)k[1])<<16); + break; + case 5 : b+=k8[4]; /* fall through */ + case 4 : a+=k[0]+(((uint32_t)k[1])<<16); + break; + case 3 : a+=((uint32_t)k8[2])<<16; /* fall through */ + case 2 : a+=k[0]; + break; + case 1 : a+=k8[0]; + break; + case 0 : return NON_ZERO_64(b, c); /* zero length strings require no mixing */ + } + + } else { /* need to read the key one byte at a time */ + const uint8_t *k = (const uint8_t *)key; + + /*--------------- all but the last block: affect some 32 bits of (a,b,c) */ + while (length > 12) + { + a += k[0]; + a += ((uint32_t)k[1])<<8; + a += ((uint32_t)k[2])<<16; + a += ((uint32_t)k[3])<<24; + b += k[4]; + b += ((uint32_t)k[5])<<8; + b += ((uint32_t)k[6])<<16; + b += ((uint32_t)k[7])<<24; + c += k[8]; + c += ((uint32_t)k[9])<<8; + c += ((uint32_t)k[10])<<16; + c += ((uint32_t)k[11])<<24; + mix(a,b,c); + length -= 12; + k += 12; + } + + /*-------------------------------- last block: affect all 32 bits of (c) */ + switch(length) /* all the case statements fall through */ + { + case 12: c+=((uint32_t)k[11])<<24; + /* FALLTHROUGH */ + case 11: c+=((uint32_t)k[10])<<16; + /* FALLTHROUGH */ + case 10: c+=((uint32_t)k[9])<<8; + /* FALLTHROUGH */ + case 9 : c+=k[8]; + /* FALLTHROUGH */ + case 8 : b+=((uint32_t)k[7])<<24; + /* FALLTHROUGH */ + case 7 : b+=((uint32_t)k[6])<<16; + /* FALLTHROUGH */ + case 6 : b+=((uint32_t)k[5])<<8; + /* FALLTHROUGH */ + case 5 : b+=k[4]; + /* FALLTHROUGH */ + case 4 : a+=((uint32_t)k[3])<<24; + /* FALLTHROUGH */ + case 3 : a+=((uint32_t)k[2])<<16; + /* FALLTHROUGH */ + case 2 : a+=((uint32_t)k[1])<<8; + /* FALLTHROUGH */ + case 1 : a+=k[0]; + break; + case 0 : return NON_ZERO_64(b, c); + } + } + + final(a,b,c); + return NON_ZERO_64(b, c); +} +#else +#define hashlittle2(key, len) hashlittle(key, len) +#endif diff --git a/io.c b/io.c index f3d802ec..a99ac0ec 100644 --- a/io.c +++ b/io.c @@ -1784,6 +1784,13 @@ int32 read_int(int f) return num; } +uint32 read_uint(int f) +{ + char b[4]; + read_buf(f, b, 4); + return IVAL(b, 0); +} + int32 read_varint(int f) { union { diff --git a/lib/md-defines.h b/lib/md-defines.h index 1410af5f..6ef6a689 100644 --- a/lib/md-defines.h +++ b/lib/md-defines.h @@ -1,11 +1,28 @@ /* Keep this simple so both C and ASM can use it */ +/* These allow something like CFLAGS=-DDISABLE_SHA512_DIGEST */ +#ifdef DISABLE_SHA256_DIGEST +#undef SHA256_DIGEST_LENGTH +#endif +#ifdef DISABLE_SHA512_DIGEST +#undef SHA512_DIGEST_LENGTH +#endif + #define MD4_DIGEST_LEN 16 #define MD5_DIGEST_LEN 16 +#if defined SHA512_DIGEST_LENGTH +#define MAX_DIGEST_LEN SHA512_DIGEST_LENGTH +#elif defined SHA256_DIGEST_LENGTH +#define MAX_DIGEST_LEN SHA256_DIGEST_LENGTH +#elif defined SHA_DIGEST_LENGTH +#define MAX_DIGEST_LEN SHA_DIGEST_LENGTH +#else #define MAX_DIGEST_LEN MD5_DIGEST_LEN +#endif #define CSUM_CHUNK 64 +#define CSUM_gone -1 #define CSUM_NONE 0 #define CSUM_MD4_ARCHAIC 1 #define CSUM_MD4_BUSTED 2 @@ -15,3 +32,6 @@ #define CSUM_XXH64 6 #define CSUM_XXH3_64 7 #define CSUM_XXH3_128 8 +#define CSUM_SHA1 9 +#define CSUM_SHA256 10 +#define CSUM_SHA512 11 diff --git a/lib/md5.c b/lib/md5.c index 2366339e..f36c5ba7 100644 --- a/lib/md5.c +++ b/lib/md5.c @@ -20,7 +20,6 @@ #include "rsync.h" -#if !defined USE_OPENSSL || USE_MD5_ASM /* { */ void md5_begin(md_context *ctx) { ctx->A = 0x67452301; @@ -224,7 +223,6 @@ void md5_result(md_context *ctx, uchar digest[MD5_DIGEST_LEN]) SIVALu(digest, 8, ctx->C); SIVALu(digest, 12, ctx->D); } -#endif /* } */ #ifdef TEST_MD5 /* { */ diff --git a/lib/mdigest.h b/lib/mdigest.h index f1d6d934..9d52ef5f 100644 --- a/lib/mdigest.h +++ b/lib/mdigest.h @@ -1,8 +1,8 @@ /* The include file for both the MD4 and MD5 routines. */ #ifdef USE_OPENSSL -#include "openssl/md4.h" -#include "openssl/md5.h" +#include +#include #endif #include "md-defines.h" @@ -17,14 +17,6 @@ void mdfour_begin(md_context *md); void mdfour_update(md_context *md, const uchar *in, uint32 length); void mdfour_result(md_context *md, uchar digest[MD4_DIGEST_LEN]); -#if defined USE_OPENSSL && !defined USE_MD5_ASM -#define md5_context MD5_CTX -#define md5_begin MD5_Init -#define md5_update MD5_Update -#define md5_result(cptr, digest) MD5_Final(digest, cptr) -#else -#define md5_context md_context void md5_begin(md_context *ctx); void md5_update(md_context *ctx, const uchar *input, uint32 length); void md5_result(md_context *ctx, uchar digest[MD5_DIGEST_LEN]); -#endif diff --git a/log.c b/log.c index 44344e2a..e4ba1cce 100644 --- a/log.c +++ b/log.c @@ -36,8 +36,6 @@ extern int protocol_version; extern int always_checksum; extern int preserve_mtimes; extern int msgs2stderr; -extern int xfersum_type; -extern int checksum_type; extern int stdout_format_has_i; extern int stdout_format_has_o_or_i; extern int logfile_format_has_i; @@ -62,6 +60,8 @@ extern unsigned int module_dirlen; extern char sender_file_sum[MAX_DIGEST_LEN]; extern const char undetermined_hostname[]; +extern struct name_num_item *xfer_sum_nni, *file_sum_nni; + static int log_initialised; static int logfile_was_closed; static FILE *logfile_fp; @@ -680,12 +680,12 @@ static void log_formatted(enum logcode code, const char *format, const char *op, n = NULL; if (S_ISREG(file->mode)) { if (always_checksum) - n = sum_as_hex(checksum_type, F_SUM(file), 1); + n = sum_as_hex(file_sum_nni->num, F_SUM(file), 1); else if (iflags & ITEM_TRANSFER) - n = sum_as_hex(xfersum_type, sender_file_sum, 0); + n = sum_as_hex(xfer_sum_nni->num, sender_file_sum, 0); } if (!n) { - int sum_len = csum_len_for_type(always_checksum ? checksum_type : xfersum_type, + int sum_len = csum_len_for_type(always_checksum ? file_sum_nni->num : xfer_sum_nni->num, always_checksum); memset(buf2, ' ', sum_len*2); buf2[sum_len*2] = '\0'; diff --git a/m4/have_type.m4 b/m4/have_type.m4 index 704ca33b..12fc719e 100644 --- a/m4/have_type.m4 +++ b/m4/have_type.m4 @@ -1,6 +1,5 @@ dnl AC_HAVE_TYPE(TYPE,INCLUDES) AC_DEFUN([AC_HAVE_TYPE], [ -AC_REQUIRE([AC_HEADER_STDC]) cv=`echo "$1" | sed 'y%./+- %__p__%'` AC_MSG_CHECKING(for $1) AC_CACHE_VAL([ac_cv_type_$cv], diff --git a/main.c b/main.c index 9ebfbea7..d2a7b9b5 100644 --- a/main.c +++ b/main.c @@ -660,6 +660,16 @@ static pid_t do_cmd(char *cmd, char *machine, char *user, char **remote_argv, in return pid; } +/* Older versions turn an empty string as a reference to the current directory. + * We now treat this as an error unless --old-args was used. */ +static char *dot_dir_or_error() +{ + if (old_style_args || am_server) + return "."; + rprintf(FERROR, "Empty destination arg specified (use \".\" or see --old-args).\n"); + exit_cleanup(RERR_SYNTAX); +} + /* The receiving side operates in one of two modes: * * 1. it receives any number of files into a destination directory, @@ -687,9 +697,8 @@ static char *get_local_name(struct file_list *flist, char *dest_path) if (!dest_path || list_only) return NULL; - /* Treat an empty string as a copy into the current directory. */ if (!*dest_path) - dest_path = "."; + dest_path = dot_dir_or_error(); if (daemon_filter_list.head) { char *slash = strrchr(dest_path, '/'); @@ -1432,6 +1441,8 @@ static int start_client(int argc, char *argv[]) if (argc > 1) { p = argv[--argc]; + if (!*p) + p = dot_dir_or_error(); remote_argv = argv + argc; } else { static char *dotarg[1] = { "." }; @@ -1743,6 +1754,17 @@ int main(int argc,char *argv[]) unset_env_var("DISPLAY"); +#if defined USE_OPENSSL && defined SET_OPENSSL_CONF +#define TO_STR2(x) #x +#define TO_STR(x) TO_STR2(x) + /* ./configure --with-openssl-conf=/etc/ssl/openssl-rsync.cnf + * defines SET_OPENSSL_CONF as that unquoted pathname. */ + if (!getenv("OPENSSL_CONF")) /* Don't override it if it's already set. */ + set_env_str("OPENSSL_CONF", TO_STR(SET_OPENSSL_CONF)); +#undef TO_STR +#undef TO_STR2 +#endif + memset(&stats, 0, sizeof(stats)); /* Even a non-daemon runs needs the default config values to be set, e.g. diff --git a/match.c b/match.c index 9d5c9259..6243994c 100644 --- a/match.c +++ b/match.c @@ -3,7 +3,7 @@ * * Copyright (C) 1996 Andrew Tridgell * Copyright (C) 1996 Paul Mackerras - * Copyright (C) 2003-2020 Wayne Davison + * Copyright (C) 2003-2022 Wayne Davison * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -24,7 +24,9 @@ extern int checksum_seed; extern int append_mode; -extern int xfersum_type; + +extern struct name_num_item *xfer_sum_nni; +extern int xfer_sum_len; int updating_basis_file; char sender_file_sum[MAX_DIGEST_LEN]; @@ -356,15 +358,13 @@ static void hash_search(int f,struct sum_struct *s, **/ void match_sums(int f, struct sum_struct *s, struct map_struct *buf, OFF_T len) { - int sum_len; - last_match = 0; false_alarms = 0; hash_hits = 0; matches = 0; data_transfer = 0; - sum_init(xfersum_type, checksum_seed); + sum_init(xfer_sum_nni, checksum_seed); if (append_mode > 0) { if (append_mode == 2) { @@ -405,22 +405,22 @@ void match_sums(int f, struct sum_struct *s, struct map_struct *buf, OFF_T len) matched(f, s, buf, len, -1); } - sum_len = sum_end(sender_file_sum); + sum_end(sender_file_sum); /* If we had a read error, send a bad checksum. We use all bits * off as long as the checksum doesn't happen to be that, in * which case we turn the last 0 bit into a 1. */ if (buf && buf->status != 0) { int i; - for (i = 0; i < sum_len && sender_file_sum[i] == 0; i++) {} - memset(sender_file_sum, 0, sum_len); - if (i == sum_len) + for (i = 0; i < xfer_sum_len && sender_file_sum[i] == 0; i++) {} + memset(sender_file_sum, 0, xfer_sum_len); + if (i == xfer_sum_len) sender_file_sum[i-1]++; } if (DEBUG_GTE(DELTASUM, 2)) rprintf(FINFO,"sending file_sum\n"); - write_buf(f, sender_file_sum, sum_len); + write_buf(f, sender_file_sum, xfer_sum_len); if (DEBUG_GTE(DELTASUM, 2)) { rprintf(FINFO, "false_alarms=%d hash_hits=%d matches=%d\n", diff --git a/mkgitver b/mkgitver index fe8a3d15..0102b089 100755 --- a/mkgitver +++ b/mkgitver @@ -6,9 +6,11 @@ if [ ! -f git-version.h ]; then touch git-version.h fi -if [ -e "$srcdir/.git" ]; then - gitver=`git describe --abbrev=8 2>/dev/null | sed -n '/^v3\.[0-9][0-9]*\.[0-9][0-9]*\(-\|$\)/p'` - if [ -n "$gitver" ]; then +if test -d "$srcdir/.git" || test -f "$srcdir/.git"; then + gitver=`git describe --abbrev=8 2>/dev/null` + # NOTE: I'm avoiding "|" in sed since I'm not sure if sed -r is portable and "\|" fails on some OSes. + verchk=`echo "$gitver-" | sed -n '/^v3\.[0-9][0-9]*\.[0-9][0-9]*\(pre[0-9]*\)*-/p'` + if [ -n "$verchk" ]; then echo "#define RSYNC_GITVER \"$gitver\"" >git-version.h.new if ! diff git-version.h.new git-version.h >/dev/null; then echo "Updating git-version.h" diff --git a/options.c b/options.c index 3f8d5d08..d38bbe8d 100644 --- a/options.c +++ b/options.c @@ -1926,7 +1926,7 @@ int parse_arguments(int *argc_p, const char ***argv_p) saw_stderr_opt = 1; if (version_opt_cnt) { - print_rsync_version(FINFO); + print_rsync_version(version_opt_cnt > 1 && !am_server ? FNONE : FINFO); exit_cleanup(0); } diff --git a/packaging/lsb/rsync.spec b/packaging/lsb/rsync.spec index 8221547f..f2d7aa44 100644 --- a/packaging/lsb/rsync.spec +++ b/packaging/lsb/rsync.spec @@ -1,6 +1,6 @@ Summary: A fast, versatile, remote (and local) file-copying tool Name: rsync -Version: 3.2.6 +Version: 3.2.7 %define fullversion %{version} Release: 1 %define srcdir src @@ -79,8 +79,8 @@ rm -rf $RPM_BUILD_ROOT %dir /etc/rsync-ssl/certs %changelog -* Fri Sep 09 2022 Wayne Davison -Released 3.2.6. +* Thu Oct 20 2022 Wayne Davison +Released 3.2.7. * Fri Mar 21 2008 Wayne Davison Added installation of /etc/xinetd.d/rsync file and some commented-out diff --git a/packaging/openssl-rsync.cnf b/packaging/openssl-rsync.cnf new file mode 100644 index 00000000..7432285d --- /dev/null +++ b/packaging/openssl-rsync.cnf @@ -0,0 +1,18 @@ +# This config file can be used with rsync to enable legacy digests +# (such as MD4) by using the OPENSSL_CONF environment variable. +# See rsync's configure --with-openssl-conf=/path/name option. + +openssl_conf = openssl_init + +[openssl_init] +providers = provider_sect + +[provider_sect] +default = default_sect +legacy = legacy_sect + +[default_sect] +activate = 1 + +[legacy_sect] +activate = 1 diff --git a/packaging/pkglib.py b/packaging/pkglib.py index 6f5557aa..c4c5741d 100644 --- a/packaging/pkglib.py +++ b/packaging/pkglib.py @@ -176,7 +176,7 @@ def mandate_gensend_hook(): print('Creating hook file:', hook) cmd_chk(['./rsync', '-a', 'packaging/pre-push', hook]) else: - ct = cmd_txt(['fgrep', 'make gensend', hook], discard='output') + ct = cmd_txt(['grep', 'make gensend', hook], discard='output') if ct.rc: die('Please add a "make gensend" into your', hook, 'script.') diff --git a/packaging/release-rsync b/packaging/release-rsync index f37bd184..511e90f1 100755 --- a/packaging/release-rsync +++ b/packaging/release-rsync @@ -105,6 +105,8 @@ def main(): if not re.match(r'^del', ans, flags=re.I): die("Aborted") cmd_chk(['git', 'tag', '-d', v_ver]) + if os.path.isdir('patches/.git'): + cmd_chk(f"cd patches && git tag -d '{v_ver}'") version = re.sub(r'[-.]*pre[-.]*', 'pre', version) if 'pre' in version and not curversion.endswith('dev'): @@ -254,7 +256,7 @@ About to: """) ans = input(" ") - s = cmd_run(['git', 'commit', '-a', '-m', f'Preparing for release of {version}']) + s = cmd_run(['git', 'commit', '-a', '-m', f'Preparing for release of {version} [buildall]']) if s.returncode: die('Aborting') diff --git a/receiver.c b/receiver.c index 0f5d92d2..c9d7e01d 100644 --- a/receiver.c +++ b/receiver.c @@ -56,7 +56,6 @@ extern int inplace; extern int inplace_partial; extern int allowed_lull; extern int delay_updates; -extern int xfersum_type; extern BOOL want_progress_now; extern mode_t orig_umask; extern struct stats stats; @@ -68,6 +67,9 @@ extern struct file_list *cur_flist, *first_flist, *dir_flist; extern filter_rule_list daemon_filter_list; extern OFF_T preallocated_len; +extern struct name_num_item *xfer_sum_nni; +extern int xfer_sum_len; + static struct bitbag *delayed_bits = NULL; static int phase = 0, redoing = 0; static flist_ndx_list batch_redo_list; @@ -240,7 +242,6 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r, static char file_sum1[MAX_DIGEST_LEN]; struct map_struct *mapbuf; struct sum_struct sum; - int sum_len; int32 len; OFF_T total_size = F_LENGTH(file); OFF_T offset = 0; @@ -280,7 +281,7 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r, } else mapbuf = NULL; - sum_init(xfersum_type, checksum_seed); + sum_init(xfer_sum_nni, checksum_seed); if (append_mode > 0) { OFF_T j; @@ -393,7 +394,7 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r, if (INFO_GTE(PROGRESS, 1)) end_progress(total_size); - sum_len = sum_end(file_sum1); + sum_end(file_sum1); if (do_fsync && fd != -1 && fsync(fd) != 0) { rsyserr(FERROR, errno, "fsync failed on %s", full_fname(fname)); @@ -403,10 +404,10 @@ static int receive_data(int f_in, char *fname_r, int fd_r, OFF_T size_r, if (mapbuf) unmap_file(mapbuf); - read_buf(f_in, sender_file_sum, sum_len); + read_buf(f_in, sender_file_sum, xfer_sum_len); if (DEBUG_GTE(DELTASUM, 2)) rprintf(FINFO,"got file_sum\n"); - if (fd != -1 && memcmp(file_sum1, sender_file_sum, sum_len) != 0) + if (fd != -1 && memcmp(file_sum1, sender_file_sum, xfer_sum_len) != 0) return 0; return 1; } diff --git a/rsync.1.md b/rsync.1.md index c62e82ea..ee0a4f39 100644 --- a/rsync.1.md +++ b/rsync.1.md @@ -152,7 +152,43 @@ rsync daemon by leaving off the module name: > rsync somehost.mydomain.com:: -See the following section for more details. +## COPYING TO A DIFFERENT NAME + +When you want to copy a directory to a different name, use a trailing slash on +the source directory to put the contents of the directory into any destination +directory you like: + +> rsync -ai foo/ bar/ + +Rsync also has the ability to customize a destination file's name when copying +a single item. The rules for this are: + +- The transfer list must consist of a single item (either a file or an empty + directory) +- The final element of the destination path must not exist as a directory +- The destination path must not have been specified with a trailing slash + +Under those circumstances, rsync will set the name of the destination's single +item to the last element of the destination path. Keep in mind that it is best +to only use this idiom when copying a file and use the above trailing-slash +idiom when copying a directory. + +The following example copies the `foo.c` file as `bar.c` in the `save` dir +(assuming that `bar.c` isn't a directory): + +> rsync -ai src/foo.c save/bar.c + +The single-item copy rule might accidentally bite you if you unknowingly copy a +single item and specify a destination dir that doesn't exist (without using a +trailing slash). For example, if `src/*.c` matches one file and `save/dir` +doesn't exist, this will confuse you by naming the destination file `save/dir`: + +> rsync -ai src/*.c save/dir + +To prevent such an accident, either make sure the destination dir exists or +specify the destination path with a trailing slash: + +> rsync -ai src/*.c save/dir/ ## SORTED TRANSFER ORDER @@ -400,7 +436,7 @@ has its own detailed description later in this manpage. --append-verify --append w/old data in file checksum --dirs, -d transfer directories without recursing --old-dirs, --old-d works like --dirs when talking to old rsync ---mkpath create the destination's path component +--mkpath create destination's missing path components --links, -l copy symlinks as symlinks --copy-links, -L transform symlink into referent file/dir --copy-unsafe-links only "unsafe" symlinks are transformed @@ -580,11 +616,14 @@ expand it. 0. `--version`, `-V` - Print the rsync version plus other info and exit. + Print the rsync version plus other info and exit. When repeated, the + information is output is a JSON format that is still fairly readable + (client side only). - The output includes the default list of checksum algorithms, the default - list of compression algorithms, a list of compiled-in capabilities, a link - to the rsync web site, and some license/copyright info. + The output includes a list of compiled-in capabilities, a list of + optimizations, the default list of checksum algorithms, the default list of + compression algorithms, the default list of daemon auth digests, a link to + the rsync web site, and a few other items. 0. `--verbose`, `-v` @@ -856,7 +895,7 @@ expand it. that until a bunch of recursive copying has finished). However, these early directories don't yet have their completed mode, mtime, or ownership set -- they have more restrictive rights until the subdirectory's copying - actually begins. This early-creation idiom can be avoiding by using the + actually begins. This early-creation idiom can be avoided by using the [`--omit-dir-times`](#opt) option. Incremental recursion can be disabled using the @@ -1110,23 +1149,28 @@ expand it. 0. `--mkpath` - Create a missing path component of the destination arg. This allows rsync - to create multiple levels of missing destination dirs and to create a path - in which to put a single renamed file. Keep in mind that you'll need to - supply a trailing slash if you want the entire destination path to be - treated as a directory when copying a single arg (making rsync behave the - same way that it would if the path component of the destination had already - existed). + Create all missing path components of the destination path. - For example, the following creates a copy of file foo as bar in the sub/dir - directory, creating dirs "sub" and "sub/dir" if either do not yet exist: + By default, rsync allows only the final component of the destination path + to not exist, which is an attempt to help you to validate your destination + path. With this option, rsync creates all the missing destination-path + components, just as if `mkdir -p $DEST_PATH` had been run on the receiving + side. - > rsync -ai --mkpath foo sub/dir/bar + When specifying a destination path, including a trailing slash ensures that + the whole path is treated as directory names to be created, even when the + file list has a single item. See the [COPYING TO A DIFFERENT NAME](#) + section for full details on how rsync decides if a final destination-path + component should be created as a directory or not. - If you instead ran the following, it would have created file foo in the - sub/dir/bar directory: + If you would like the newly-created destination dirs to match the dirs on + the sending side, you should be using [`--relative`](#opt) (`-R`) instead + of `--mkpath`. For instance, the following two commands result in the same + destination tree, but only the second command ensures that the + "some/extra/path" components match the dirs on the sending side: - > rsync -ai --mkpath foo sub/dir/bar/ + > rsync -ai --mkpath host:some/extra/path/*.c some/extra/path/ + > rsync -aiR host:some/extra/path/*.c ./ 0. `--links`, `-l` @@ -1557,6 +1601,15 @@ expand it. will make the update fairly efficient if the files haven't actually changed, you're much better off using `-t`). + A modern rsync that is using transfer protocol 30 or 31 conveys a modify + time using up to 8-bytes. If rsync is forced to speak an older protocol + (perhaps due to the remote rsync being older than 3.0.0) a modify time is + conveyed using 4-bytes. Prior to 3.2.7, these shorter values could convey + a date range of 13-Dec-1901 to 19-Jan-2038. Beginning with 3.2.7, these + 4-byte values now convey a date range of 1-Jan-1970 to 7-Feb-2106. If you + have files dated older than 1970, make sure your rsync executables are + upgraded so that the full range of dates can be conveyed. + 0. `--atimes`, `-U` This tells rsync to set the access (use) times of the destination files to @@ -1727,6 +1780,7 @@ expand it. - `xxh64` (aka `xxhash`) - `md5` - `md4` + - `sha1` - `none` Run `rsync --version` to see the default checksum list compiled into your @@ -2384,6 +2438,8 @@ expand it. This option tells rsync to stop trying to protect the arg values on the remote side from unintended word-splitting or other misinterpretation. + It also allows the client to treat an empty arg as a "." instead of + generating an error. The default in a modern rsync is for "shell-active" characters (including spaces) to be backslash-escaped in the args that are sent to the remote @@ -3524,13 +3580,17 @@ expand it. > rsync -av --list-only foo* dest/ - Starting with rsync 3.1.0, the sizes output by `--list-only` are affected - by the [`--human-readable`](#opt) option. By default they will contain - digit separators, but higher levels of readability will output the sizes - with unit suffixes. Note also that the column width for the size output - has increased from 11 to 14 characters for all human-readable levels. Use - `--no-h` if you want just digits in the sizes, and the old column width of - 11 characters. + This option always uses an output format that looks similar to this: + + > drwxrwxr-x 4,096 2022/09/30 12:53:11 support + > -rw-rw-r-- 80 2005/01/11 10:37:37 support/Makefile + + The only option that affects this output style is (as of 3.1.0) the + [`--human-readable`](#opt) (`-h`) option. The default is to output sizes + as byte counts with digit separators (in a 14-character-width column). + Specifying at least one `-h` option makes the sizes output with unit + suffixes. If you want old-style bytecount sizes without digit separators + (and an 11-character-width column) use `--no-h`. Compatibility note: when requesting a remote listing of files from an rsync that is version 2.6.3 or older, you may encounter an error if you ask for a diff --git a/rsync.h b/rsync.h index e29c37c3..d3709fe0 100644 --- a/rsync.h +++ b/rsync.h @@ -366,16 +366,10 @@ enum delret { #include #endif -#ifdef TIME_WITH_SYS_TIME -#include -#include -#else #ifdef HAVE_SYS_TIME_H #include -#else +#endif #include -#endif -#endif #ifdef HAVE_FCNTL_H #include @@ -826,6 +820,7 @@ extern int uid_ndx; extern int gid_ndx; extern int acls_ndx; extern int xattrs_ndx; +extern int file_sum_extra_cnt; #ifdef USE_FLEXIBLE_ARRAY #define FILE_STRUCT_LEN (sizeof (struct file_struct)) @@ -836,7 +831,7 @@ extern int xattrs_ndx; #define DEV_EXTRA_CNT 2 #define DIRNODE_EXTRA_CNT 3 #define EXTRA64_CNT ((sizeof (union file_extras64) + EXTRA_LEN - 1) / EXTRA_LEN) -#define SUM_EXTRA_CNT ((MAX_DIGEST_LEN + EXTRA_LEN - 1) / EXTRA_LEN) +#define SUM_EXTRA_CNT file_sum_extra_cnt #define REQ_EXTRA(f,ndx) ((union file_extras*)(f) - (ndx)) #define OPT_EXTRA(f,bump) ((union file_extras*)(f) - file_extra_cnt - 1 - (bump)) @@ -1023,6 +1018,7 @@ typedef struct filter_struct { int slash_cnt; struct filter_list_struct *mergelist; } u; + uchar elide; } filter_rule; typedef struct filter_list_struct { @@ -1162,16 +1158,16 @@ typedef struct { #define NSTR_COMPRESS 1 struct name_num_item { - int num; - const char *name, *main_name; + int num, flags; + const char *name; + struct name_num_item *main_nni; }; struct name_num_obj { const char *type; - const char *negotiated_name; + struct name_num_item *negotiated_nni; uchar *saw; int saw_len; - int negotiated_num; struct name_num_item *list; }; diff --git a/rsyncd.conf.5.md b/rsyncd.conf.5.md index 400ad107..91aaf6f9 100644 --- a/rsyncd.conf.5.md +++ b/rsyncd.conf.5.md @@ -164,6 +164,16 @@ the values of parameters. See the GLOBAL PARAMETERS section for more details. available in this module. You must specify this parameter for each module in `rsyncd.conf`. + If the value contains a "/./" element then the path will be divided at that + point into a chroot dir and an inner-chroot subdir. If [`use chroot`](#) + is set to false, though, the extraneous dot dir is just cleaned out of the + path. An example of this idiom is: + + > path = /var/rsync/./module1 + + This will (when chrooting) chroot to "/var/rsync" and set the inside-chroot + path to "/module1". + You may base the path's value off of an environment variable by surrounding the variable name with percent signs. You can even reference a variable that is set by rsync when the user connects. For example, this would use @@ -187,29 +197,47 @@ the values of parameters. See the GLOBAL PARAMETERS section for more details. path, and of complicating the preservation of users and groups by name (see below). - As an additional safety feature, you can specify a dot-dir in the module's - "[path](#)" to indicate the point where the chroot should occur. This allows - rsync to run in a chroot with a non-"/" path for the top of the transfer - hierarchy. Doing this guards against unintended library loading (since - those absolute paths will not be inside the transfer hierarchy unless you - have used an unwise pathname), and lets you setup libraries for the chroot - that are outside of the transfer. For example, specifying - "/var/rsync/./module1" will chroot to the "/var/rsync" directory and set - the inside-chroot path to "/module1". If you had omitted the dot-dir, the - chroot would have used the whole path, and the inside-chroot path would - have been "/". + If `use chroot` is not set, it defaults to trying to enable a chroot but + allows the daemon to continue (after logging a warning) if it fails. The + one exception to this is when a module's [`path`](#) has a "/./" chroot + divider in it -- this causes an unset value to be treated as true for that + module. - When both "use chroot" and "[daemon chroot](#)" are false, OR the inside-chroot - path of "use chroot" is not "/", rsync will: (1) munge symlinks by default - for security reasons (see "[munge symlinks](#)" for a way to turn this off, but - only if you trust your users), (2) substitute leading slashes in absolute - paths with the module's path (so that options such as `--backup-dir`, - `--compare-dest`, etc. interpret an absolute path as rooted in the module's - "[path](#)" dir), and (3) trim ".." path elements from args if rsync believes - they would escape the module hierarchy. The default for "use chroot" is - true, and is the safer choice (especially if the module is not read-only). + Prior to rsync 3.2.7, the default value was "true". The new "unset" + default makes it easier to setup an rsync daemon as a non-root user or to + run a daemon on a system where chroot fails. Explicitly setting the value + to "true" in rsyncd.conf will always require the chroot to succeed. - When this parameter is enabled *and* the "[name converter](#)" parameter is + It is also possible to specify a dot-dir in the module's "[path](#)" to + indicate that you want to chdir to the earlier part of the path and then + serve files from inside the latter part of the path (with sanitizing and + default symlink munging). This can be useful if you need some library dirs + inside the chroot (typically for uid & gid lookups) but don't want to put + the lib dir into the top of the served path (even though they can be hidden + with an [`exclude`](#) directive). However, a better choice for a modern + rsync setup is to use a [`name converter`](#)" and try to avoid inner lib + dirs altogether. See also the [`daemon chroot`](#) parameter, which causes + rsync to chroot into its own chroot area before doing any path-related + chrooting. + + If the daemon is serving the "/" dir (either directly or due to being + chrooted to the module's path), rsync does not do any path sanitizing or + (default) munging. + + When it has to limit access to a particular subdir (either due to chroot + being disabled or having an inside-chroot path set), rsync will munge + symlinks (by default) and sanitize paths. Those that dislike munged + symlinks (and really, really trust their users to not break out of the + subdir) can disable the symlink munging via the "[munge symlinks](#)" + parameter. + + When rsync is sanitizing paths, it trims ".." path elements from args that + it believes would escape the module hierarchy. It also substitutes leading + slashes in absolute paths with the module's path (so that options such as + `--backup-dir` & `--compare-dest` interpret an absolute path as rooted in + the module's "[path](#)" dir). + + When a chroot is in effect *and* the "[name converter](#)" parameter is *not* set, the "[numeric ids](#)" parameter will default to being enabled (disabling name lookups). This means that if you manually setup name-lookup libraries in your chroot (instead of using a name converter) diff --git a/support/json-rsync-version b/support/json-rsync-version new file mode 100755 index 00000000..31fed7f1 --- /dev/null +++ b/support/json-rsync-version @@ -0,0 +1,93 @@ +#!/usr/bin/python3 + +import sys, argparse, subprocess, json + +TWEAK_NAME = { + 'asm': 'asm_roll', + 'ASM': 'asm_roll', + 'hardlink_special': 'hardlink_specials', + 'protect_args': 'secluded_args', + 'protected_args': 'secluded_args', + 'SIMD': 'SIMD_roll', + } + +MOVE_OPTIM = set('asm_roll SIMD_roll'.split()) + +def main(): + if not args.rsync or args.rsync == '-': + ver_out = sys.stdin.read().strip() + else: + ver_out = subprocess.check_output([args.rsync, '--version', '--version'], encoding='utf-8').strip() + if ver_out.startswith('{'): + print(ver_out) + return + info = { } + misplaced_optims = { } + for line in ver_out.splitlines(): + if line.startswith('rsync '): + prog, vstr, ver, pstr, vstr2, proto = line.split() + info['program'] = prog + if ver.startswith('v'): + ver = ver[1:] + info[vstr] = ver + if '.' not in proto: + proto += '.0' + else: + proto = proto.replace('.PR', '.') + info[pstr] = proto + elif line.startswith('Copyright '): + info['copyright'] = line[10:] + elif line.startswith('Web site: '): + info['url'] = line[10:] + elif line.startswith(' '): + if not saw_comma and ',' in line: + saw_comma = True + info[sect_name] = { } + if saw_comma: + for x in line.strip(' ,').split(', '): + if ' ' in x: + val, var = x.split(' ', 1) + if val == 'no': + val = False + elif val.endswith('-bit'): + var = var[:-1] + '_bits' + val = int(val.split('-')[0]) + else: + var = x + val = True + var = var.replace(' ', '_').replace('-', '_') + if var in TWEAK_NAME: + var = TWEAK_NAME[var] + if sect_name[0] != 'o' and var in MOVE_OPTIM: + misplaced_optims[var] = val + else: + info[sect_name][var] = val + else: + info[sect_name] += [ x for x in line.split() if not x.startswith('(') ] + elif line == '': + break + else: + sect_name = line.strip(' :').replace(' ', '_').lower() + info[sect_name] = [ ] + saw_comma = False + for chk in 'capabilities optimizations'.split(): + if chk not in info: + info[chk] = { } + if misplaced_optims: + info['optimizations'].update(misplaced_optims) + for chk in 'checksum_list compress_list daemon_auth_list'.split(): + if chk not in info: + info[chk] = [ ] + info['license'] = 'GPLv3' if ver[0] == '3' else 'GPLv2' + info['caveat'] = 'rsync comes with ABSOLUTELY NO WARRANTY' + print(json.dumps(info)) + + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description="Output rsync's version data in JSON format, even if the rsync doesn't support a native json-output method.", add_help=False) + parser.add_argument('rsync', nargs='?', help="Specify an rsync command to run. Otherwise stdin is consumed.") + parser.add_argument("--help", "-h", action="help", help="Output this help message and exit.") + args = parser.parse_args() + main() + +# vim: sw=4 et diff --git a/testsuite/acls-default.test b/testsuite/acls-default.test index a0a482ce..d8fba7fe 100644 --- a/testsuite/acls-default.test +++ b/testsuite/acls-default.test @@ -7,7 +7,7 @@ . $suitedir/rsync.fns -$RSYNC --version | grep "[, ] ACLs" >/dev/null || test_skipped "Rsync is configured without ACL support" +$RSYNC -VV | grep '"ACLs": true' >/dev/null || test_skipped "Rsync is configured without ACL support" case "$setfacl_nodef" in true) test_skipped "I don't know how to use your setfacl command" ;; diff --git a/testsuite/acls.test b/testsuite/acls.test index 23449018..693da667 100644 --- a/testsuite/acls.test +++ b/testsuite/acls.test @@ -7,7 +7,7 @@ . $suitedir/rsync.fns -$RSYNC --version | grep "[, ] ACLs" >/dev/null || test_skipped "Rsync is configured without ACL support" +$RSYNC -VV | grep '"ACLs": true' >/dev/null || test_skipped "Rsync is configured without ACL support" makepath "$fromdir/foo" echo something >"$fromdir/file1" diff --git a/testsuite/atimes.test b/testsuite/atimes.test index 3bdb1d46..4d46eb05 100644 --- a/testsuite/atimes.test +++ b/testsuite/atimes.test @@ -4,7 +4,7 @@ . "$suitedir/rsync.fns" -$RSYNC --version | grep "[, ] atimes" >/dev/null || test_skipped "Rsync is configured without atimes support" +$RSYNC -VV | grep '"atimes": true' >/dev/null || test_skipped "Rsync is configured without atimes support" mkdir "$fromdir" diff --git a/testsuite/chown.test b/testsuite/chown.test index 5dadb836..b53413e1 100644 --- a/testsuite/chown.test +++ b/testsuite/chown.test @@ -15,7 +15,7 @@ case $0 in *fake*) - $RSYNC --version | grep "[, ] xattrs" >/dev/null || test_skipped "Rsync needs xattrs for fake device tests" + $RSYNC -VV | grep '"xattrs": true' >/dev/null || test_skipped "Rsync needs xattrs for fake device tests" RSYNC="$RSYNC --fake-super" TLS_ARGS="$TLS_ARGS --fake-super" case "$HOST_OS" in diff --git a/testsuite/crtimes.test b/testsuite/crtimes.test index 07d9269b..456f0a5f 100644 --- a/testsuite/crtimes.test +++ b/testsuite/crtimes.test @@ -4,7 +4,7 @@ . "$suitedir/rsync.fns" -$RSYNC --version | grep "[, ] crtimes" >/dev/null || test_skipped "Rsync is configured without crtimes support" +$RSYNC -VV | grep '"crtimes": true' >/dev/null || test_skipped "Rsync is configured without crtimes support" # Setting an older time via touch sets the create time to the mtime. # Setting it to a newer time affects just the mtime. diff --git a/testsuite/daemon.test b/testsuite/daemon.test index 80d2baf6..60aa334b 100644 --- a/testsuite/daemon.test +++ b/testsuite/daemon.test @@ -81,7 +81,7 @@ drwxr-xr-x DIR ####/##/## ##:##:## foo EOT diff $diffopt "$chkfile" "$outfile" || test_fail "test 3 failed" -if $RSYNC --version | grep "[, ] atimes" >/dev/null; then +if $RSYNC -VV | grep '"atimes": true' >/dev/null; then checkdiff "$RSYNC -rU localhost::test-from/f*" \ "sed -e '$FILE_REPL' -e '$DIR_REPL' -e '$LS_REPL'" </dev/null || test_skipped "Rsync needs xattrs for fake device tests" + $RSYNC -VV | grep '"xattrs": true' >/dev/null || test_skipped "Rsync needs xattrs for fake device tests" RSYNC="$RSYNC --fake-super" TLS_ARGS="$TLS_ARGS --fake-super" case "$HOST_OS" in @@ -94,7 +94,7 @@ esac # TODO: Need to test whether hardlinks are possible on this OS/filesystem -$RSYNC --version | grep "[, ] hardlink-special" >/dev/null && CAN_HLINK_SPECIAL=yes || CAN_HLINK_SPECIAL=no +$RSYNC -VV | grep '"hardlink_specials": true' >/dev/null && CAN_HLINK_SPECIAL=yes || CAN_HLINK_SPECIAL=no mkdir "$fromdir" mkdir "$todir" diff --git a/testsuite/exclude-lsh.test b/testsuite/exclude-lsh.test new file mode 120000 index 00000000..84bc98ac --- /dev/null +++ b/testsuite/exclude-lsh.test @@ -0,0 +1 @@ +exclude.test \ No newline at end of file diff --git a/testsuite/exclude.test b/testsuite/exclude.test index 4b1a1a05..56b68b8c 100644 --- a/testsuite/exclude.test +++ b/testsuite/exclude.test @@ -15,6 +15,19 @@ CVSIGNORE='*.junk' export CVSIGNORE +case $0 in +*-lsh.*) + RSYNC_RSH="$scratchdir/src/support/lsh.sh" + export RSYNC_RSH + rpath=" --rsync-path='$RSYNC'" + host='lh:' + ;; +*) + rpath='' + host='' + ;; +esac + # Build some files/dirs/links to copy makepath "$fromdir/foo/down/to/you" @@ -106,8 +119,8 @@ home-cvs-exclude EOF # Start with a check of --prune-empty-dirs: -$RSYNC -av -f -_foo/too/ -f -_foo/down/ -f -_foo/and/ -f -_new/ "$fromdir/" "$chkdir/" -checkit "$RSYNC -av --prune-empty-dirs '$fromdir/' '$todir/'" "$chkdir" "$todir" +$RSYNC -av --rsync-path="$RSYNC" -f -_foo/too/ -f -_foo/down/ -f -_foo/and/ -f -_new/ "$host$fromdir/" "$chkdir/" +checkit "$RSYNC -av$rpath --prune-empty-dirs '$host$fromdir/' '$todir/'" "$chkdir" "$todir" rm -rf "$todir" # Add a directory symlink. @@ -120,7 +133,7 @@ touch "$scratchdir/up1/same-newness" "$scratchdir/up2/same-newness" touch "$scratchdir/up1/extra-src" "$scratchdir/up2/extra-dest" # Create chkdir with what we expect to be excluded. -checkit "$RSYNC -avv '$fromdir/' '$chkdir/'" "$fromdir" "$chkdir" +checkit "$RSYNC -avv$rpath '$host$fromdir/' '$chkdir/'" "$fromdir" "$chkdir" sleep 1 # Ensures that the rm commands will tweak the directory times. rm -r "$chkdir"/foo/down rm -r "$chkdir"/mid/for/foo/and @@ -135,12 +148,12 @@ touch "$scratchdir/up1/src-newness" "$scratchdir/up2/dst-newness" # Un-tweak the directory times in our first (weak) exclude test (though # it's a good test of the --existing option). -$RSYNC -av --existing --include='*/' --exclude='*' "$fromdir/" "$chkdir/" +$RSYNC -av --rsync-path="$RSYNC" --existing --include='*/' --exclude='*' "$host$fromdir/" "$chkdir/" # Now, test if rsync excludes the same files. -checkit "$RSYNC -avv --exclude-from='$excl' \ - --delete-during '$fromdir/' '$todir/'" "$chkdir" "$todir" +checkit "$RSYNC -avv$rpath --exclude-from='$excl' \ + --delete-during '$host$fromdir/' '$todir/'" "$chkdir" "$todir" # Modify the chk dir by removing cvs-ignored files and then tweaking the dir times. @@ -150,13 +163,15 @@ rm "$chkdir"/bar/down/to/foo/*.junk rm "$chkdir"/bar/down/to/home-cvs-exclude rm "$chkdir"/mid/one-in-one-out -$RSYNC -av --existing --filter='exclude,! */' "$fromdir/" "$chkdir/" +$RSYNC -av --rsync-path="$RSYNC" --existing --filter='exclude,! */' "$host$fromdir/" "$chkdir/" # Now, test if rsync excludes the same files, this time with --cvs-exclude # and --delete-excluded. -checkit "$RSYNC -avvC --filter='merge $excl' --delete-excluded \ - --delete-during '$fromdir/' '$todir/'" "$chkdir" "$todir" +# The -C option gets applied in a different order when pushing & pulling, so we instead +# add the 2 --cvs-exclude filter rules (":C" & "-C") via -f to keep the order the same. +checkit "$RSYNC -avv$rpath --filter='merge $excl' -f:C -f-C --delete-excluded \ + --delete-during '$host$fromdir/' '$todir/'" "$chkdir" "$todir" # Modify the chk dir for our merge-exclude test and then tweak the dir times. @@ -165,19 +180,19 @@ rm "$chkdir"/bar/down/to/bar/baz/*.deep cp_touch "$fromdir"/bar/down/to/foo/*.junk "$chkdir"/bar/down/to/foo cp_touch "$fromdir"/bar/down/to/foo/to "$chkdir"/bar/down/to/foo -$RSYNC -av --existing -f 'show .filt*' -f 'hide,! */' --del "$fromdir/" "$todir/" +$RSYNC -av --rsync-path="$RSYNC" --existing -f 'show .filt*' -f 'hide,! */' --del "$host$fromdir/" "$todir/" echo retained >"$todir"/bar/down/to/bar/baz/nodel.deep cp_touch "$todir"/bar/down/to/bar/baz/nodel.deep "$chkdir"/bar/down/to/bar/baz -$RSYNC -av --existing --filter='-! */' "$fromdir/" "$chkdir/" +$RSYNC -av --rsync-path="$RSYNC" --existing --filter='-! */' "$host$fromdir/" "$chkdir/" # Now, test if rsync excludes the same files, this time with a merge-exclude # file. checkit "sed '/!/d' '$excl' | - $RSYNC -avv -f dir-merge_.filt -f merge_- \ - --delete-during '$fromdir/' '$todir/'" "$chkdir" "$todir" + $RSYNC -avv$rpath -f dir-merge_.filt -f merge_- \ + --delete-during '$host$fromdir/' '$todir/'" "$chkdir" "$todir" # Remove the files that will be deleted. @@ -188,14 +203,14 @@ rm "$chkdir"/bar/down/to/foo/.filt2 rm "$chkdir"/bar/down/to/bar/.filt2 rm "$chkdir"/mid/.filt -$RSYNC -av --protocol=28 --existing --include='*/' --exclude='*' "$fromdir/" "$chkdir/" +$RSYNC -av --rsync-path="$RSYNC" --existing --include='*/' --exclude='*' "$host$fromdir/" "$chkdir/" # Now, try the prior command with --delete-before and some side-specific # rules. checkit "sed '/!/d' '$excl' | - $RSYNC -avv -f :s_.filt -f .s_- -f P_nodel.deep \ - --delete-before '$fromdir/' '$todir/'" "$chkdir" "$todir" + $RSYNC -avv$rpath -f :s_.filt -f .s_- -f P_nodel.deep \ + --delete-before '$host$fromdir/' '$todir/'" "$chkdir" "$todir" # Next, we'll test some rule-restricted filter files. @@ -206,26 +221,26 @@ cat >"$fromdir/bar/down/to/foo/.excl" <f$all_plus extra-src diff --git a/testsuite/itemize.test b/testsuite/itemize.test index 7c29f696..c1c57c59 100644 --- a/testsuite/itemize.test +++ b/testsuite/itemize.test @@ -25,7 +25,7 @@ ln "$fromdir/foo/config1" "$fromdir/foo/extra" rm -f "$to2dir" # Check if rsync is set to hard-link symlinks. -if $RSYNC --version | grep "[, ] hardlink-symlinks" >/dev/null; then +if $RSYNC -VV | grep '"hardlink_symlinks": true' >/dev/null; then L=hL sym_dots="$allspace" L_sym_dots=".L$allspace" @@ -45,7 +45,7 @@ case "$RSYNC" in T=.T ;; *) - if $RSYNC --version | grep "[, ] symtimes" >/dev/null; then + if $RSYNC -VV | grep '"symtimes": true' >/dev/null; then T=.t else T=.T diff --git a/testsuite/xattrs.test b/testsuite/xattrs.test index 455abef1..d94d5f95 100644 --- a/testsuite/xattrs.test +++ b/testsuite/xattrs.test @@ -8,7 +8,7 @@ . $suitedir/rsync.fns lnkdir="$tmpdir/lnk" -$RSYNC --version | grep "[, ] xattrs" >/dev/null || test_skipped "Rsync is configured without xattr support" +$RSYNC -VV | grep '"xattrs": true' >/dev/null || test_skipped "Rsync is configured without xattr support" case "$HOST_OS" in darwin*) diff --git a/usage.c b/usage.c index 048fd4cb..a5b59ad8 100644 --- a/usage.c +++ b/usage.c @@ -22,9 +22,9 @@ #include "latest-year.h" #include "git-version.h" #include "default-cvsignore.h" +#include "itypes.h" -extern struct name_num_obj valid_checksums; -extern struct name_num_obj valid_compressions; +extern struct name_num_obj valid_checksums, valid_compressions, valid_auth_checksums; static char *istring(const char *fmt, int val) { @@ -37,7 +37,8 @@ static char *istring(const char *fmt, int val) static void print_info_flags(enum logcode f) { STRUCT_STAT *dumstat; - char line_buf[75]; + BOOL as_json = f == FNONE ? 1 : 0; /* We use 1 == first attribute, 2 == need closing array */ + char line_buf[75], item_buf[32]; int line_len, j; char *info_flags[] = { @@ -164,46 +165,136 @@ static void print_info_flags(enum logcode f) for (line_len = 0, j = 0; ; j++) { char *str = info_flags[j], *next_nfo = str ? info_flags[j+1] : NULL; - int str_len = str && *str != '*' ? strlen(str) : 1000; int need_comma = next_nfo && *next_nfo != '*' ? 1 : 0; - if (line_len && line_len + 1 + str_len + need_comma >= (int)sizeof line_buf) { - rprintf(f, " %s\n", line_buf); + int item_len; + if (!str || *str == '*') + item_len = 1000; + else if (as_json) { + char *space = strchr(str, ' '); + int is_no = space && strncmp(str, "no ", 3) == 0; + int is_bits = space && isDigit(str); + char *quot = space && !is_no && !is_bits ? "\"" : ""; + char *item = space ? space + 1 : str; + char *val = !space ? "true" : is_no ? "false" : str; + int val_len = !space ? 4 : is_no ? 5 : space - str; + if (is_bits && (space = strchr(val, '-')) != NULL) + val_len = space - str; + item_len = snprintf(item_buf, sizeof item_buf, + " \"%s%s\": %s%.*s%s%s", item, is_bits ? "bits" : "", + quot, val_len, val, quot, need_comma ? "," : ""); + if (is_bits) + item_buf[strlen(item)+2-1] = '_'; /* Turn the 's' into a '_' */ + for (space = item; (space = strpbrk(space, " -")) != NULL; space++) + item_buf[space - item + 2] = '_'; + } else + item_len = snprintf(item_buf, sizeof item_buf, " %s%s", str, need_comma ? "," : ""); + if (line_len && line_len + item_len >= (int)sizeof line_buf) { + if (as_json) + printf(" %s\n", line_buf); + else + rprintf(f, " %s\n", line_buf); line_len = 0; } if (!str) break; if (*str == '*') { - rprintf(f, "%s:\n", str+1); - continue; + if (as_json) { + if (as_json == 2) + printf(" }"); + else + as_json = 2; + printf(",\n \"%c%s\": {\n", toLower(str+1), str+2); + } else + rprintf(f, "%s:\n", str+1); + } else { + strlcpy(line_buf + line_len, item_buf, sizeof line_buf - line_len); + line_len += item_len; } - line_len += snprintf(line_buf+line_len, sizeof line_buf - line_len, " %s%s", str, need_comma ? "," : ""); } + if (as_json == 2) + printf(" }"); } +static void output_nno_list(enum logcode f, const char *name, struct name_num_obj *nno) +{ + char namebuf[64], tmpbuf[256]; + char *tok, *next_tok, *comma = ","; + char *cp; + + /* Using '(' ensures that we get a trailing "none" but also includes aliases. */ + get_default_nno_list(nno, tmpbuf, sizeof tmpbuf - 1, '('); + if (f != FNONE) { + rprintf(f, "%s:\n", name); + rprintf(f, " %s\n", tmpbuf); + return; + } + + strlcpy(namebuf, name, sizeof namebuf); + for (cp = namebuf; *cp; cp++) { + if (*cp == ' ') + *cp = '_'; + else if (isUpper(cp)) + *cp = toLower(cp); + } + + printf(",\n \"%s\": [\n ", namebuf); + + for (tok = strtok(tmpbuf, " "); tok; tok = next_tok) { + next_tok = strtok(NULL, " "); + if (*tok != '(') /* Ignore the alises in the JSON output */ + printf(" \"%s\"%s", tok, comma + (next_tok ? 0 : 1)); + } + + printf("\n ]"); +} + +/* A request of f == FNONE wants json on stdout. */ void print_rsync_version(enum logcode f) { - char tmpbuf[256], *subprotocol = ""; + char copyright[] = "(C) 1996-" LATEST_YEAR " by Andrew Tridgell, Wayne Davison, and others."; + char url[] = "https://rsync.samba.org/"; + BOOL first_line = 1; +#define json_line(name, value) \ + do { \ + printf("%c\n \"%s\": \"%s\"", first_line ? '{' : ',', name, value); \ + first_line = 0; \ + } while (0) + + if (f == FNONE) { + char verbuf[32]; + json_line("program", RSYNC_NAME); + json_line("version", rsync_version()); + (void)snprintf(verbuf, sizeof verbuf, "%d.%d", PROTOCOL_VERSION, SUBPROTOCOL_VERSION); + json_line("protocol", verbuf); + json_line("copyright", copyright); + json_line("url", url); + } else { #if SUBPROTOCOL_VERSION != 0 - subprotocol = istring(".PR%d", SUBPROTOCOL_VERSION); + char *subprotocol = istring(".PR%d", SUBPROTOCOL_VERSION); +#else + char *subprotocol = ""; #endif - rprintf(f, "%s version %s protocol version %d%s\n", - RSYNC_NAME, rsync_version(), PROTOCOL_VERSION, subprotocol); - - rprintf(f, "Copyright (C) 1996-" LATEST_YEAR " by Andrew Tridgell, Wayne Davison, and others.\n"); - rprintf(f, "Web site: https://rsync.samba.org/\n"); + rprintf(f, "%s version %s protocol version %d%s\n", + RSYNC_NAME, rsync_version(), PROTOCOL_VERSION, subprotocol); + rprintf(f, "Copyright %s\n", copyright); + rprintf(f, "Web site: %s\n", url); + } print_info_flags(f); init_checksum_choices(); - rprintf(f, "Checksum list:\n"); - get_default_nno_list(&valid_checksums, tmpbuf, sizeof tmpbuf, '('); - rprintf(f, " %s\n", tmpbuf); + output_nno_list(f, "Checksum list", &valid_checksums); + output_nno_list(f, "Compress list", &valid_compressions); + output_nno_list(f, "Daemon auth list", &valid_auth_checksums); - rprintf(f, "Compress list:\n"); - get_default_nno_list(&valid_compressions, tmpbuf, sizeof tmpbuf, '('); - rprintf(f, " %s\n", tmpbuf); + if (f == FNONE) { + json_line("license", "GPLv3"); + json_line("caveat", "rsync comes with ABSOLUTELY NO WARRANTY"); + printf("\n}\n"); + return; + } #ifdef MAINTAINER_MODE rprintf(f, "Panic Action: \"%s\"\n", get_panic_action()); @@ -265,11 +356,13 @@ void daemon_usage(enum logcode F) const char *rsync_version(void) { + char *ver; #ifdef RSYNC_GITVER - return RSYNC_GITVER; + ver = RSYNC_GITVER; #else - return RSYNC_VERSION; + ver = RSYNC_VERSION; #endif + return *ver == 'v' ? ver+1 : ver; } const char *default_cvsignore(void) diff --git a/util1.c b/util1.c index 671f3c75..da50ff1e 100644 --- a/util1.c +++ b/util1.c @@ -1487,12 +1487,19 @@ const char *find_filename_suffix(const char *fn, int fn_len, int *len_ptr) #define UNIT (1 << 16) -uint32 fuzzy_distance(const char *s1, unsigned len1, const char *s2, unsigned len2) +uint32 fuzzy_distance(const char *s1, unsigned len1, const char *s2, unsigned len2, uint32 upperlimit) { uint32 a[MAXPATHLEN], diag, above, left, diag_inc, above_inc, left_inc; int32 cost; unsigned i1, i2; + /* Check to see if the Levenshtein distance must be greater than the + * upper limit defined by the previously found lowest distance using + * the heuristic that the Levenshtein distance is greater than the + * difference in length of the two strings */ + if ((len1 > len2 ? len1 - len2 : len2 - len1) * UNIT > upperlimit) + return 0xFFFFU * UNIT + 1; + if (!len1 || !len2) { if (!len1) { s1 = s2; diff --git a/version.h b/version.h index 04e2db7a..fdfce4c4 100644 --- a/version.h +++ b/version.h @@ -1,2 +1,2 @@ -#define RSYNC_VERSION "3.2.6" +#define RSYNC_VERSION "3.2.7" #define MAINTAINER_TZ_OFFSET -7.0 diff --git a/xattrs.c b/xattrs.c index 1f2bfacd..26e50a6f 100644 --- a/xattrs.c +++ b/xattrs.c @@ -39,9 +39,13 @@ extern int preserve_specials; extern int checksum_seed; extern int saw_xattr_filter; +extern struct name_num_item *xattr_sum_nni; +extern int xattr_sum_len; + #define RSYNC_XAL_INITIAL 5 #define RSYNC_XAL_LIST_INITIAL 100 +#define MAX_XATTR_DIGEST_LEN MD5_DIGEST_LEN #define MAX_FULL_DATUM 32 #define HAS_PREFIX(str, prfx) (*(str) == *(prfx) && strncmp(str, prfx, sizeof (prfx) - 1) == 0) @@ -269,8 +273,8 @@ static int rsync_xal_get(const char *fname, item_list *xalp) if (datum_len > MAX_FULL_DATUM) { /* For large datums, we store a flag and a checksum. */ - name_offset = 1 + MAX_DIGEST_LEN; - sum_init(-1, checksum_seed); + name_offset = 1 + MAX_XATTR_DIGEST_LEN; + sum_init(xattr_sum_nni, checksum_seed); sum_update(ptr, datum_len); free(ptr); @@ -377,20 +381,14 @@ static int64 xattr_lookup_hash(const item_list *xalp) { const rsync_xa *rxas = xalp->items; size_t i; - int64 key = hashlittle(&xalp->count, sizeof xalp->count); + int64 key = hashlittle2(&xalp->count, sizeof xalp->count); for (i = 0; i < xalp->count; i++) { - key += hashlittle(rxas[i].name, rxas[i].name_len); + key += hashlittle2(rxas[i].name, rxas[i].name_len); if (rxas[i].datum_len > MAX_FULL_DATUM) - key += hashlittle(rxas[i].datum, MAX_DIGEST_LEN); + key += hashlittle2(rxas[i].datum, xattr_sum_len); else - key += hashlittle(rxas[i].datum, rxas[i].datum_len); - } - - if (key == 0) { - /* This is very unlikely, but we should never - * return 0 as hashtable_find() doesn't like it. */ - return 1; + key += hashlittle2(rxas[i].datum, rxas[i].datum_len); } return key; @@ -435,7 +433,7 @@ static int find_matching_xattr(const item_list *xalp) if (rxas1[j].datum_len > MAX_FULL_DATUM) { if (memcmp(rxas1[j].datum + 1, rxas2[j].datum + 1, - MAX_DIGEST_LEN) != 0) + xattr_sum_len) != 0) break; } else { if (memcmp(rxas1[j].datum, rxas2[j].datum, @@ -471,8 +469,6 @@ static int rsync_xal_store(item_list *xalp) if (rsync_xal_h == NULL) rsync_xal_h = hashtable_create(512, HT_KEY64); - if (rsync_xal_h == NULL) - out_of_memory("rsync_xal_h hashtable_create()"); new_ref = new0(rsync_xa_list_ref); new_ref->ndx = ndx; @@ -535,7 +531,7 @@ int send_xattr(int f, stat_x *sxp) #endif write_buf(f, name, name_len); if (rxa->datum_len > MAX_FULL_DATUM) - write_buf(f, rxa->datum + 1, MAX_DIGEST_LEN); + write_buf(f, rxa->datum + 1, xattr_sum_len); else write_bigbuf(f, rxa->datum, rxa->datum_len); } @@ -588,7 +584,7 @@ int xattr_diff(struct file_struct *file, stat_x *sxp, int find_all) else if (snd_rxa->datum_len > MAX_FULL_DATUM) { same = cmp == 0 && snd_rxa->datum_len == rec_rxa->datum_len && memcmp(snd_rxa->datum + 1, rec_rxa->datum + 1, - MAX_DIGEST_LEN) == 0; + xattr_sum_len) == 0; /* Flag unrequested items that we need. */ if (!same && find_all && snd_rxa->datum[0] == XSTATE_ABBREV) snd_rxa->datum[0] = XSTATE_TODO; @@ -797,7 +793,7 @@ void receive_xattr(int f, struct file_struct *file) rsync_xa *rxa; size_t name_len = read_varint(f); size_t datum_len = read_varint(f); - size_t dget_len = datum_len > MAX_FULL_DATUM ? 1 + MAX_DIGEST_LEN : datum_len; + size_t dget_len = datum_len > MAX_FULL_DATUM ? 1 + (size_t)xattr_sum_len : datum_len; size_t extra_len = MIGHT_NEED_RPRE ? RPRE_LEN : 0; if (SIZE_MAX - dget_len < extra_len || SIZE_MAX - dget_len - extra_len < name_len) overflow_exit("receive_xattr"); @@ -812,7 +808,7 @@ void receive_xattr(int f, struct file_struct *file) read_buf(f, ptr, dget_len); else { *ptr = XSTATE_ABBREV; - read_buf(f, ptr + 1, MAX_DIGEST_LEN); + read_buf(f, ptr + 1, xattr_sum_len); } if (saw_xattr_filter) { @@ -943,7 +939,7 @@ static int rsync_xal_set(const char *fname, item_list *xalp, rsync_xa *rxas = xalp->items; ssize_t list_len; size_t i, len; - char *name, *ptr, sum[MAX_DIGEST_LEN]; + char *name, *ptr, sum[MAX_XATTR_DIGEST_LEN]; #ifdef HAVE_LINUX_XATTRS int user_only = am_root <= 0; #endif @@ -958,7 +954,6 @@ static int rsync_xal_set(const char *fname, item_list *xalp, name = rxas[i].name; if (XATTR_ABBREV(rxas[i])) { - int sum_len; /* See if the fnamecmp version is identical. */ len = name_len = rxas[i].name_len; if ((ptr = get_xattr_data(fnamecmp, name, &len, 1)) == NULL) { @@ -975,10 +970,10 @@ static int rsync_xal_set(const char *fname, item_list *xalp, goto still_abbrev; } - sum_init(-1, checksum_seed); + sum_init(xattr_sum_nni, checksum_seed); sum_update(ptr, len); - sum_len = sum_end(sum); - if (memcmp(sum, rxas[i].datum + 1, sum_len) != 0) { + sum_end(sum); + if (memcmp(sum, rxas[i].datum + 1, xattr_sum_len) != 0) { free(ptr); goto still_abbrev; } diff -upN a/aclocal.m4 b/aclocal.m4 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -1,6 +1,6 @@ -# generated automatically by aclocal 1.16.1 -*- Autoconf -*- +# generated automatically by aclocal 1.16.5 -*- Autoconf -*- -# Copyright (C) 1996-2018 Free Software Foundation, Inc. +# Copyright (C) 1996-2021 Free Software Foundation, Inc. # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -12,6 +12,6 @@ # PARTICULAR PURPOSE. m4_ifndef([AC_CONFIG_MACRO_DIRS], [m4_defun([_AM_CONFIG_MACRO_DIRS], [])m4_defun([AC_CONFIG_MACRO_DIRS], [_AM_CONFIG_MACRO_DIRS($@)])]) -m4_include([m4/have_type.m4]) -m4_include([m4/header_major_fixed.m4]) -m4_include([m4/socklen_t.m4]) +m4_include([../m4/have_type.m4]) +m4_include([../m4/header_major_fixed.m4]) +m4_include([../m4/socklen_t.m4]) diff -upN a/config.h.in b/config.h.in --- a/config.h.in +++ b/config.h.in @@ -787,10 +787,6 @@ */ #undef SUPPORT_ZSTD -/* Define to 1 if you can safely include both and . This - macro is obsolete. */ -#undef TIME_WITH_SYS_TIME - /* Define to 1 if you want rsync to make use of iconv_open() */ #undef USE_ICONV_OPEN diff -upN a/configure.sh b/configure.sh --- a/configure.sh +++ b/configure.sh @@ -744,6 +744,7 @@ enable_debug enable_profile enable_md2man enable_maintainer_mode +with_openssl_conf with_rrsync with_included_popt with_included_zlib @@ -1427,6 +1428,8 @@ Optional Features: Optional Packages: --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) + --with-openssl-conf=PATH + set default OPENSSL_CONF path for rsync --with-rrsync also install the rrsync script and its manpage --with-included-popt use bundled popt library, not from system --with-included-zlib use bundled zlib library, not from system @@ -2794,7 +2797,6 @@ as_fn_append ac_header_c_list " strings. as_fn_append ac_header_c_list " sys/stat.h sys_stat_h HAVE_SYS_STAT_H" as_fn_append ac_header_c_list " sys/types.h sys_types_h HAVE_SYS_TYPES_H" as_fn_append ac_header_c_list " unistd.h unistd_h HAVE_UNISTD_H" -as_fn_append ac_header_c_list " sys/time.h sys_time_h HAVE_SYS_TIME_H" # Test code for whether the C++ compiler supports C++98 (global declarations) ac_cxx_conftest_cxx98_globals=' // Does the compiler advertise C++98 conformance? @@ -4605,16 +4607,6 @@ fi fi - - -# Obsolete code to be removed. -if test $ac_cv_header_sys_time_h = yes; then - -printf "%s\n" "#define TIME_WITH_SYS_TIME 1" >>confdefs.h - -fi -# End of obsolete code. - { printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for sys/wait.h that is POSIX.1 compatible" >&5 printf %s "checking for sys/wait.h that is POSIX.1 compatible... " >&6; } if test ${ac_cv_header_sys_wait_h+y} @@ -6727,7 +6719,6 @@ fi printf "%s\n" "$MKDIR_P" >&6; } - # Extract the first word of "perl", so it can be a program name with args. set dummy perl; ac_word=$2 { printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 @@ -6912,6 +6903,21 @@ if test x"$GCC" = x"yes"; then fi +# Check whether --with-openssl-conf was given. +if test ${with_openssl_conf+y} +then : + withval=$with_openssl_conf; +fi + +case "$with_openssl_conf" in + *^-/a-zA-Z0-9.,=@+_*) as_fn_error $? "Invalid path given to --with-openssl-conf" "$LINENO" 5 ;; + /*) CFLAGS="$CFLAGS -DSET_OPENSSL_CONF=$with_openssl_conf" ;; + no|'') ;; + yes) as_fn_error $? "No path given to --with-openssl-conf" "$LINENO" 5 ;; + *) as_fn_error $? "Non absolute path given to --with-openssl-conf" "$LINENO" 5 ;; +esac + + # Check whether --with-rrsync was given. if test ${with_rrsync+y} then : @@ -9579,84 +9585,6 @@ esac fi -# Autoupdate added the next two lines to ensure that your configure -# script's behavior did not change. They are probably safe to remove. - -{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for egrep" >&5 -printf %s "checking for egrep... " >&6; } -if test ${ac_cv_path_EGREP+y} -then : - printf %s "(cached) " >&6 -else $as_nop - if echo a | $GREP -E '(a|b)' >/dev/null 2>&1 - then ac_cv_path_EGREP="$GREP -E" - else - if test -z "$EGREP"; then - ac_path_EGREP_found=false - # Loop through the user's path and test for each of PROGNAME-LIST - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH$PATH_SEPARATOR/usr/xpg4/bin -do - IFS=$as_save_IFS - case $as_dir in #((( - '') as_dir=./ ;; - */) ;; - *) as_dir=$as_dir/ ;; - esac - for ac_prog in egrep - do - for ac_exec_ext in '' $ac_executable_extensions; do - ac_path_EGREP="$as_dir$ac_prog$ac_exec_ext" - as_fn_executable_p "$ac_path_EGREP" || continue -# Check for GNU ac_path_EGREP and select it if it is found. - # Check for GNU $ac_path_EGREP -case `"$ac_path_EGREP" --version 2>&1` in -*GNU*) - ac_cv_path_EGREP="$ac_path_EGREP" ac_path_EGREP_found=:;; -*) - ac_count=0 - printf %s 0123456789 >"conftest.in" - while : - do - cat "conftest.in" "conftest.in" >"conftest.tmp" - mv "conftest.tmp" "conftest.in" - cp "conftest.in" "conftest.nl" - printf "%s\n" 'EGREP' >> "conftest.nl" - "$ac_path_EGREP" 'EGREP$' < "conftest.nl" >"conftest.out" 2>/dev/null || break - diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break - as_fn_arith $ac_count + 1 && ac_count=$as_val - if test $ac_count -gt ${ac_path_EGREP_max-0}; then - # Best one so far, save it but keep looking for a better one - ac_cv_path_EGREP="$ac_path_EGREP" - ac_path_EGREP_max=$ac_count - fi - # 10*(2^10) chars as input seems more than enough - test $ac_count -gt 10 && break - done - rm -f conftest.in conftest.tmp conftest.nl conftest.out;; -esac - - $ac_path_EGREP_found && break 3 - done - done - done -IFS=$as_save_IFS - if test -z "$ac_cv_path_EGREP"; then - as_fn_error $? "no acceptable egrep could be found in $PATH$PATH_SEPARATOR/usr/xpg4/bin" "$LINENO" 5 - fi -else - ac_cv_path_EGREP=$EGREP -fi - - fi -fi -{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_EGREP" >&5 -printf "%s\n" "$ac_cv_path_EGREP" >&6; } - EGREP="$ac_cv_path_EGREP" - - - - cv=`echo "struct addrinfo" | sed 'y%./+- %__p__%'` { printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for struct addrinfo" >&5 @@ -9708,7 +9636,6 @@ printf "%s\n" "#define $ac_tr_hdr 1" >>c fi - cv=`echo "struct sockaddr_storage" | sed 'y%./+- %__p__%'` { printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for struct sockaddr_storage" >&5 printf %s "checking for struct sockaddr_storage... " >&6; } @@ -9941,7 +9868,6 @@ fi - cv=`echo "struct stat64" | sed 'y%./+- %__p__%'` { printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for struct stat64" >&5 printf %s "checking for struct stat64... " >&6; } diff -upN a/rrsync.1 b/rrsync.1 --- a/rrsync.1 +++ b/rrsync.1 @@ -1,4 +1,4 @@ -.TH "rrsync" "1" "9 Sep 2022" "rrsync from rsync 3.2.6" "User Commands" +.TH "rrsync" "1" "20 Oct 2022" "rrsync from rsync 3.2.7" "User Commands" .\" prefix=/usr .P .SH "NAME" @@ -150,7 +150,7 @@ command="rrsync -ro results" ssh-rsa AAA .P .SH "VERSION" .P -This manpage is current for version 3.2.6 of rsync. +This manpage is current for version 3.2.7 of rsync. .P .SH "CREDITS" .P diff -upN a/rrsync.1.html b/rrsync.1.html --- a/rrsync.1.html +++ b/rrsync.1.html @@ -151,7 +151,7 @@ command="rrsync -ro results" s

SEE ALSO

rsync(1), rsyncd.conf(5)

VERSION

-

This manpage is current for version 3.2.6 of rsync.

+

This manpage is current for version 3.2.7 of rsync.

CREDITS

rsync is distributed under the GNU General Public License. See the file COPYING for details.

@@ -160,5 +160,5 @@ project is AUTHOR

The original rrsync perl script was written by Joe Smith. Many people have later contributed to it. The python version was created by Wayne Davison.

-

9 Sep 2022

+

20 Oct 2022

diff -upN a/rsync-ssl.1 b/rsync-ssl.1 --- a/rsync-ssl.1 +++ b/rsync-ssl.1 @@ -1,4 +1,4 @@ -.TH "rsync-ssl" "1" "9 Sep 2022" "rsync-ssl from rsync 3.2.6" "User Commands" +.TH "rsync-ssl" "1" "20 Oct 2022" "rsync-ssl from rsync 3.2.7" "User Commands" .\" prefix=/usr .P .SH "NAME" @@ -126,7 +126,7 @@ Please report bugs! See the web site at .P .SH "VERSION" .P -This manpage is current for version 3.2.6 of rsync. +This manpage is current for version 3.2.7 of rsync. .P .SH "CREDITS" .P diff -upN a/rsync-ssl.1.html b/rsync-ssl.1.html --- a/rsync-ssl.1.html +++ b/rsync-ssl.1.html @@ -140,7 +140,7 @@ exported RSYNC_SSL_TYPE environment vari

BUGS

Please report bugs! See the web site at https://rsync.samba.org/.

VERSION

-

This manpage is current for version 3.2.6 of rsync.

+

This manpage is current for version 3.2.7 of rsync.

CREDITS

Rsync is distributed under the GNU General Public License. See the file COPYING for details.

@@ -150,5 +150,5 @@ FAQ-O-Matic which may cover questions un

This manpage was written by Wayne Davison.

Mailing lists for support and development are available at https://lists.samba.org/.

-

9 Sep 2022

+

20 Oct 2022

diff -upN a/rsync.1 b/rsync.1 --- a/rsync.1 +++ b/rsync.1 @@ -1,4 +1,4 @@ -.TH "rsync" "1" "9 Sep 2022" "rsync 3.2.6" "User Commands" +.TH "rsync" "1" "20 Oct 2022" "rsync 3.2.7" "User Commands" .\" prefix=/usr .P .SH "NAME" @@ -186,7 +186,62 @@ rsync somehost.mydomain.com:: .fi .RE .P -See the following section for more details. +.SH "COPYING TO A DIFFERENT NAME" +.P +When you want to copy a directory to a different name, use a trailing slash on +the source directory to put the contents of the directory into any destination +directory you like: +.RS 4 +.P +.nf +rsync -ai foo/ bar/ +.fi +.RE +.P +Rsync also has the ability to customize a destination file's name when copying +a single item. The rules for this are: +.P +.IP o +The transfer list must consist of a single item (either a file or an empty +directory) +.IP o +The final element of the destination path must not exist as a directory +.IP o +The destination path must not have been specified with a trailing slash +.P +Under those circumstances, rsync will set the name of the destination's single +item to the last element of the destination path. Keep in mind that it is best +to only use this idiom when copying a file and use the above trailing-slash +idiom when copying a directory. +.P +The following example copies the \fBfoo.c\fP file as \fBbar.c\fP in the \fBsave\fP dir +(assuming that \fBbar.c\fP isn't a directory): +.RS 4 +.P +.nf +rsync -ai src/foo.c save/bar.c +.fi +.RE +.P +The single-item copy rule might accidentally bite you if you unknowingly copy a +single item and specify a destination dir that doesn't exist (without using a +trailing slash). For example, if \fBsrc/*.c\fP matches one file and \fBsave/dir\fP +doesn't exist, this will confuse you by naming the destination file \fBsave/dir\fP: +.RS 4 +.P +.nf +rsync -ai src/*.c save/dir +.fi +.RE +.P +To prevent such an accident, either make sure the destination dir exists or +specify the destination path with a trailing slash: +.RS 4 +.P +.nf +rsync -ai src/*.c save/dir/ +.fi +.RE .P .SH "SORTED TRANSFER ORDER" .P @@ -478,7 +533,7 @@ has its own detailed description later i --append-verify --append w/old data in file checksum --dirs, -d transfer directories without recursing --old-dirs, --old-d works like --dirs when talking to old rsync ---mkpath create the destination's path component +--mkpath create destination's missing path components --links, -l copy symlinks as symlinks --copy-links, -L transform symlink into referent file/dir --copy-unsafe-links only "unsafe" symlinks are transformed @@ -648,11 +703,14 @@ Print a short help page describing the o You can also use \fB\-h\fP for \fB\-\-help\fP when it is used without any other options (since it normally means \fB\-\-human-readable\fP). .IP "\fB\-\-version\fP, \fB\-V\fP" -Print the rsync version plus other info and exit. -.IP -The output includes the default list of checksum algorithms, the default -list of compression algorithms, a list of compiled-in capabilities, a link -to the rsync web site, and some license/copyright info. +Print the rsync version plus other info and exit. When repeated, the +information is output is a JSON format that is still fairly readable +(client side only). +.IP +The output includes a list of compiled-in capabilities, a list of +optimizations, the default list of checksum algorithms, the default list of +compression algorithms, the default list of daemon auth digests, a link to +the rsync web site, and a few other items. .IP "\fB\-\-verbose\fP, \fB\-v\fP" This option increases the amount of information you are given during the transfer. By default, rsync works silently. A single \fB\-v\fP will give you @@ -922,7 +980,7 @@ modify time of the finished directory ri that until a bunch of recursive copying has finished). However, these early directories don't yet have their completed mode, mtime, or ownership set\ \-\- they have more restrictive rights until the subdirectory's copying -actually begins. This early-creation idiom can be avoiding by using the +actually begins. This early-creation idiom can be avoided by using the \fB\-\-omit-dir-times\fP option. .IP Incremental recursion can be disabled using the @@ -1183,29 +1241,30 @@ There is also a backward-compatibility h (\fB\-\-old-d\fP) that tells rsync to use a hack of \fB\-r\ \-\-exclude='/*/*'\fP to get an older rsync to list a single directory without recursing. .IP "\fB\-\-mkpath\fP" -Create a missing path component of the destination arg. This allows rsync -to create multiple levels of missing destination dirs and to create a path -in which to put a single renamed file. Keep in mind that you'll need to -supply a trailing slash if you want the entire destination path to be -treated as a directory when copying a single arg (making rsync behave the -same way that it would if the path component of the destination had already -existed). +Create all missing path components of the destination path. .IP -For example, the following creates a copy of file foo as bar in the sub/dir -directory, creating dirs "sub" and "sub/dir" if either do not yet exist: -.RS 4 -.IP -.nf -rsync -ai --mkpath foo sub/dir/bar -.fi -.RE +By default, rsync allows only the final component of the destination path +to not exist, which is an attempt to help you to validate your destination +path. With this option, rsync creates all the missing destination-path +components, just as if \fBmkdir\ \-p\ $DEST_PATH\fP had been run on the receiving +side. .IP -If you instead ran the following, it would have created file foo in the -sub/dir/bar directory: +When specifying a destination path, including a trailing slash ensures that +the whole path is treated as directory names to be created, even when the +file list has a single item. See the COPYING TO A DIFFERENT NAME +section for full details on how rsync decides if a final destination-path +component should be created as a directory or not. +.IP +If you would like the newly-created destination dirs to match the dirs on +the sending side, you should be using \fB\-\-relative\fP (\fB\-R\fP) instead +of \fB\-\-mkpath\fP. For instance, the following two commands result in the same +destination tree, but only the second command ensures that the +"some/extra/path" components match the dirs on the sending side: .RS 4 .IP .nf -rsync -ai --mkpath foo sub/dir/bar/ +rsync -ai --mkpath host:some/extra/path/*.c some/extra/path/ +rsync -aiR host:some/extra/path/*.c ./ .fi .RE .IP "\fB\-\-links\fP, \fB\-l\fP" @@ -1639,6 +1698,15 @@ next transfer to behave as if it used \f causing all files to be updated (though rsync's delta-transfer algorithm will make the update fairly efficient if the files haven't actually changed, you're much better off using \fB\-t\fP). +.IP +A modern rsync that is using transfer protocol 30 or 31 conveys a modify +time using up to 8-bytes. If rsync is forced to speak an older protocol +(perhaps due to the remote rsync being older than 3.0.0) a modify time is +conveyed using 4-bytes. Prior to 3.2.7, these shorter values could convey +a date range of 13-Dec-1901 to 19-Jan-2038. Beginning with 3.2.7, these +4-byte values now convey a date range of 1-Jan-1970 to 7-Feb-2106. If you +have files dated older than 1970, make sure your rsync executables are +upgraded so that the full range of dates can be conveyed. .IP "\fB\-\-atimes\fP, \fB\-U\fP" This tells rsync to set the access (use) times of the destination files to the same value as the source files. @@ -1796,6 +1864,8 @@ The checksum options that you may be abl .IP o \fBmd4\fP .IP o +\fBsha1\fP +.IP o \fBnone\fP .RE .IP @@ -2427,6 +2497,8 @@ all names read from a .cvsignore file ar .IP "\fB\-\-old-args\fP" This option tells rsync to stop trying to protect the arg values on the remote side from unintended word-splitting or other misinterpretation. +It also allows the client to treat an empty arg as a "." instead of +generating an error. .IP The default in a modern rsync is for "shell-active" characters (including spaces) to be backslash-escaped in the args that are sent to the remote @@ -3628,13 +3700,21 @@ rsync -av --list-only foo* dest/ .fi .RE .IP -Starting with rsync 3.1.0, the sizes output by \fB\-\-list-only\fP are affected -by the \fB\-\-human-readable\fP option. By default they will contain -digit separators, but higher levels of readability will output the sizes -with unit suffixes. Note also that the column width for the size output -has increased from 11 to 14 characters for all human-readable levels. Use -\fB\-\-no-h\fP if you want just digits in the sizes, and the old column width of -11 characters. +This option always uses an output format that looks similar to this: +.RS 4 +.IP +.nf +drwxrwxr-x 4,096 2022/09/30 12:53:11 support +-rw-rw-r-- 80 2005/01/11 10:37:37 support/Makefile +.fi +.RE +.IP +The only option that affects this output style is (as of 3.1.0) the +\fB\-\-human-readable\fP (\fB\-h\fP) option. The default is to output sizes +as byte counts with digit separators (in a 14-character-width column). +Specifying at least one \fB\-h\fP option makes the sizes output with unit +suffixes. If you want old-style bytecount sizes without digit separators +(and an 11-character-width column) use \fB\-\-no-h\fP. .IP Compatibility note: when requesting a remote listing of files from an rsync that is version 2.6.3 or older, you may encounter an error if you ask for a @@ -4924,7 +5004,7 @@ Please report bugs! See the web site at .P .SH "VERSION" .P -This manpage is current for version 3.2.6 of rsync. +This manpage is current for version 3.2.7 of rsync. .P .SH "INTERNAL OPTIONS" .P diff -upN a/rsync.1.html b/rsync.1.html --- a/rsync.1.html +++ b/rsync.1.html @@ -167,7 +167,46 @@ rsync daemon by leaving off the module n 
rsync somehost.mydomain.com::
-

See the following section for more details.

+

COPYING TO A DIFFERENT NAME

+

When you want to copy a directory to a different name, use a trailing slash on +the source directory to put the contents of the directory into any destination +directory you like:

+
+
rsync -ai foo/ bar/
+
+
+

Rsync also has the ability to customize a destination file's name when copying +a single item. The rules for this are:

+
    +
  • The transfer list must consist of a single item (either a file or an empty +directory)
    • +
  • The final element of the destination path must not exist as a directory
    • +
  • The destination path must not have been specified with a trailing slash
    • +
+

Under those circumstances, rsync will set the name of the destination's single +item to the last element of the destination path. Keep in mind that it is best +to only use this idiom when copying a file and use the above trailing-slash +idiom when copying a directory.

+

The following example copies the foo.c file as bar.c in the save dir +(assuming that bar.c isn't a directory):

+
+
rsync -ai src/foo.c save/bar.c
+
+
+

The single-item copy rule might accidentally bite you if you unknowingly copy a +single item and specify a destination dir that doesn't exist (without using a +trailing slash). For example, if src/*.c matches one file and save/dir +doesn't exist, this will confuse you by naming the destination file save/dir:

+
+
rsync -ai src/*.c save/dir
+
+
+

To prevent such an accident, either make sure the destination dir exists or +specify the destination path with a trailing slash:

+
+
rsync -ai src/*.c save/dir/
+
+

SORTED TRANSFER ORDER

Rsync always sorts the specified filenames into its internal transfer list. This handles the merging together of the contents of identically named @@ -389,7 +428,7 @@ has its own detailed description later i --append-verify --append w/old data in file checksum --dirs, -d transfer directories without recursing --old-dirs, --old-d works like --dirs when talking to old rsync ---mkpath create the destination's path component +--mkpath create destination's missing path components --links, -l copy symlinks as symlinks --copy-links, -L transform symlink into referent file/dir --copy-unsafe-links only "unsafe" symlinks are transformed @@ -555,10 +594,13 @@ options (since it normally means

--version, -V
-

Print the rsync version plus other info and exit.

-

The output includes the default list of checksum algorithms, the default -list of compression algorithms, a list of compiled-in capabilities, a link -to the rsync web site, and some license/copyright info.

+

Print the rsync version plus other info and exit. When repeated, the +information is output is a JSON format that is still fairly readable +(client side only).

+

The output includes a list of compiled-in capabilities, a list of +optimizations, the default list of checksum algorithms, the default list of +compression algorithms, the default list of daemon auth digests, a link to +the rsync web site, and a few other items.

--verbose, -v
@@ -823,7 +865,7 @@ modify time of the finished directory ri that until a bunch of recursive copying has finished). However, these early directories don't yet have their completed mode, mtime, or ownership set -⁠-⁠ they have more restrictive rights until the subdirectory's copying -actually begins. This early-creation idiom can be avoiding by using the +actually begins. This early-creation idiom can be avoided by using the --omit-dir-times option.

Incremental recursion can be disabled using the --no-inc-recursive (--no-i-r) option.

@@ -1065,23 +1107,25 @@ an older rsync to list a single director
--mkpath
-

Create a missing path component of the destination arg. This allows rsync -to create multiple levels of missing destination dirs and to create a path -in which to put a single renamed file. Keep in mind that you'll need to -supply a trailing slash if you want the entire destination path to be -treated as a directory when copying a single arg (making rsync behave the -same way that it would if the path component of the destination had already -existed).

-

For example, the following creates a copy of file foo as bar in the sub/dir -directory, creating dirs "sub" and "sub/dir" if either do not yet exist:

-
-
rsync -ai --mkpath foo sub/dir/bar
-
-
-

If you instead ran the following, it would have created file foo in the -sub/dir/bar directory:

+

Create all missing path components of the destination path.

+

By default, rsync allows only the final component of the destination path +to not exist, which is an attempt to help you to validate your destination +path. With this option, rsync creates all the missing destination-path +components, just as if mkdir -p $DEST_PATH had been run on the receiving +side.

+

When specifying a destination path, including a trailing slash ensures that +the whole path is treated as directory names to be created, even when the +file list has a single item. See the COPYING TO A DIFFERENT NAME +section for full details on how rsync decides if a final destination-path +component should be created as a directory or not.

+

If you would like the newly-created destination dirs to match the dirs on +the sending side, you should be using --relative (-R) instead +of --mkpath. For instance, the following two commands result in the same +destination tree, but only the second command ensures that the +"some/extra/path" components match the dirs on the sending side:

-
rsync -ai --mkpath foo sub/dir/bar/
+
rsync -ai --mkpath host:some/extra/path/*.c some/extra/path/
+rsync -aiR host:some/extra/path/*.c ./
@@ -1478,6 +1522,14 @@ next transfer to behave as if it used -t).

+

A modern rsync that is using transfer protocol 30 or 31 conveys a modify +time using up to 8-bytes. If rsync is forced to speak an older protocol +(perhaps due to the remote rsync being older than 3.0.0) a modify time is +conveyed using 4-bytes. Prior to 3.2.7, these shorter values could convey +a date range of 13-Dec-1901 to 19-Jan-2038. Beginning with 3.2.7, these +4-byte values now convey a date range of 1-Jan-1970 to 7-Feb-2106. If you +have files dated older than 1970, make sure your rsync executables are +upgraded so that the full range of dates can be conveyed.

--atimes, -U
@@ -1637,6 +1689,7 @@ checksums, and the second name affects t
  • xxh64 (aka xxhash)
  • md5
  • md4
    • +
  • sha1
  • none

    • Run rsync --version to see the default checksum list compiled into your @@ -2236,7 +2289,9 @@ all names read from a .cvsignore file ar

    --old-args

    This option tells rsync to stop trying to protect the arg values on the -remote side from unintended word-splitting or other misinterpretation.

    +remote side from unintended word-splitting or other misinterpretation. +It also allows the client to treat an empty arg as a "." instead of +generating an error.

    The default in a modern rsync is for "shell-active" characters (including spaces) to be backslash-escaped in the args that are sent to the remote shell. The wildcard characters *, ?, [, & ] are not escaped in @@ -3296,13 +3351,18 @@ wild-card arg to try to infer this optio 

    rsync -av --list-only foo* dest/
    -

    Starting with rsync 3.1.0, the sizes output by --list-only are affected -by the --human-readable option. By default they will contain -digit separators, but higher levels of readability will output the sizes -with unit suffixes. Note also that the column width for the size output -has increased from 11 to 14 characters for all human-readable levels. Use ---no-h if you want just digits in the sizes, and the old column width of -11 characters.

    +

    This option always uses an output format that looks similar to this:

    +
    +
    drwxrwxr-x          4,096 2022/09/30 12:53:11 support
+-rw-rw-r--             80 2005/01/11 10:37:37 support/Makefile
+
    +
    +

    The only option that affects this output style is (as of 3.1.0) the +--human-readable (-h) option. The default is to output sizes +as byte counts with digit separators (in a 14-character-width column). +Specifying at least one -h option makes the sizes output with unit +suffixes. If you want old-style bytecount sizes without digit separators +(and an 11-character-width column) use --no-h.

    Compatibility note: when requesting a remote listing of files from an rsync that is version 2.6.3 or older, you may encounter an error if you ask for a non-recursive listing. This is because a file listing implies the @@ -4416,7 +4476,7 @@ the comments on the https://rsync.samba.org/.

    VERSION

    -

    This manpage is current for version 3.2.6 of rsync.

    +

    This manpage is current for version 3.2.7 of rsync.

    INTERNAL OPTIONS

    The options --server and --sender are used internally by rsync, and should never be typed by a user under normal circumstances. Some awareness of these @@ -4447,5 +4507,5 @@ people have later contributed to it. It Davison.

    Mailing lists for support and development are available at https://lists.samba.org/.

    -

    9 Sep 2022

    +

    20 Oct 2022

    diff -upN a/rsyncd.conf.5 b/rsyncd.conf.5 --- a/rsyncd.conf.5 +++ b/rsyncd.conf.5 @@ -1,4 +1,4 @@ -.TH "rsyncd.conf" "5" "9 Sep 2022" "rsyncd.conf from rsync 3.2.6" "User Commands" +.TH "rsyncd.conf" "5" "20 Oct 2022" "rsyncd.conf from rsync 3.2.7" "User Commands" .\" prefix=/usr .P .SH "NAME" @@ -159,6 +159,20 @@ This parameter specifies the directory i available in this module. You must specify this parameter for each module in \fBrsyncd.conf\fP. .IP +If the value contains a "/./" element then the path will be divided at that +point into a chroot dir and an inner-chroot subdir. If \fBuse\ chroot\fP +is set to false, though, the extraneous dot dir is just cleaned out of the +path. An example of this idiom is: +.RS 4 +.IP +.nf +path = /var/rsync/./module1 +.fi +.RE +.IP +This will (when chrooting) chroot to "/var/rsync" and set the inside-chroot +path to "/module1". +.IP You may base the path's value off of an environment variable by surrounding the variable name with percent signs. You can even reference a variable that is set by rsync when the user connects. For example, this would use @@ -184,29 +198,47 @@ follow symbolic links that are either ab path, and of complicating the preservation of users and groups by name (see below). .IP -As an additional safety feature, you can specify a dot-dir in the module's -"path" to indicate the point where the chroot should occur. This allows -rsync to run in a chroot with a non-"/" path for the top of the transfer -hierarchy. Doing this guards against unintended library loading (since -those absolute paths will not be inside the transfer hierarchy unless you -have used an unwise pathname), and lets you setup libraries for the chroot -that are outside of the transfer. For example, specifying -"/var/rsync/./module1" will chroot to the "/var/rsync" directory and set -the inside-chroot path to "/module1". If you had omitted the dot-dir, the -chroot would have used the whole path, and the inside-chroot path would -have been "/". -.IP -When both "use chroot" and "daemon chroot" are false, OR the inside-chroot -path of "use chroot" is not "/", rsync will: (1) munge symlinks by default -for security reasons (see "munge symlinks" for a way to turn this off, but -only if you trust your users), (2) substitute leading slashes in absolute -paths with the module's path (so that options such as \fB\-\-backup-dir\fP, -\fB\-\-compare-dest\fP, etc. interpret an absolute path as rooted in the module's -"path" dir), and (3) trim ".." path elements from args if rsync believes -they would escape the module hierarchy. The default for "use chroot" is -true, and is the safer choice (especially if the module is not read-only). +If \fBuse\ chroot\fP is not set, it defaults to trying to enable a chroot but +allows the daemon to continue (after logging a warning) if it fails. The +one exception to this is when a module's \fBpath\fP has a "/./" chroot +divider in it\ \-\- this causes an unset value to be treated as true for that +module. +.IP +Prior to rsync 3.2.7, the default value was "true". The new "unset" +default makes it easier to setup an rsync daemon as a non-root user or to +run a daemon on a system where chroot fails. Explicitly setting the value +to "true" in rsyncd.conf will always require the chroot to succeed. +.IP +It is also possible to specify a dot-dir in the module's "path" to +indicate that you want to chdir to the earlier part of the path and then +serve files from inside the latter part of the path (with sanitizing and +default symlink munging). This can be useful if you need some library dirs +inside the chroot (typically for uid & gid lookups) but don't want to put +the lib dir into the top of the served path (even though they can be hidden +with an \fBexclude\fP directive). However, a better choice for a modern +rsync setup is to use a \fBname\ converter\fP" and try to avoid inner lib +dirs altogether. See also the \fBdaemon\ chroot\fP parameter, which causes +rsync to chroot into its own chroot area before doing any path-related +chrooting. +.IP +If the daemon is serving the "/" dir (either directly or due to being +chrooted to the module's path), rsync does not do any path sanitizing or +(default) munging. +.IP +When it has to limit access to a particular subdir (either due to chroot +being disabled or having an inside-chroot path set), rsync will munge +symlinks (by default) and sanitize paths. Those that dislike munged +symlinks (and really, really trust their users to not break out of the +subdir) can disable the symlink munging via the "munge symlinks" +parameter. +.IP +When rsync is sanitizing paths, it trims ".." path elements from args that +it believes would escape the module hierarchy. It also substitutes leading +slashes in absolute paths with the module's path (so that options such as +\fB\-\-backup-dir\fP & \fB\-\-compare-dest\fP interpret an absolute path as rooted in +the module's "path" dir). .IP -When this parameter is enabled \fIand\fP the "name converter" parameter is +When a chroot is in effect \fIand\fP the "name converter" parameter is \fInot\fP set, the "numeric ids" parameter will default to being enabled (disabling name lookups). This means that if you manually setup name-lookup libraries in your chroot (instead of using a name converter) @@ -1256,7 +1288,7 @@ https://rsync.samba.org/. .P .SH "VERSION" .P -This manpage is current for version 3.2.6 of rsync. +This manpage is current for version 3.2.7 of rsync. .P .SH "CREDITS" .P diff -upN a/rsyncd.conf.5.html b/rsyncd.conf.5.html --- a/rsyncd.conf.5.html +++ b/rsyncd.conf.5.html @@ -170,6 +170,16 @@ no comment.

    This parameter specifies the directory in the daemon's filesystem to make available in this module. You must specify this parameter for each module in rsyncd.conf.

    +

    If the value contains a "/./" element then the path will be divided at that +point into a chroot dir and an inner-chroot subdir. If use chroot +is set to false, though, the extraneous dot dir is just cleaned out of the +path. An example of this idiom is:

    +
    +
    path = /var/rsync/./module1
+
    +
    +

    This will (when chrooting) chroot to "/var/rsync" and set the inside-chroot +path to "/module1".

    You may base the path's value off of an environment variable by surrounding the variable name with percent signs. You can even reference a variable that is set by rsync when the user connects. For example, this would use @@ -193,27 +203,41 @@ the disadvantages of requiring super-use follow symbolic links that are either absolute or outside of the new root path, and of complicating the preservation of users and groups by name (see below).

    -

    As an additional safety feature, you can specify a dot-dir in the module's -"path" to indicate the point where the chroot should occur. This allows -rsync to run in a chroot with a non-"/" path for the top of the transfer -hierarchy. Doing this guards against unintended library loading (since -those absolute paths will not be inside the transfer hierarchy unless you -have used an unwise pathname), and lets you setup libraries for the chroot -that are outside of the transfer. For example, specifying -"/var/rsync/./module1" will chroot to the "/var/rsync" directory and set -the inside-chroot path to "/module1". If you had omitted the dot-dir, the -chroot would have used the whole path, and the inside-chroot path would -have been "/".

    -

    When both "use chroot" and "daemon chroot" are false, OR the inside-chroot -path of "use chroot" is not "/", rsync will: (1) munge symlinks by default -for security reasons (see "munge symlinks" for a way to turn this off, but -only if you trust your users), (2) substitute leading slashes in absolute -paths with the module's path (so that options such as --backup-dir, ---compare-dest, etc. interpret an absolute path as rooted in the module's -"path" dir), and (3) trim ".." path elements from args if rsync believes -they would escape the module hierarchy. The default for "use chroot" is -true, and is the safer choice (especially if the module is not read-only).

    -

    When this parameter is enabled and the "name converter" parameter is +

    If use chroot is not set, it defaults to trying to enable a chroot but +allows the daemon to continue (after logging a warning) if it fails. The +one exception to this is when a module's path has a "/./" chroot +divider in it -⁠-⁠ this causes an unset value to be treated as true for that +module.

    +

    Prior to rsync 3.2.7, the default value was "true". The new "unset" +default makes it easier to setup an rsync daemon as a non-root user or to +run a daemon on a system where chroot fails. Explicitly setting the value +to "true" in rsyncd.conf will always require the chroot to succeed.

    +

    It is also possible to specify a dot-dir in the module's "path" to +indicate that you want to chdir to the earlier part of the path and then +serve files from inside the latter part of the path (with sanitizing and +default symlink munging). This can be useful if you need some library dirs +inside the chroot (typically for uid & gid lookups) but don't want to put +the lib dir into the top of the served path (even though they can be hidden +with an exclude directive). However, a better choice for a modern +rsync setup is to use a name converter" and try to avoid inner lib +dirs altogether. See also the daemon chroot parameter, which causes +rsync to chroot into its own chroot area before doing any path-related +chrooting.

    +

    If the daemon is serving the "/" dir (either directly or due to being +chrooted to the module's path), rsync does not do any path sanitizing or +(default) munging.

    +

    When it has to limit access to a particular subdir (either due to chroot +being disabled or having an inside-chroot path set), rsync will munge +symlinks (by default) and sanitize paths. Those that dislike munged +symlinks (and really, really trust their users to not break out of the +subdir) can disable the symlink munging via the "munge symlinks" +parameter.

    +

    When rsync is sanitizing paths, it trims ".." path elements from args that +it believes would escape the module hierarchy. It also substitutes leading +slashes in absolute paths with the module's path (so that options such as +--backup-dir & --compare-dest interpret an absolute path as rooted in +the module's "path" dir).

    +

    When a chroot is in effect and the "name converter" parameter is not set, the "numeric ids" parameter will default to being enabled (disabling name lookups). This means that if you manually setup name-lookup libraries in your chroot (instead of using a name converter) @@ -1162,7 +1186,7 @@ susan:herpass

    Please report bugs! The rsync bug tracking system is online at https://rsync.samba.org/.

    VERSION

    -

    This manpage is current for version 3.2.6 of rsync.

    +

    This manpage is current for version 3.2.7 of rsync.

    CREDITS

    Rsync is distributed under the GNU General Public License. See the file COPYING for details.

    @@ -1177,5 +1201,5 @@ people have later contributed to it. It Davison.

    Mailing lists for support and development are available at https://lists.samba.org/.

    -

    9 Sep 2022

    +

    20 Oct 2022