This patch introduces a global duplicate filter, primarily for deliveries to recipients that are listed under multiple local(8) aliases. The patch is relative to Postfix 2.5-20070501, but will apply with trivial effort to 2.4 as well. It is written to minimize the amount of code changes, and therefore is not maximally elegant. First I'll describe the problem, then the solution's user interface, its implementation, and finally, why the current implementation is not great. The cause of the problem ======================== By design, Postfix delivers mail for different local(8) destinations in parallel (local_destination_recipient_limit=1). This has both performance and security benefits. - A destination with a slow command (in an alias or .forward file) does not slow down deliveries to other local destinations. - A malicious user (or broken destination) cannot occupy all the delivery agent processes and thus block other mail. The number of delivery processes per local(8) destination is controlled with local_destination_concurrrency_limit (default: 2). The downside is that members of multiple local(8) aliases can receive multiple copies when one message is sent to multiple aliases. Technically, this is correct behavior, because those messages are not exact duplicates: they have different envelope sender attributes (Return-Path: headers) and different Delivered-To: headers. But many people don't care about these subtle details. Feature user interface ====================== The patch introduces two controls, as described in the following man page fragments. duplicate_filter_style (default: strict) The duplicate recipient filter policy: strict or pragmatic. strict Eliminate duplicate recipients only when they have the exact same original recipient information and DSN (delivery status notification) options. This avoids loss of mail with "one domain per mailbox" applications, and implements strict adherence to DSN standards. The X-Original-To: header distinguishes between accounts within single-mailbox domains. pragmatic Eliminate duplicate recipients even when they have different original recipient information or DSN (delivery status notification) options. This avoids duplicate deliveries to users that subscribe to multiple distribution lists. global_duplicate_filter_limit (default: 0) The size of the filter for duplicate deliveries to recipients that are listed under multiple local(8) aliases. Specify a limit that is several times larger than the largest alias expansion. The limit needs to be doubled on systems that use an after-queue content filter, and needs to be even larger when the mail system is congested with incoming or deferred mail. A filter with 10,000 entries can use up to 3MB of memory. This filter will fail to eliminate duplicate deliveries to recipients of multiple local(8) aliases, when one or more of the following conditions is true: o The filter size is too small. o Mail is submitted as one separate message for each local(8) alias, instead of being submitted as one message that is addressed to multiple local(8) aliases. o A local(8) alias pipes mail into an external mailing list manager that re-injects mail back into Postfix. This is essentially the same problem as the previous one. o A local(8) alias has no "owner-listname" alias. o The queue manager is restarted while a mailing list delivery is in progress. This causes loss of information about recipients that the queue manager has already seen. o The mail system is congested. Competing mail deliveries (incoming, deferred) push information out of the duplicate filter before it is needed. This requires a larger filter size. Feature implementation ====================== First, each queue file gets a few extra records, in a manner that is entirely compatible with earlier Postfix versions: - Every queue file is labeled with an actual transaction ID that is preserved only with "postsuper -r". The actual transaction ID does not show up in message headers. - Every message is labeled with an original transaction ID that is preserved with local(8) forwarding and with "postsuper -r". The original transaction ID does not show up in message headers. The queue manager uses this to eliminate duplicate deliveries to recipients who are subscribed to multiple local(8) aliases (within the restrictions outlined above in the man page fragments): - The queue manager maintains a cache that is indexed by (recipient, original TID, transport, nexthop). - If a given (recipient, original TID, transport, nexthop) are not found in the cache, a cache entry is made with as value the actual TID, and the recipient is scheduled for delivery. - If a given (recipient, original TID, transport, nexthop) are found in the cache with the same actual TID, the recipient is scheduled for delivery. Postfix assumes that this is a repeated delivery attempt of deferred mail. - If a given (recipient, original TID, transport, nexthop) are found in the cache with a different actual TID, Postfix deletes the recipient from the queue file and skips delivery. Implementation critique ======================= - It is always on. In principle the feature could be limited to mail that is being forwarded; and a timer could be set to purge all cache entries with the same original TID after some delay. - It has too many limitations. In particular, it fails to eliminate duplicate deliveries to members of multiple local(8) aliases when the system is congested with non-list mail. This makes the feature difficult to configure. This point is related to the next critique. - The queue manager is the wrong place. A possible remedy is to have local(8) send its forwarded mail through a new local server process whose sole purpose is to weed out duplicates. This server can purge all cache entries with the same original TID after some delay. The server can even terminate after the cache becomes empty, which makes Postfix more robust against memory corruption problems (bad hardware or software). diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/TODO ./TODO *** /var/tmp/postfix-2.5-20070501/TODO Wed Dec 31 19:00:00 1969 --- ./TODO Mon May 7 15:00:20 2007 *************** *** 0 **** --- 1 ---- + qmgr save duplicate cache diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/mantools/postlink ./mantools/postlink *** /var/tmp/postfix-2.5-20070501/mantools/postlink Tue May 1 10:19:10 2007 --- ./mantools/postlink Tue May 8 10:02:27 2007 *************** *** 159,165 **** --- 159,167 ---- s;\bdisable_vrfy_command\b;$&;g; s;\bdont_remove\b;$&;g; s;\bdouble_bounce_sender\b;$&;g; + s;\bglobal_dupli[-]*\n* *[]*cate_filter_limit\b;$&;g; s;\bdupli[-]*\n* *[]*cate_filter_limit\b;$&;g; + s;\bdupli[-]*\n* *[]*cate_filter_style\b;$&;g; s;\bempty_address_recip[-]*\n* *[]*ient\b;$&;g; s;\benable_original_recip[-]*\n* *[]*ient\b;$&;g; s;\benable_errors_to\b;$&;g; diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/proto/postconf.proto ./proto/postconf.proto *** /var/tmp/postfix-2.5-20070501/proto/postconf.proto Tue May 1 17:51:47 2007 --- ./proto/postconf.proto Tue May 8 15:15:08 2007 *************** *** 10587,10589 **** --- 10587,10657 ---- behavior was hard-coded to be "always on".

This feature is available in Postfix 2.5 and later.

+ + %PARAM duplicate_filter_style strict + +

The duplicate recipient filter policy: strict or pragmatic. + With Postfix ≤ 2.2 use enable_original_recipient instead. With + Postfix 2.3 and 2.4, duplicate recipient elimination is currently + broken, because there is no option to ignore differences in DSN + (delivery status notification) attributes.

+ +
+ +
strict
Eliminate duplicate recipients only when + they have the exact same original recipient information and DSN + (delivery status notification) options. This avoids loss of mail + with "one domain per mailbox" applications, and implements strict + adherence to DSN standards. The X-Original-To: header distinguishes + between accounts within single-mailbox domains.
+ +
pragmatic
Eliminate duplicate recipients even + when they have different original recipient information or DSN + (delivery status notification) options. This avoids duplicate + deliveries to users that subscribe to multiple distribution lists. +
+ +
+ +

This feature is available in Postfix 2.5 and later.

+ + %PARAM global_duplicate_filter_limit 0 + +

The size of the filter for duplicate deliveries to recipients + that are listed under multiple local(8) aliases.

+ +

Specify a limit that is several times larger than the largest + alias expansion. The limit needs to be doubled on systems that use + an after-queue content filter, and needs to be even larger when the + mail system is congested with incoming or deferred mail. A filter + with 10,000 entries can use up to 3MB of memory.

+ +

This filter will fail to eliminate duplicate deliveries to + recipients of multiple local(8) aliases, when one or more of the + following conditions is true:

+ + + +

This feature is available in Postfix 2.5 and later.

diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/cleanup/Makefile.in ./src/cleanup/Makefile.in *** /var/tmp/postfix-2.5-20070501/src/cleanup/Makefile.in Sat Mar 17 13:51:40 2007 --- ./src/cleanup/Makefile.in Mon May 7 20:55:44 2007 *************** *** 310,315 **** --- 310,316 ---- cleanup.o: ../../include/attr.h cleanup.o: ../../include/been_here.h cleanup.o: ../../include/cleanup_user.h + cleanup.o: ../../include/ctable.h cleanup.o: ../../include/dict.h cleanup.o: ../../include/header_opts.h cleanup.o: ../../include/htable.h *************** *** 329,338 **** --- 330,341 ---- cleanup.o: ../../include/mymalloc.h cleanup.o: ../../include/nvtable.h cleanup.o: ../../include/rec_type.h + cleanup.o: ../../include/recipient_list.h cleanup.o: ../../include/record.h cleanup.o: ../../include/resolve_clnt.h cleanup.o: ../../include/string_list.h cleanup.o: ../../include/sys_defs.h + cleanup.o: ../../include/tcache.h cleanup.o: ../../include/tok822.h cleanup.o: ../../include/vbuf.h cleanup.o: ../../include/vstream.h *************** *** 344,349 **** --- 347,353 ---- cleanup_addr.o: ../../include/been_here.h cleanup_addr.o: ../../include/canon_addr.h cleanup_addr.o: ../../include/cleanup_user.h + cleanup_addr.o: ../../include/ctable.h cleanup_addr.o: ../../include/dict.h cleanup_addr.o: ../../include/dsn_mask.h cleanup_addr.o: ../../include/ext_prop.h *************** *** 365,374 **** --- 369,380 ---- cleanup_addr.o: ../../include/mymalloc.h cleanup_addr.o: ../../include/nvtable.h cleanup_addr.o: ../../include/rec_type.h + cleanup_addr.o: ../../include/recipient_list.h cleanup_addr.o: ../../include/resolve_clnt.h cleanup_addr.o: ../../include/string_list.h cleanup_addr.o: ../../include/stringops.h cleanup_addr.o: ../../include/sys_defs.h + cleanup_addr.o: ../../include/tcache.h cleanup_addr.o: ../../include/tok822.h cleanup_addr.o: ../../include/vbuf.h cleanup_addr.o: ../../include/vstream.h *************** *** 380,385 **** --- 386,392 ---- cleanup_api.o: ../../include/been_here.h cleanup_api.o: ../../include/bounce.h cleanup_api.o: ../../include/cleanup_user.h + cleanup_api.o: ../../include/ctable.h cleanup_api.o: ../../include/deliver_request.h cleanup_api.o: ../../include/dict.h cleanup_api.o: ../../include/dsn.h *************** *** 407,412 **** --- 414,420 ---- cleanup_api.o: ../../include/resolve_clnt.h cleanup_api.o: ../../include/string_list.h cleanup_api.o: ../../include/sys_defs.h + cleanup_api.o: ../../include/tcache.h cleanup_api.o: ../../include/tok822.h cleanup_api.o: ../../include/vbuf.h cleanup_api.o: ../../include/vstream.h *************** *** 416,421 **** --- 424,430 ---- cleanup_body_edit.o: ../../include/argv.h cleanup_body_edit.o: ../../include/been_here.h cleanup_body_edit.o: ../../include/cleanup_user.h + cleanup_body_edit.o: ../../include/ctable.h cleanup_body_edit.o: ../../include/dict.h cleanup_body_edit.o: ../../include/header_opts.h cleanup_body_edit.o: ../../include/htable.h *************** *** 430,439 **** --- 439,450 ---- cleanup_body_edit.o: ../../include/mymalloc.h cleanup_body_edit.o: ../../include/nvtable.h cleanup_body_edit.o: ../../include/rec_type.h + cleanup_body_edit.o: ../../include/recipient_list.h cleanup_body_edit.o: ../../include/record.h cleanup_body_edit.o: ../../include/resolve_clnt.h cleanup_body_edit.o: ../../include/string_list.h cleanup_body_edit.o: ../../include/sys_defs.h + cleanup_body_edit.o: ../../include/tcache.h cleanup_body_edit.o: ../../include/tok822.h cleanup_body_edit.o: ../../include/vbuf.h cleanup_body_edit.o: ../../include/vstream.h *************** *** 445,450 **** --- 456,462 ---- cleanup_bounce.o: ../../include/been_here.h cleanup_bounce.o: ../../include/bounce.h cleanup_bounce.o: ../../include/cleanup_user.h + cleanup_bounce.o: ../../include/ctable.h cleanup_bounce.o: ../../include/deliver_request.h cleanup_bounce.o: ../../include/dict.h cleanup_bounce.o: ../../include/dsn.h *************** *** 476,481 **** --- 488,494 ---- cleanup_bounce.o: ../../include/string_list.h cleanup_bounce.o: ../../include/stringops.h cleanup_bounce.o: ../../include/sys_defs.h + cleanup_bounce.o: ../../include/tcache.h cleanup_bounce.o: ../../include/tok822.h cleanup_bounce.o: ../../include/vbuf.h cleanup_bounce.o: ../../include/vstream.h *************** *** 486,491 **** --- 499,505 ---- cleanup_envelope.o: ../../include/attr.h cleanup_envelope.o: ../../include/been_here.h cleanup_envelope.o: ../../include/cleanup_user.h + cleanup_envelope.o: ../../include/ctable.h cleanup_envelope.o: ../../include/dict.h cleanup_envelope.o: ../../include/dsn_mask.h cleanup_envelope.o: ../../include/header_opts.h *************** *** 506,516 **** --- 520,532 ---- cleanup_envelope.o: ../../include/qmgr_user.h cleanup_envelope.o: ../../include/rec_attr_map.h cleanup_envelope.o: ../../include/rec_type.h + cleanup_envelope.o: ../../include/recipient_list.h cleanup_envelope.o: ../../include/record.h cleanup_envelope.o: ../../include/resolve_clnt.h cleanup_envelope.o: ../../include/string_list.h cleanup_envelope.o: ../../include/stringops.h cleanup_envelope.o: ../../include/sys_defs.h + cleanup_envelope.o: ../../include/tcache.h cleanup_envelope.o: ../../include/tok822.h cleanup_envelope.o: ../../include/vbuf.h cleanup_envelope.o: ../../include/verp_sender.h *************** *** 522,527 **** --- 538,544 ---- cleanup_extracted.o: ../../include/attr.h cleanup_extracted.o: ../../include/been_here.h cleanup_extracted.o: ../../include/cleanup_user.h + cleanup_extracted.o: ../../include/ctable.h cleanup_extracted.o: ../../include/dict.h cleanup_extracted.o: ../../include/dsn_mask.h cleanup_extracted.o: ../../include/header_opts.h *************** *** 542,552 **** --- 559,571 ---- cleanup_extracted.o: ../../include/qmgr_user.h cleanup_extracted.o: ../../include/rec_attr_map.h cleanup_extracted.o: ../../include/rec_type.h + cleanup_extracted.o: ../../include/recipient_list.h cleanup_extracted.o: ../../include/record.h cleanup_extracted.o: ../../include/resolve_clnt.h cleanup_extracted.o: ../../include/string_list.h cleanup_extracted.o: ../../include/stringops.h cleanup_extracted.o: ../../include/sys_defs.h + cleanup_extracted.o: ../../include/tcache.h cleanup_extracted.o: ../../include/tok822.h cleanup_extracted.o: ../../include/vbuf.h cleanup_extracted.o: ../../include/vstream.h *************** *** 556,561 **** --- 575,581 ---- cleanup_final.o: ../../include/argv.h cleanup_final.o: ../../include/been_here.h cleanup_final.o: ../../include/cleanup_user.h + cleanup_final.o: ../../include/ctable.h cleanup_final.o: ../../include/dict.h cleanup_final.o: ../../include/header_opts.h cleanup_final.o: ../../include/htable.h *************** *** 570,578 **** --- 590,600 ---- cleanup_final.o: ../../include/mymalloc.h cleanup_final.o: ../../include/nvtable.h cleanup_final.o: ../../include/rec_type.h + cleanup_final.o: ../../include/recipient_list.h cleanup_final.o: ../../include/resolve_clnt.h cleanup_final.o: ../../include/string_list.h cleanup_final.o: ../../include/sys_defs.h + cleanup_final.o: ../../include/tcache.h cleanup_final.o: ../../include/tok822.h cleanup_final.o: ../../include/vbuf.h cleanup_final.o: ../../include/vstream.h *************** *** 582,587 **** --- 604,610 ---- cleanup_init.o: ../../include/argv.h cleanup_init.o: ../../include/been_here.h cleanup_init.o: ../../include/cleanup_user.h + cleanup_init.o: ../../include/ctable.h cleanup_init.o: ../../include/dict.h cleanup_init.o: ../../include/ext_prop.h cleanup_init.o: ../../include/flush_clnt.h *************** *** 602,611 **** --- 625,637 ---- cleanup_init.o: ../../include/mymalloc.h cleanup_init.o: ../../include/name_mask.h cleanup_init.o: ../../include/nvtable.h + cleanup_init.o: ../../include/rcpt_key.h + cleanup_init.o: ../../include/recipient_list.h cleanup_init.o: ../../include/resolve_clnt.h cleanup_init.o: ../../include/string_list.h cleanup_init.o: ../../include/stringops.h cleanup_init.o: ../../include/sys_defs.h + cleanup_init.o: ../../include/tcache.h cleanup_init.o: ../../include/tok822.h cleanup_init.o: ../../include/vbuf.h cleanup_init.o: ../../include/vstream.h *************** *** 615,620 **** --- 641,647 ---- cleanup_map11.o: ../../include/argv.h cleanup_map11.o: ../../include/been_here.h cleanup_map11.o: ../../include/cleanup_user.h + cleanup_map11.o: ../../include/ctable.h cleanup_map11.o: ../../include/dict.h cleanup_map11.o: ../../include/header_opts.h cleanup_map11.o: ../../include/htable.h *************** *** 631,639 **** --- 658,668 ---- cleanup_map11.o: ../../include/nvtable.h cleanup_map11.o: ../../include/quote_822_local.h cleanup_map11.o: ../../include/quote_flags.h + cleanup_map11.o: ../../include/recipient_list.h cleanup_map11.o: ../../include/resolve_clnt.h cleanup_map11.o: ../../include/string_list.h cleanup_map11.o: ../../include/sys_defs.h + cleanup_map11.o: ../../include/tcache.h cleanup_map11.o: ../../include/tok822.h cleanup_map11.o: ../../include/vbuf.h cleanup_map11.o: ../../include/vstream.h *************** *** 643,648 **** --- 672,678 ---- cleanup_map1n.o: ../../include/argv.h cleanup_map1n.o: ../../include/been_here.h cleanup_map1n.o: ../../include/cleanup_user.h + cleanup_map1n.o: ../../include/ctable.h cleanup_map1n.o: ../../include/dict.h cleanup_map1n.o: ../../include/header_opts.h cleanup_map1n.o: ../../include/htable.h *************** *** 660,668 **** --- 690,700 ---- cleanup_map1n.o: ../../include/nvtable.h cleanup_map1n.o: ../../include/quote_822_local.h cleanup_map1n.o: ../../include/quote_flags.h + cleanup_map1n.o: ../../include/recipient_list.h cleanup_map1n.o: ../../include/resolve_clnt.h cleanup_map1n.o: ../../include/string_list.h cleanup_map1n.o: ../../include/sys_defs.h + cleanup_map1n.o: ../../include/tcache.h cleanup_map1n.o: ../../include/tok822.h cleanup_map1n.o: ../../include/vbuf.h cleanup_map1n.o: ../../include/vstream.h *************** *** 672,677 **** --- 704,710 ---- cleanup_masquerade.o: ../../include/argv.h cleanup_masquerade.o: ../../include/been_here.h cleanup_masquerade.o: ../../include/cleanup_user.h + cleanup_masquerade.o: ../../include/ctable.h cleanup_masquerade.o: ../../include/dict.h cleanup_masquerade.o: ../../include/header_opts.h cleanup_masquerade.o: ../../include/htable.h *************** *** 688,697 **** --- 721,732 ---- cleanup_masquerade.o: ../../include/nvtable.h cleanup_masquerade.o: ../../include/quote_822_local.h cleanup_masquerade.o: ../../include/quote_flags.h + cleanup_masquerade.o: ../../include/recipient_list.h cleanup_masquerade.o: ../../include/resolve_clnt.h cleanup_masquerade.o: ../../include/string_list.h cleanup_masquerade.o: ../../include/stringops.h cleanup_masquerade.o: ../../include/sys_defs.h + cleanup_masquerade.o: ../../include/tcache.h cleanup_masquerade.o: ../../include/tok822.h cleanup_masquerade.o: ../../include/vbuf.h cleanup_masquerade.o: ../../include/vstream.h *************** *** 703,708 **** --- 738,744 ---- cleanup_message.o: ../../include/been_here.h cleanup_message.o: ../../include/cleanup_user.h cleanup_message.o: ../../include/conv_time.h + cleanup_message.o: ../../include/ctable.h cleanup_message.o: ../../include/dict.h cleanup_message.o: ../../include/dsn_util.h cleanup_message.o: ../../include/ext_prop.h *************** *** 728,739 **** --- 764,777 ---- cleanup_message.o: ../../include/quote_822_local.h cleanup_message.o: ../../include/quote_flags.h cleanup_message.o: ../../include/rec_type.h + cleanup_message.o: ../../include/recipient_list.h cleanup_message.o: ../../include/record.h cleanup_message.o: ../../include/resolve_clnt.h cleanup_message.o: ../../include/split_at.h cleanup_message.o: ../../include/string_list.h cleanup_message.o: ../../include/stringops.h cleanup_message.o: ../../include/sys_defs.h + cleanup_message.o: ../../include/tcache.h cleanup_message.o: ../../include/tok822.h cleanup_message.o: ../../include/vbuf.h cleanup_message.o: ../../include/vstream.h *************** *** 744,749 **** --- 782,788 ---- cleanup_milter.o: ../../include/attr.h cleanup_milter.o: ../../include/been_here.h cleanup_milter.o: ../../include/cleanup_user.h + cleanup_milter.o: ../../include/ctable.h cleanup_milter.o: ../../include/dict.h cleanup_milter.o: ../../include/dsn_mask.h cleanup_milter.o: ../../include/header_opts.h *************** *** 768,778 **** --- 807,819 ---- cleanup_milter.o: ../../include/quote_flags.h cleanup_milter.o: ../../include/rec_attr_map.h cleanup_milter.o: ../../include/rec_type.h + cleanup_milter.o: ../../include/recipient_list.h cleanup_milter.o: ../../include/record.h cleanup_milter.o: ../../include/resolve_clnt.h cleanup_milter.o: ../../include/string_list.h cleanup_milter.o: ../../include/stringops.h cleanup_milter.o: ../../include/sys_defs.h + cleanup_milter.o: ../../include/tcache.h cleanup_milter.o: ../../include/tok822.h cleanup_milter.o: ../../include/vbuf.h cleanup_milter.o: ../../include/vstream.h *************** *** 782,787 **** --- 823,829 ---- cleanup_out.o: ../../include/argv.h cleanup_out.o: ../../include/been_here.h cleanup_out.o: ../../include/cleanup_user.h + cleanup_out.o: ../../include/ctable.h cleanup_out.o: ../../include/dict.h cleanup_out.o: ../../include/header_opts.h cleanup_out.o: ../../include/htable.h *************** *** 798,808 **** --- 840,852 ---- cleanup_out.o: ../../include/mymalloc.h cleanup_out.o: ../../include/nvtable.h cleanup_out.o: ../../include/rec_type.h + cleanup_out.o: ../../include/recipient_list.h cleanup_out.o: ../../include/record.h cleanup_out.o: ../../include/resolve_clnt.h cleanup_out.o: ../../include/split_at.h cleanup_out.o: ../../include/string_list.h cleanup_out.o: ../../include/sys_defs.h + cleanup_out.o: ../../include/tcache.h cleanup_out.o: ../../include/tok822.h cleanup_out.o: ../../include/vbuf.h cleanup_out.o: ../../include/vstream.h *************** *** 814,819 **** --- 858,864 ---- cleanup_out_recipient.o: ../../include/been_here.h cleanup_out_recipient.o: ../../include/bounce.h cleanup_out_recipient.o: ../../include/cleanup_user.h + cleanup_out_recipient.o: ../../include/ctable.h cleanup_out_recipient.o: ../../include/deliver_request.h cleanup_out_recipient.o: ../../include/dict.h cleanup_out_recipient.o: ../../include/dsn.h *************** *** 837,847 **** --- 882,894 ---- cleanup_out_recipient.o: ../../include/msg_stats.h cleanup_out_recipient.o: ../../include/mymalloc.h cleanup_out_recipient.o: ../../include/nvtable.h + cleanup_out_recipient.o: ../../include/rcpt_key.h cleanup_out_recipient.o: ../../include/rec_type.h cleanup_out_recipient.o: ../../include/recipient_list.h cleanup_out_recipient.o: ../../include/resolve_clnt.h cleanup_out_recipient.o: ../../include/string_list.h cleanup_out_recipient.o: ../../include/sys_defs.h + cleanup_out_recipient.o: ../../include/tcache.h cleanup_out_recipient.o: ../../include/tok822.h cleanup_out_recipient.o: ../../include/trace.h cleanup_out_recipient.o: ../../include/vbuf.h *************** *** 852,857 **** --- 899,905 ---- cleanup_region.o: ../../include/argv.h cleanup_region.o: ../../include/been_here.h cleanup_region.o: ../../include/cleanup_user.h + cleanup_region.o: ../../include/ctable.h cleanup_region.o: ../../include/dict.h cleanup_region.o: ../../include/header_opts.h cleanup_region.o: ../../include/htable.h *************** *** 865,873 **** --- 913,923 ---- cleanup_region.o: ../../include/msg.h cleanup_region.o: ../../include/mymalloc.h cleanup_region.o: ../../include/nvtable.h + cleanup_region.o: ../../include/recipient_list.h cleanup_region.o: ../../include/resolve_clnt.h cleanup_region.o: ../../include/string_list.h cleanup_region.o: ../../include/sys_defs.h + cleanup_region.o: ../../include/tcache.h cleanup_region.o: ../../include/tok822.h cleanup_region.o: ../../include/vbuf.h cleanup_region.o: ../../include/vstream.h *************** *** 878,883 **** --- 928,934 ---- cleanup_rewrite.o: ../../include/attr.h cleanup_rewrite.o: ../../include/been_here.h cleanup_rewrite.o: ../../include/cleanup_user.h + cleanup_rewrite.o: ../../include/ctable.h cleanup_rewrite.o: ../../include/dict.h cleanup_rewrite.o: ../../include/header_opts.h cleanup_rewrite.o: ../../include/htable.h *************** *** 895,904 **** --- 946,957 ---- cleanup_rewrite.o: ../../include/nvtable.h cleanup_rewrite.o: ../../include/quote_822_local.h cleanup_rewrite.o: ../../include/quote_flags.h + cleanup_rewrite.o: ../../include/recipient_list.h cleanup_rewrite.o: ../../include/resolve_clnt.h cleanup_rewrite.o: ../../include/rewrite_clnt.h cleanup_rewrite.o: ../../include/string_list.h cleanup_rewrite.o: ../../include/sys_defs.h + cleanup_rewrite.o: ../../include/tcache.h cleanup_rewrite.o: ../../include/tok822.h cleanup_rewrite.o: ../../include/vbuf.h cleanup_rewrite.o: ../../include/vstream.h *************** *** 909,914 **** --- 962,968 ---- cleanup_state.o: ../../include/attr.h cleanup_state.o: ../../include/been_here.h cleanup_state.o: ../../include/cleanup_user.h + cleanup_state.o: ../../include/ctable.h cleanup_state.o: ../../include/dict.h cleanup_state.o: ../../include/header_opts.h cleanup_state.o: ../../include/htable.h *************** *** 924,932 **** --- 978,988 ---- cleanup_state.o: ../../include/mime_state.h cleanup_state.o: ../../include/mymalloc.h cleanup_state.o: ../../include/nvtable.h + cleanup_state.o: ../../include/recipient_list.h cleanup_state.o: ../../include/resolve_clnt.h cleanup_state.o: ../../include/string_list.h cleanup_state.o: ../../include/sys_defs.h + cleanup_state.o: ../../include/tcache.h cleanup_state.o: ../../include/tok822.h cleanup_state.o: ../../include/vbuf.h cleanup_state.o: ../../include/vstream.h diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/cleanup/cleanup.c ./src/cleanup/cleanup.c *** /var/tmp/postfix-2.5-20070501/src/cleanup/cleanup.c Tue May 1 13:43:57 2007 --- ./src/cleanup/cleanup.c Tue May 8 13:23:05 2007 *************** *** 290,295 **** --- 290,299 ---- /* from each original recipient. /* .IP "\fBvirtual_alias_recursion_limit (1000)\fR" /* The maximal nesting depth of virtual alias expansion. + /* .PP + /* Available in Postfix version 2.5 and later: + /* .IP "\fBduplicate_filter_style (strict)\fR" + /* The duplicate recipient filter policy: strict or pragmatic. /* MISCELLANEOUS CONTROLS /* .ad /* .fi diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/cleanup/cleanup.h ./src/cleanup/cleanup.h *** /var/tmp/postfix-2.5-20070501/src/cleanup/cleanup.h Thu Feb 15 14:34:34 2007 --- ./src/cleanup/cleanup.h Tue May 8 13:25:17 2007 *************** *** 52,57 **** --- 52,59 ---- MAIL_STREAM *handle; /* mail stream handle */ char *queue_name; /* queue name */ char *queue_id; /* queue file basename */ + char *act_tid; /* actual transaction id */ + char *org_tid; /* original transaction id */ struct timeval arrival_time; /* arrival time */ char *fullname; /* envelope sender full name */ char *sender; /* envelope sender address */ *************** *** 139,144 **** --- 141,148 ---- extern int cleanup_masq_flags; extern MAPS *cleanup_send_bcc_maps; extern MAPS *cleanup_rcpt_bcc_maps; + + extern int cleanup_rcpt_key_flags; /* * Character filters. diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/cleanup/cleanup_envelope.c ./src/cleanup/cleanup_envelope.c *** /var/tmp/postfix-2.5-20070501/src/cleanup/cleanup_envelope.c Tue Jan 16 14:08:07 2007 --- ./src/cleanup/cleanup_envelope.c Tue May 8 13:59:29 2007 *************** *** 236,241 **** --- 236,271 ---- } /* + * The invisible Postfix original and actual transaction IDs are + * assigned when mail enters the queue. The org_tid is propagated + * when mail is forwarded internally, and the act_tid is generated + * when a queue file is created. The queue manager discards a + * (recipient + org_tid) instance when it has already been seen + * together with a different act_tid. + * + * Since the act_tid protects a (recipient + org_tid) against duplicate + * elimination, either both transaction IDs must be preserved after + * "postsuper -r", or none must be preserved. We preserve both. + * + * XXX The queue_id is unique to within a second, but we include the + * time in microseconds anyway just in case the queue ID generating + * algorithm changes. + */ + if (state->act_tid == 0) { + vstring_sprintf(state->temp1, "%ld.%ld.%s", + (long) state->arrival_time.tv_sec, + (long) state->arrival_time.tv_usec, + state->queue_id); + state->act_tid = mystrdup(STR(state->temp1)); + } + cleanup_out_format(state, REC_TYPE_ATTR, "%s=%s", + MAIL_ATTR_ACT_TID, state->act_tid); + if (state->org_tid == 0) + state->org_tid = mystrdup(state->act_tid); + cleanup_out_format(state, REC_TYPE_ATTR, "%s=%s", + MAIL_ATTR_ORG_TID, state->org_tid); + + /* * XXX This works by accident, because the sender is recorded at the * beginning of the envelope segment. */ *************** *** 357,362 **** --- 387,404 ---- /* Generate our own expiration time base record. */ cleanup_out_format(state, REC_TYPE_ATTR, "%s=%ld", MAIL_ATTR_CREATE_TIME, (long) time((time_t *) 0)); + return; + } + if (type == REC_TYPE_ATID) { + /* First instance wins. */ + if (state->act_tid == 0 && *buf) + state->act_tid = mystrdup(buf); + return; + } + if (type == REC_TYPE_OTID) { + /* First instance wins. */ + if (state->org_tid == 0 && *buf) + state->org_tid = mystrdup(buf); return; } if (type == REC_TYPE_FULL) { diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/cleanup/cleanup_init.c ./src/cleanup/cleanup_init.c *** /var/tmp/postfix-2.5-20070501/src/cleanup/cleanup_init.c Tue May 1 09:42:14 2007 --- ./src/cleanup/cleanup_init.c Mon May 7 20:42:43 2007 *************** *** 93,98 **** --- 93,99 ---- #include /* milter_macro_v */ #include #include + #include /* Application-specific. */ *************** *** 111,116 **** --- 112,122 ---- VSTRING *cleanup_trace_path; /* + * Control of recipient duplicate elimination. + */ + int cleanup_rcpt_key_flags = RCPT_KEY_FLAG_FOLD; + + /* * Tunable parameters. */ int var_hopcount_limit; /* max mailer hop count */ *************** *** 161,166 **** --- 167,173 ---- char *var_milt_unk_macros; /* unknown command macros */ char *var_cleanup_milters; /* non-SMTP mail */ int var_auto_8bit_enc_hdr; /* auto-detect 8bit encoding header */ + char *var_dup_filter_style; /* strict or pragmatic */ CONFIG_INT_TABLE cleanup_int_table[] = { VAR_HOPCOUNT_LIMIT, DEF_HOPCOUNT_LIMIT, &var_hopcount_limit, 1, 0, *************** *** 223,228 **** --- 230,236 ---- VAR_MILT_EOD_MACROS, DEF_MILT_EOD_MACROS, &var_milt_eod_macros, 0, 0, VAR_MILT_UNK_MACROS, DEF_MILT_UNK_MACROS, &var_milt_unk_macros, 0, 0, VAR_CLEANUP_MILTERS, DEF_CLEANUP_MILTERS, &var_cleanup_milters, 0, 0, + VAR_DUP_FILTER_STYLE, DEF_DUP_FILTER_STYLE, &var_dup_filter_style, 1, 0, 0, }; *************** *** 433,436 **** --- 441,450 ---- cleanup_strip_chars = vstring_alloc(strlen(var_msg_strip_chars)); unescape(cleanup_strip_chars, var_msg_strip_chars); } + + /* + * Duplicate filter style. + */ + cleanup_rcpt_key_flags |= + rcpt_key_style(VAR_DUP_FILTER_STYLE, var_dup_filter_style); } diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/cleanup/cleanup_out_recipient.c ./src/cleanup/cleanup_out_recipient.c *** /var/tmp/postfix-2.5-20070501/src/cleanup/cleanup_out_recipient.c Thu Jan 5 19:22:04 2006 --- ./src/cleanup/cleanup_out_recipient.c Tue May 8 14:17:06 2007 *************** *** 79,84 **** --- 79,85 ---- #include /* cleanup_trace_path */ #include #include + #include /* Application-specific. */ *************** *** 112,117 **** --- 113,120 ---- const char *orcpt, const char *recip) { + RECIPIENT rcpt; + char *key; ARGV *argv; char **cpp; *************** *** 124,129 **** --- 127,137 ---- dsn_orcpt = ""; /* + * Not elegant, but avoids up-stream damage. + */ + RECIPIENT_ASSIGN(&rcpt, (off_t) 0, dsn_orcpt, dsn_notify, orcpt, recip); + + /* * Distinguish between different original recipient addresses that map * onto the same mailbox. The recipient will use our original recipient * message header to figure things out. *************** *** 132,139 **** if ((state->flags & CLEANUP_FLAG_MAP_OK) == 0 || cleanup_virt_alias_maps == 0) { ! if (been_here(state->dups, "%s\n%d\n%s\n%s", ! dsn_orcpt, dsn_notify, orcpt, recip) == 0) { if (dsn_notify) cleanup_out_format(state, REC_TYPE_ATTR, "%s=%d", MAIL_ATTR_DSN_NOTIFY, dsn_notify); --- 140,148 ---- if ((state->flags & CLEANUP_FLAG_MAP_OK) == 0 || cleanup_virt_alias_maps == 0) { ! key = vstring_str(rcpt_key_format(state->temp1, &rcpt, ! cleanup_rcpt_key_flags)); ! if (been_here_fixed(state->dups, key) == 0) { if (dsn_notify) cleanup_out_format(state, REC_TYPE_ATTR, "%s=%d", MAIL_ATTR_DSN_NOTIFY, dsn_notify); *************** *** 183,189 **** * notification records. */ else { - RECIPIENT rcpt; DSN dsn; argv = cleanup_map1n_internal(state, recip, cleanup_virt_alias_maps, --- 192,197 ---- *************** *** 192,205 **** && (argv->argc > 1 || strcmp(recip, argv->argv[0]) != 0)) { (void) DSN_SIMPLE(&dsn, "2.0.0", "alias expanded"); dsn.action = "expanded"; - RECIPIENT_ASSIGN(&rcpt, 0, dsn_orcpt, dsn_notify, orcpt, recip); cleanup_trace_append(state, &rcpt, &dsn); dsn_notify = (dsn_notify == DSN_NOTIFY_SUCCESS ? DSN_NOTIFY_NEVER : dsn_notify & ~DSN_NOTIFY_SUCCESS); } for (cpp = argv->argv; *cpp; cpp++) { ! if (been_here(state->dups, "%s\n%d\n%s\n%s", ! dsn_orcpt, dsn_notify, orcpt, *cpp) == 0) { if (dsn_notify) cleanup_out_format(state, REC_TYPE_ATTR, "%s=%d", MAIL_ATTR_DSN_NOTIFY, dsn_notify); --- 200,215 ---- && (argv->argc > 1 || strcmp(recip, argv->argv[0]) != 0)) { (void) DSN_SIMPLE(&dsn, "2.0.0", "alias expanded"); dsn.action = "expanded"; cleanup_trace_append(state, &rcpt, &dsn); dsn_notify = (dsn_notify == DSN_NOTIFY_SUCCESS ? DSN_NOTIFY_NEVER : dsn_notify & ~DSN_NOTIFY_SUCCESS); + rcpt.dsn_notify = dsn_notify; } for (cpp = argv->argv; *cpp; cpp++) { ! rcpt.address = *cpp; ! key = vstring_str(rcpt_key_format(state->temp1, &rcpt, ! cleanup_rcpt_key_flags)); ! if (been_here_fixed(state->dups, key) == 0) { if (dsn_notify) cleanup_out_format(state, REC_TYPE_ATTR, "%s=%d", MAIL_ATTR_DSN_NOTIFY, dsn_notify); diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/cleanup/cleanup_state.c ./src/cleanup/cleanup_state.c *** /var/tmp/postfix-2.5-20070501/src/cleanup/cleanup_state.c Wed Jan 17 19:38:29 2007 --- ./src/cleanup/cleanup_state.c Tue May 8 13:53:39 2007 *************** *** 71,76 **** --- 71,78 ---- state->handle = 0; state->queue_name = 0; state->queue_id = 0; + state->act_tid = 0; + state->org_tid = 0; state->arrival_time.tv_sec = state->arrival_time.tv_usec = 0; state->fullname = 0; state->sender = 0; *************** *** 146,151 **** --- 148,157 ---- myfree(state->queue_name); if (state->queue_id) myfree(state->queue_id); + if (state->act_tid) + myfree(state->act_tid); + if (state->org_tid) + myfree(state->org_tid); been_here_free(state->dups); if (state->reason) myfree(state->reason); diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/global/Makefile.in ./src/global/Makefile.in *** /var/tmp/postfix-2.5-20070501/src/global/Makefile.in Tue May 1 14:20:45 2007 --- ./src/global/Makefile.in Mon May 7 20:55:40 2007 *************** *** 28,34 **** tok822_resolve.c tok822_rewrite.c tok822_tree.c trace.c \ user_acl.c valid_mailhost_addr.c verify.c verify_clnt.c \ verp_sender.c wildcard_inet_addr.c xtext.c delivered_hdr.c \ ! fold_addr.c OBJS = abounce.o anvil_clnt.o been_here.o bounce.o bounce_log.o \ canon_addr.o cfg_parser.o cleanup_strerror.o cleanup_strflags.o \ clnt_stream.o conv_time.o db_common.o debug_peer.o debug_process.o \ --- 28,34 ---- tok822_resolve.c tok822_rewrite.c tok822_tree.c trace.c \ user_acl.c valid_mailhost_addr.c verify.c verify_clnt.c \ verp_sender.c wildcard_inet_addr.c xtext.c delivered_hdr.c \ ! fold_addr.c rcpt_key.c OBJS = abounce.o anvil_clnt.o been_here.o bounce.o bounce_log.o \ canon_addr.o cfg_parser.o cleanup_strerror.o cleanup_strflags.o \ clnt_stream.o conv_time.o db_common.o debug_peer.o debug_process.o \ *************** *** 58,64 **** tok822_resolve.o tok822_rewrite.o tok822_tree.o trace.o \ user_acl.o valid_mailhost_addr.o verify.o verify_clnt.o \ verp_sender.o wildcard_inet_addr.o xtext.o delivered_hdr.o \ ! fold_addr.o HDRS = abounce.h anvil_clnt.h been_here.h bounce.h bounce_log.h \ canon_addr.h cfg_parser.h cleanup_user.h clnt_stream.h config.h \ conv_time.h db_common.h debug_peer.h debug_process.h defer.h \ --- 58,64 ---- tok822_resolve.o tok822_rewrite.o tok822_tree.o trace.o \ user_acl.o valid_mailhost_addr.o verify.o verify_clnt.o \ verp_sender.o wildcard_inet_addr.o xtext.o delivered_hdr.o \ ! fold_addr.o rcpt_key.o HDRS = abounce.h anvil_clnt.h been_here.h bounce.h bounce_log.h \ canon_addr.h cfg_parser.h cleanup_user.h clnt_stream.h config.h \ conv_time.h db_common.h debug_peer.h debug_process.h defer.h \ *************** *** 82,88 **** string_list.h strip_addr.h sys_exits.h timed_ipc.h tok822.h \ trace.h user_acl.h valid_mailhost_addr.h verify.h verify_clnt.h \ verp_sender.h wildcard_inet_addr.h xtext.h delivered_hdr.h \ ! fold_addr.h TESTSRC = rec2stream.c stream2rec.c recdump.c DEFS = -I. -I$(INC_DIR) -D$(SYSTYPE) CFLAGS = $(DEBUG) $(OPT) $(DEFS) --- 82,88 ---- string_list.h strip_addr.h sys_exits.h timed_ipc.h tok822.h \ trace.h user_acl.h valid_mailhost_addr.h verify.h verify_clnt.h \ verp_sender.h wildcard_inet_addr.h xtext.h delivered_hdr.h \ ! fold_addr.h rcpt_key.h TESTSRC = rec2stream.c stream2rec.c recdump.c DEFS = -I. -I$(INC_DIR) -D$(SYSTYPE) CFLAGS = $(DEBUG) $(OPT) $(DEFS) *************** *** 1525,1530 **** --- 1525,1540 ---- rcpt_buf.o: rcpt_buf.c rcpt_buf.o: rcpt_buf.h rcpt_buf.o: recipient_list.h + rcpt_key.o: ../../include/msg.h + rcpt_key.o: ../../include/name_code.h + rcpt_key.o: ../../include/stringops.h + rcpt_key.o: ../../include/sys_defs.h + rcpt_key.o: ../../include/vbuf.h + rcpt_key.o: ../../include/vstring.h + rcpt_key.o: mail_params.h + rcpt_key.o: rcpt_key.c + rcpt_key.o: rcpt_key.h + rcpt_key.o: recipient_list.h rcpt_print.o: ../../include/attr.h rcpt_print.o: ../../include/iostuff.h rcpt_print.o: ../../include/sys_defs.h diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/global/deliver_pass.c ./src/global/deliver_pass.c *** /var/tmp/postfix-2.5-20070501/src/global/deliver_pass.c Mon Jun 5 20:47:38 2006 --- ./src/global/deliver_pass.c Mon May 7 10:11:52 2007 *************** *** 101,106 **** --- 101,107 ---- ATTR_TYPE_INT, MAIL_ATTR_FLAGS, request->flags, ATTR_TYPE_STR, MAIL_ATTR_QUEUE, request->queue_name, ATTR_TYPE_STR, MAIL_ATTR_QUEUEID, request->queue_id, + ATTR_TYPE_STR, MAIL_ATTR_ORG_TID, request->org_tid, ATTR_TYPE_LONG, MAIL_ATTR_OFFSET, request->data_offset, ATTR_TYPE_LONG, MAIL_ATTR_SIZE, request->data_size, ATTR_TYPE_STR, MAIL_ATTR_NEXTHOP, nexthop, diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/global/deliver_request.c ./src/global/deliver_request.c *** /var/tmp/postfix-2.5-20070501/src/global/deliver_request.c Thu Jun 15 14:07:15 2006 --- ./src/global/deliver_request.c Mon May 7 11:36:56 2007 *************** *** 12,17 **** --- 12,18 ---- /* int flags; /* char *queue_name; /* char *queue_id; + /* char *org_tid; /* long data_offset; /* long data_size; /* char *nexthop; *************** *** 184,189 **** --- 185,191 ---- struct stat st; static VSTRING *queue_name; static VSTRING *queue_id; + static VSTRING *org_tid; static VSTRING *nexthop; static VSTRING *encoding; static VSTRING *address; *************** *** 208,213 **** --- 210,216 ---- if (queue_name == 0) { queue_name = vstring_alloc(10); queue_id = vstring_alloc(10); + org_tid = vstring_alloc(10); nexthop = vstring_alloc(10); encoding = vstring_alloc(10); address = vstring_alloc(10); *************** *** 231,236 **** --- 234,240 ---- ATTR_TYPE_INT, MAIL_ATTR_FLAGS, &request->flags, ATTR_TYPE_STR, MAIL_ATTR_QUEUE, queue_name, ATTR_TYPE_STR, MAIL_ATTR_QUEUEID, queue_id, + ATTR_TYPE_STR, MAIL_ATTR_ORG_TID, org_tid, ATTR_TYPE_LONG, MAIL_ATTR_OFFSET, &request->data_offset, ATTR_TYPE_LONG, MAIL_ATTR_SIZE, &request->data_size, ATTR_TYPE_STR, MAIL_ATTR_NEXTHOP, nexthop, *************** *** 248,254 **** ATTR_TYPE_STR, MAIL_ATTR_SASL_SENDER, sasl_sender, ATTR_TYPE_STR, MAIL_ATTR_RWR_CONTEXT, rewrite_context, ATTR_TYPE_INT, MAIL_ATTR_RCPT_COUNT, &rcpt_count, ! ATTR_TYPE_END) != 20) { msg_warn("%s: error receiving common attributes", myname); return (-1); } --- 252,258 ---- ATTR_TYPE_STR, MAIL_ATTR_SASL_SENDER, sasl_sender, ATTR_TYPE_STR, MAIL_ATTR_RWR_CONTEXT, rewrite_context, ATTR_TYPE_INT, MAIL_ATTR_RCPT_COUNT, &rcpt_count, ! ATTR_TYPE_END) != 21) { msg_warn("%s: error receiving common attributes", myname); return (-1); } *************** *** 262,267 **** --- 266,272 ---- request->queue_name = mystrdup(vstring_str(queue_name)); request->queue_id = mystrdup(vstring_str(queue_id)); + request->org_tid = mystrdup(vstring_str(org_tid)); request->nexthop = mystrdup(vstring_str(nexthop)); request->encoding = mystrdup(vstring_str(encoding)); request->sender = mystrdup(vstring_str(address)); diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/global/deliver_request.h ./src/global/deliver_request.h *** /var/tmp/postfix-2.5-20070501/src/global/deliver_request.h Tue Feb 21 14:26:59 2006 --- ./src/global/deliver_request.h Mon May 7 10:12:11 2007 *************** *** 32,37 **** --- 32,38 ---- int flags; /* see below */ char *queue_name; /* message queue name */ char *queue_id; /* message queue id */ + char *org_tid; /* original transaction */ long data_offset; /* offset to message */ long data_size; /* message size */ char *nexthop; /* next hop name */ diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/global/mail_params.h ./src/global/mail_params.h *** /var/tmp/postfix-2.5-20070501/src/global/mail_params.h Tue May 1 10:19:27 2007 --- ./src/global/mail_params.h Tue May 8 10:59:37 2007 *************** *** 590,595 **** --- 590,605 ---- #define DEF_DUP_FILTER_LIMIT 1000 extern int var_dup_filter_limit; + #define VAR_QMGR_DUPF_LIMIT "global_duplicate_filter_limit" + #define DEF_QMGR_DUPF_LIMIT 0 + extern int var_qmgr_dupf_limit; + + #define DUP_FILTER_STYLE_STRICT "strict" + #define DUP_FILTER_STYLE_PRAGMA "pragmatic" + #define VAR_DUP_FILTER_STYLE "duplicate_filter_style" + #define DEF_DUP_FILTER_STYLE DUP_FILTER_STYLE_STRICT + extern char *var_dup_filter_style; + #define VAR_TLS_RAND_EXCH_NAME "tls_random_exchange_name" #define DEF_TLS_RAND_EXCH_NAME "${config_directory}/prng_exch" extern char *var_tls_rand_exch_name; diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/global/mail_proto.h ./src/global/mail_proto.h *** /var/tmp/postfix-2.5-20070501/src/global/mail_proto.h Wed Nov 1 15:03:12 2006 --- ./src/global/mail_proto.h Mon May 7 10:28:46 2007 *************** *** 143,148 **** --- 143,154 ---- #define MAIL_ATTR_CRYPTO_KEYSIZE "encryption_keysize" /* + * Attributes for duplicate elimination. + */ + #define MAIL_ATTR_ACT_TID "actual_tid" + #define MAIL_ATTR_ORG_TID "original_tid" + + /* * Suffixes for sender_name, sender_domain etc. */ #define MAIL_ATTR_S_NAME "_name" diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/global/rcpt_key.c ./src/global/rcpt_key.c *** /var/tmp/postfix-2.5-20070501/src/global/rcpt_key.c Wed Dec 31 19:00:00 1969 --- ./src/global/rcpt_key.c Tue May 8 13:13:01 2007 *************** *** 0 **** --- 1,133 ---- + /*++ + /* NAME + /* rcpt_key 3 + /* SUMMARY + /* format recipient lookup key + /* SYNOPSIS + /* #include + /* + /* VSTRING *rcpt_key_format(buf, rcpt, flags) + /* VSTRING *buf; + /* RECIPIENT *rcpt; + /* int flags; + /* + /* int rcpt_key_style(name, value) + /* const char *name; + /* const char *value; + /* DESCRIPTION + /* rcpt_key_format() formats a recipient, typically for use in + /* recipient duplicate filtering. + /* + /* rcpt_key_style() converts the specified style to a bit-mask. + /* + /* Arguments: + /* .IP buf + /* Storage for the result, and the function result value. + /* .IP rcpt + /* Recipient information. + /* .IP flags + /* The bit-wise OR of zero or more of the following: + /* .RS + /* .IP RCPT_KEY_FLAG_APPEND + /* Do not truncate the buffer to zero length before updating. + /* .IP RCPT_KEY_FLAG_FOLD + /* Case fold the result. + /* .IP RCPT_KEY_FLAG_ADDR + /* Include the recipient address in the result. + /* This attribute is required. + /* .IP RCPT_KEY_FLAG_X_ORIG + /* Include the x-original-to address in the result. + /* .IP RCPT_KEY_FLAG_DSN_ORIG + /* Include the DSN original address in the result. + /* .IP RCPT_KEY_FLAG_DSN_NOTIFY + /* Include the DSN NOTIFY options in the result. + /* .RE + /* .IP name + /* Context for error messages. + /* .IP value + /* Recipient lookup key formatting style: "strict" or "pragmatic". + /* DIAGNOSTICS + /* Fatal: incorrect formatting style; out of memory. + /* SEE ALSO + /* rcpt_cache(3) recipient duplicate detector + /* LICENSE + /* .ad + /* .fi + /* The Secure Mailer license must be distributed with this software. + /* AUTHOR(S) + /* Wietse Venema + /* IBM T.J. Watson Research + /* P.O. Box 704 + /* Yorktown Heights, NY 10598, USA + /*--*/ + + /* System library. */ + + #include + #include + + #ifdef STRCASECMP_IN_STRINGS_H + #include + #endif + + /* Utility library. */ + + #include + #include + #include + + /* Global library. */ + + #include + #include + + /* rcpt_key_format - format recipient lookup key */ + + VSTRING *rcpt_key_format(VSTRING *buf, RECIPIENT *rcpt, int flags) + { + int (*cmp) (const char *, const char *); + + cmp = (flags & RCPT_KEY_FLAG_FOLD) ? strcasecmp : strcmp; + + /* + * To save some memory, represent a repeated string by a control + * character; \1 means the string was the same as the first string + * attribute. + */ + #define RCPT_KEY_ADDR(str) \ + ((str) == 0 ? "" : cmp(rcpt->address, (str)) == 0 ? "\1" : (str)) + + if ((flags & RCPT_KEY_FLAG_APPEND) == 0) + VSTRING_RESET(buf); + if (flags & RCPT_KEY_FLAG_ADDR) + vstring_sprintf_append(buf, "%s\n", rcpt->address); + else + msg_panic("rcpt_key_format: recipient address attribute not requested"); + if (flags & RCPT_KEY_FLAG_X_ORIG) + vstring_sprintf_append(buf, "%s\n", RCPT_KEY_ADDR(rcpt->orig_addr)); + if (flags & RCPT_KEY_FLAG_DSN_NOTIFY) + vstring_sprintf_append(buf, "%d\n", rcpt->dsn_notify); + if (flags & RCPT_KEY_FLAG_DSN_ORIG) + vstring_sprintf_append(buf, "%s\n", RCPT_KEY_ADDR(rcpt->dsn_orcpt)); + if (flags & RCPT_KEY_FLAG_FOLD) + lowercase(vstring_str(buf)); + + return (buf); + } + + /* rcpt_key_style - parse recipient lookup key style */ + + int rcpt_key_style(const char *name, const char *value) + { + static NAME_CODE styles[] = { + DUP_FILTER_STYLE_STRICT, RCPT_KEY_FLAG_STRICT, + DUP_FILTER_STYLE_PRAGMA, RCPT_KEY_FLAG_PRAGMA, + 0, 0, + }; + int style; + + if ((style = name_code(styles, NAME_CODE_FLAG_NONE, value)) == 0) + msg_fatal("%s: bad duplicate filter style: %s", + name, value); + return (style); + } diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/global/rcpt_key.h ./src/global/rcpt_key.h *** /var/tmp/postfix-2.5-20070501/src/global/rcpt_key.h Wed Dec 31 19:00:00 1969 --- ./src/global/rcpt_key.h Mon May 7 18:13:27 2007 *************** *** 0 **** --- 1,54 ---- + #ifndef _RCPT_KEY_H_INCLUDED_ + #define _RCPT_KEY_H_INCLUDED_ + + /*++ + /* NAME + /* rcpt_key 5 + /* SUMMARY + /* recipient lookup key formatter + /* SYNOPSIS + /* #include + /* DESCRIPTION + /* .nf + + /* + * Utility library. + */ + #include + + /* + * Global library. + */ + #include + + /* + * External interface. + */ + #define RCPT_KEY_FLAG_APPEND (1<<0) + #define RCPT_KEY_FLAG_FOLD (1<<1) + #define RCPT_KEY_FLAG_ADDR (1<<2) + #define RCPT_KEY_FLAG_X_ORIG (1<<3) + #define RCPT_KEY_FLAG_DSN_ORIG (1<<4) + #define RCPT_KEY_FLAG_DSN_NOTIFY (1<<5) + + #define RCPT_KEY_FLAG_STRICT (RCPT_KEY_FLAG_X_ORIG | \ + RCPT_KEY_FLAG_DSN_ORIG | \ + RCPT_KEY_FLAG_DSN_NOTIFY | \ + RCPT_KEY_FLAG_ADDR) + #define RCPT_KEY_FLAG_PRAGMA (RCPT_KEY_FLAG_ADDR) + + extern VSTRING *rcpt_key_format(VSTRING *, RECIPIENT *, int); + extern int rcpt_key_style(const char *, const char *); + + /* LICENSE + /* .ad + /* .fi + /* The Secure Mailer license must be distributed with this software. + /* AUTHOR(S) + /* Wietse Venema + /* IBM T.J. Watson Research + /* P.O. Box 704 + /* Yorktown Heights, NY 10598, USA + /*--*/ + + #endif diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/global/rec_attr_map.c ./src/global/rec_attr_map.c *** /var/tmp/postfix-2.5-20070501/src/global/rec_attr_map.c Mon Mar 13 18:12:50 2006 --- ./src/global/rec_attr_map.c Mon May 7 10:15:50 2007 *************** *** 48,53 **** --- 48,57 ---- return (REC_TYPE_DSN_RET); } else if (strcmp(attr_name, MAIL_ATTR_CREATE_TIME) == 0) { return (REC_TYPE_CTIME); + } else if (strcmp(attr_name, MAIL_ATTR_ACT_TID) == 0) { + return (REC_TYPE_ATID); + } else if (strcmp(attr_name, MAIL_ATTR_ORG_TID) == 0) { + return (REC_TYPE_OTID); } else { return (0); } diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/global/rec_type.h ./src/global/rec_type.h *** /var/tmp/postfix-2.5-20070501/src/global/rec_type.h Tue Mar 20 11:25:10 2007 --- ./src/global/rec_type.h Mon May 7 10:15:37 2007 *************** *** 36,41 **** --- 36,43 ---- */ #define REC_TYPE_SIZE 'C' /* first record, created by cleanup */ #define REC_TYPE_TIME 'T' /* arrival time, required */ + #define REC_TYPE_ATID '1' /* actual transaction id, optional */ + #define REC_TYPE_OTID '2' /* original transaction id, optional */ #define REC_TYPE_CTIME 'c' /* create time, optional */ #define REC_TYPE_FULL 'F' /* full name, optional */ #define REC_TYPE_INSP 'I' /* inspector transport */ *************** *** 103,109 **** * Note: REC_TYPE_FILT and REC_TYPE_CONT are encoded with the same 'L' * constant, and it is too late to change that now. */ ! #define REC_TYPE_ENVELOPE "MCTcFILSDRO/WVA>KKon" --- 105,111 ---- * Note: REC_TYPE_FILT and REC_TYPE_CONT are encoded with the same 'L' * constant, and it is too late to change that now. */ ! #define REC_TYPE_ENVELOPE "MCT12cFILSDRO/WVA>KKon" diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/local/forward.c ./src/local/forward.c *** /var/tmp/postfix-2.5-20070501/src/local/forward.c Mon Jun 26 08:59:19 2006 --- ./src/local/forward.c Mon May 7 10:27:14 2007 *************** *** 151,160 **** --- 151,166 ---- /* * Send initial message envelope information. For bounces, set the * designated sender: mailing list owner, posting user, whatever. + * + * Forward the original transaction ID, to enable duplicate recipient + * elimination. */ rec_fprintf(cleanup, REC_TYPE_TIME, REC_TYPE_TIME_FORMAT, REC_TYPE_TIME_ARG(info->posting_time)); rec_fputs(cleanup, REC_TYPE_FROM, sender); + if (request->org_tid[0]) + rec_fprintf(cleanup, REC_TYPE_ATTR, "%s=%s", + MAIL_ATTR_ORG_TID, request->org_tid); /* * Don't send the original envelope ID or full/headers return mask if it diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/qmgr/Makefile.in ./src/qmgr/Makefile.in *** /var/tmp/postfix-2.5-20070501/src/qmgr/Makefile.in Sat Mar 17 13:51:43 2007 --- ./src/qmgr/Makefile.in Tue May 8 11:04:38 2007 *************** *** 2,12 **** SRCS = qmgr.c qmgr_active.c qmgr_transport.c qmgr_queue.c qmgr_entry.c \ qmgr_message.c qmgr_deliver.c qmgr_move.c \ qmgr_job.c qmgr_peer.c \ ! qmgr_defer.c qmgr_enable.c qmgr_scan.c qmgr_bounce.c qmgr_error.c OBJS = qmgr.o qmgr_active.o qmgr_transport.o qmgr_queue.o qmgr_entry.o \ qmgr_message.o qmgr_deliver.o qmgr_move.o \ qmgr_job.o qmgr_peer.o \ ! qmgr_defer.o qmgr_enable.o qmgr_scan.o qmgr_bounce.o qmgr_error.o HDRS = qmgr.h TESTSRC = DEFS = -I. -I$(INC_DIR) -D$(SYSTYPE) --- 2,14 ---- SRCS = qmgr.c qmgr_active.c qmgr_transport.c qmgr_queue.c qmgr_entry.c \ qmgr_message.c qmgr_deliver.c qmgr_move.c \ qmgr_job.c qmgr_peer.c \ ! qmgr_defer.c qmgr_enable.c qmgr_scan.c qmgr_bounce.c qmgr_error.c \ ! qmgr_dup_rcpt.c qmgr_drop.c OBJS = qmgr.o qmgr_active.o qmgr_transport.o qmgr_queue.o qmgr_entry.o \ qmgr_message.o qmgr_deliver.o qmgr_move.o \ qmgr_job.o qmgr_peer.o \ ! qmgr_defer.o qmgr_enable.o qmgr_scan.o qmgr_bounce.o qmgr_error.o \ ! qmgr_dup_rcpt.o qmgr_drop.o HDRS = qmgr.h TESTSRC = DEFS = -I. -I$(INC_DIR) -D$(SYSTYPE) *************** *** 173,178 **** --- 175,207 ---- qmgr_deliver.o: ../../include/vstring_vstream.h qmgr_deliver.o: qmgr.h qmgr_deliver.o: qmgr_deliver.c + qmgr_drop.o: ../../include/attr.h + qmgr_drop.o: ../../include/deliver_completed.h + qmgr_drop.o: ../../include/dsn.h + qmgr_drop.o: ../../include/log_adhoc.h + qmgr_drop.o: ../../include/msg_stats.h + qmgr_drop.o: ../../include/recipient_list.h + qmgr_drop.o: ../../include/scan_dir.h + qmgr_drop.o: ../../include/sys_defs.h + qmgr_drop.o: ../../include/vbuf.h + qmgr_drop.o: ../../include/vstream.h + qmgr_drop.o: qmgr.h + qmgr_drop.o: qmgr_drop.c + qmgr_dup_rcpt.o: ../../include/ctable.h + qmgr_dup_rcpt.o: ../../include/dsn.h + qmgr_dup_rcpt.o: ../../include/mail_params.h + qmgr_dup_rcpt.o: ../../include/msg.h + qmgr_dup_rcpt.o: ../../include/mymalloc.h + qmgr_dup_rcpt.o: ../../include/rcpt_key.h + qmgr_dup_rcpt.o: ../../include/recipient_list.h + qmgr_dup_rcpt.o: ../../include/scan_dir.h + qmgr_dup_rcpt.o: ../../include/sys_defs.h + qmgr_dup_rcpt.o: ../../include/tcache.h + qmgr_dup_rcpt.o: ../../include/vbuf.h + qmgr_dup_rcpt.o: ../../include/vstream.h + qmgr_dup_rcpt.o: ../../include/vstring.h + qmgr_dup_rcpt.o: qmgr.h + qmgr_dup_rcpt.o: qmgr_dup_rcpt.c qmgr_enable.o: ../../include/dsn.h qmgr_enable.o: ../../include/msg.h qmgr_enable.o: ../../include/recipient_list.h *************** *** 225,231 **** qmgr_message.o: ../../include/attr.h qmgr_message.o: ../../include/bounce.h qmgr_message.o: ../../include/canon_addr.h - qmgr_message.o: ../../include/deliver_completed.h qmgr_message.o: ../../include/deliver_request.h qmgr_message.o: ../../include/dict.h qmgr_message.o: ../../include/dsn.h --- 254,259 ---- diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/qmgr/qmgr.c ./src/qmgr/qmgr.c *** /var/tmp/postfix-2.5-20070501/src/qmgr/qmgr.c Sat Mar 17 13:59:38 2007 --- ./src/qmgr/qmgr.c Tue May 8 11:09:45 2007 *************** *** 238,252 **** /* OTHER RESOURCE AND RATE CONTROLS /* .ad /* .fi ! /* .IP "\fBminimal_backoff_time (version dependent)\fR" ! /* The minimal time between attempts to deliver a deferred message. /* .IP "\fBmaximal_backoff_time (4000s)\fR" /* The maximal time between attempts to deliver a deferred message. /* .IP "\fBmaximal_queue_lifetime (5d)\fR" /* The maximal time a message is queued before it is sent back as /* undeliverable. ! /* .IP "\fBqueue_run_delay (version dependent)\fR" ! /* The time between deferred queue scans by the queue manager. /* .IP "\fBtransport_retry_time (60s)\fR" /* The time between attempts by the Postfix queue manager to contact /* a malfunctioning message delivery transport. --- 238,254 ---- /* OTHER RESOURCE AND RATE CONTROLS /* .ad /* .fi ! /* .IP "\fBminimal_backoff_time (300s)\fR" ! /* The minimal time between attempts to deliver a deferred message; ! /* prior to Postfix 2.4 the default value was 1000s. /* .IP "\fBmaximal_backoff_time (4000s)\fR" /* The maximal time between attempts to deliver a deferred message. /* .IP "\fBmaximal_queue_lifetime (5d)\fR" /* The maximal time a message is queued before it is sent back as /* undeliverable. ! /* .IP "\fBqueue_run_delay (300s)\fR" ! /* The time between deferred queue scans by the queue manager; ! /* prior to Postfix 2.4 the default value was 1000s. /* .IP "\fBtransport_retry_time (60s)\fR" /* The time between attempts by the Postfix queue manager to contact /* a malfunctioning message delivery transport. *************** *** 255,260 **** --- 257,269 ---- /* .IP "\fBbounce_queue_lifetime (5d)\fR" /* The maximal time a bounce message is queued before it is considered /* undeliverable. + /* .PP + /* Available in Postfix version 2.5 and later: + /* .IP "\fBglobal_duplicate_filter_limit (0)\fR" + /* The size of the filter for duplicate deliveries to recipients + /* that are listed under multiple \fBlocal\fR(8) aliases. + /* .IP "\fBduplicate_filter_style (strict)\fR" + /* The duplicate recipient filter policy: strict or pragmatic. /* MISCELLANEOUS CONTROLS /* .ad /* .fi *************** *** 390,395 **** --- 399,406 ---- int var_proc_limit; bool var_verp_bounce_off; int var_qmgr_clog_warn_time; + int var_qmgr_dupf_limit; + char *var_dup_filter_style; static QMGR_SCAN *qmgr_scans[2]; *************** *** 614,619 **** --- 625,648 ---- qmgr_scans[QMGR_SCAN_IDX_DEFERRED] = qmgr_scan_create(MAIL_QUEUE_DEFERRED); qmgr_scan_request(qmgr_scans[QMGR_SCAN_IDX_INCOMING], QMGR_SCAN_START); qmgr_deferred_run_event(0, (char *) 0); + + /* + * Initialize the duplicate recipient filter. + */ + if (var_qmgr_dupf_limit > 0) + qmgr_dup_rcpt_init(); + } + + /* pre_exit - shutdown callback */ + + static void pre_exit(char *name, char **unused_argv) + { + + /* + * Facilitate memory leak tests. + */ + if (var_qmgr_dupf_limit > 0) + qmgr_dup_rcpt_free(); } MAIL_VERSION_STAMP_DECLARE; *************** *** 624,629 **** --- 653,659 ---- { static CONFIG_STR_TABLE str_table[] = { VAR_DEFER_XPORTS, DEF_DEFER_XPORTS, &var_defer_xports, 0, 0, + VAR_DUP_FILTER_STYLE, DEF_DUP_FILTER_STYLE, &var_dup_filter_style, 1, 0, 0, }; static CONFIG_TIME_TABLE time_table[] = { *************** *** 654,659 **** --- 684,690 ---- VAR_LOCAL_RCPT_LIMIT, DEF_LOCAL_RCPT_LIMIT, &var_local_rcpt_lim, 0, 0, VAR_LOCAL_CON_LIMIT, DEF_LOCAL_CON_LIMIT, &var_local_con_lim, 0, 0, VAR_PROC_LIMIT, DEF_PROC_LIMIT, &var_proc_limit, 1, 0, + VAR_QMGR_DUPF_LIMIT, DEF_QMGR_DUPF_LIMIT, &var_qmgr_dupf_limit, 0, 0, 0, }; static CONFIG_BOOL_TABLE bool_table[] = { *************** *** 682,686 **** --- 713,718 ---- MAIL_SERVER_LOOP, qmgr_loop, MAIL_SERVER_PRE_ACCEPT, pre_accept, MAIL_SERVER_SOLITARY, + MAIL_SERVER_EXIT, pre_exit, 0); } diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/qmgr/qmgr.h ./src/qmgr/qmgr.h *** /var/tmp/postfix-2.5-20070501/src/qmgr/qmgr.h Tue Jan 16 14:25:10 2007 --- ./src/qmgr/qmgr.h Tue May 8 08:37:14 2007 *************** *** 268,273 **** --- 268,275 ---- char *queue_name; /* queue name */ char *queue_id; /* queue file */ char *encoding; /* content encoding */ + char *act_tid; /* actual transaction id */ + char *org_tid; /* original transaction id */ char *sender; /* complete address */ char *dsn_envid; /* DSN envelope ID */ int dsn_ret; /* DSN headers/full */ *************** *** 443,448 **** --- 445,462 ---- extern QMGR_TRANSPORT *qmgr_error_transport(const char *); extern QMGR_QUEUE *qmgr_error_queue(const char *, DSN *); extern char *qmgr_error_nexthop(DSN *); + + /* + * qmgr_dup_rcpt.c + */ + extern void qmgr_dup_rcpt_init(void); + extern int qmgr_dup_rcpt_detect(QMGR_MESSAGE *, const char *, const char *, RECIPIENT *); + extern void qmgr_dup_rcpt_free(void); + + /* + * qmgr_drop.c + */ + extern void qmgr_drop_recipient(QMGR_MESSAGE *, RECIPIENT *, const char *); /* LICENSE /* .ad diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/qmgr/qmgr_deliver.c ./src/qmgr/qmgr_deliver.c *** /var/tmp/postfix-2.5-20070501/src/qmgr/qmgr_deliver.c Tue Jan 16 14:25:10 2007 --- ./src/qmgr/qmgr_deliver.c Mon May 7 10:12:48 2007 *************** *** 162,167 **** --- 162,168 ---- ATTR_TYPE_INT, MAIL_ATTR_FLAGS, flags, ATTR_TYPE_STR, MAIL_ATTR_QUEUE, message->queue_name, ATTR_TYPE_STR, MAIL_ATTR_QUEUEID, message->queue_id, + ATTR_TYPE_STR, MAIL_ATTR_ORG_TID, message->org_tid, ATTR_TYPE_LONG, MAIL_ATTR_OFFSET, message->data_offset, ATTR_TYPE_LONG, MAIL_ATTR_SIZE, message->cont_length, ATTR_TYPE_STR, MAIL_ATTR_NEXTHOP, entry->queue->nexthop, diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/qmgr/qmgr_drop.c ./src/qmgr/qmgr_drop.c *** /var/tmp/postfix-2.5-20070501/src/qmgr/qmgr_drop.c Wed Dec 31 19:00:00 1969 --- ./src/qmgr/qmgr_drop.c Mon May 7 17:49:22 2007 *************** *** 0 **** --- 1,65 ---- + /*++ + /* NAME + /* qmgr_drop + /* SUMMARY + /* deal with mail that must be ignored + /* SYNOPSIS + /* #include "qmgr.h" + /* + /* void qmgr_drop_recipient(message, recipient, text) + /* QMGR_MESSAGE *message; + /* RECIPIENT *recipient; + /* const char *text; + /* DESCRIPTION + /* qmgr_drop_recipient() logs a "skipped" status record with + /* descriptive text, and marks the recipient as "done" in the + /* queue file. Unlike other operations, this does not show up + /* in verbose deliveries, nor does it produce "success" delivery + /* status notifications. + /* + /* Arguments: + /* .IP message + /* Mail delivery context. + /* .IP recipient + /* Recipient information; used for logging purposes, and for + /* marking the recipient as "done" in the queue file. + /* .IP text + /* Descriptive text for the logfile. + /* DIAGNOSTICS + /* Panic: consistency check failure. Fatal: out of memory. + /* LICENSE + /* .ad + /* .fi + /* The Secure Mailer license must be distributed with this software. + /* AUTHOR(S) + /* Wietse Venema + /* IBM T.J. Watson Research + /* P.O. Box 704 + /* Yorktown Heights, NY 10598, USA + /*--*/ + + /* System library. */ + + #include + + /* Global library. */ + + #include + #include + #include + + /* Application-specific. */ + + #include + + void qmgr_drop_recipient(QMGR_MESSAGE *message, RECIPIENT *rcpt, + const char *text) + { + MSG_STATS stats; + DSN dsn; + + log_adhoc(message->queue_id, + QMGR_MSG_STATS(&stats, message), rcpt, "none", + DSN_SIMPLE(&dsn, "2.0.0", text), "skipped"); + deliver_completed(message->fp, rcpt->offset); + } diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/qmgr/qmgr_dup_rcpt.c ./src/qmgr/qmgr_dup_rcpt.c *** /var/tmp/postfix-2.5-20070501/src/qmgr/qmgr_dup_rcpt.c Wed Dec 31 19:00:00 1969 --- ./src/qmgr/qmgr_dup_rcpt.c Tue May 8 14:41:05 2007 *************** *** 0 **** --- 1,183 ---- + /*++ + /* NAME + /* qmgr_dup_rcpt 3 + /* SUMMARY + /* duplcate recipient detection + /* SYNOPSIS + /* #include + /* + /* void qmgr_dup_rcpt_init() + /* + /* void qmgr_dup_rcpt_detect(message, transport, nexthop, rcpt) + /* QMGR_MESSAGE *message; + /* const char *transport; + /* const char *nexthop; + /* RECIPIENT *rcpt; + /* + /* void qmgr_dup_rcpt_free() + /* DESCRIPTION + /* qmgr_dup_rcpt_init() initializes the global recipient + /* duplicate filter. The filter size is controlled with the + /* qmgr_duplicate_filter_size configuration parameter. Duplicate + /* detection can be approximate or exact, as controlled with + /* the duplicate_filter_style configuration parameter. + /* + /* qmgr_dup_rcpt_detect() returns TRUE when the specified + /* recipient is deemed a duplicate. + /* + /* qmgr_dup_rcpt_free() destroys the global recipient duplicate + /* filter. This may be called at process exit to facilitate + /* memory leak testing. + /* + /* Arguments: + /* .IP message + /* Message delivery context including the invisible original + /* and actual Postfix transaction IDs. + /* .IP transport + /* Delivery method. + /* .IP nexthop + /* Recipient next-hop destination. + /* .IP rcpt + /* The recipient under test. + /* SEE ALSO + /* tcache(3) trivial cache manager. + /* rcpt_key(3) recipient lookup key formatter. + /* LICENSE + /* .ad + /* .fi + /* The Secure Mailer license must be distributed with this software. + /* AUTHOR(S) + /* Wietse Venema + /* IBM T.J. Watson Research + /* P.O. Box 704 + /* Yorktown Heights, NY 10598, USA + /*--*/ + + /* System library. */ + + #include + #include + + /* Utility library. */ + + #include + #include + #include + + /* Global library. */ + + #include + #include + #include + + /* Application_specific. */ + + #include + + /* + * Duplicate filter control. + */ + static TCACHE *qmgr_dup_rcpt_table; + static int qmgr_rcpt_key_flags = RCPT_KEY_FLAG_FOLD; + + #define STR(x) vstring_str(x) + + /* qmgr_dup_rcpt_init - initialize the duplicate filter */ + + void qmgr_dup_rcpt_init(void) + { + const char *myname = "qmgr_dup_rcpt_init"; + + if (qmgr_dup_rcpt_table) + msg_panic("%s: repeated call", myname); + + if (msg_verbose) + msg_info("%s: global duplicate filter size %d", + myname, var_qmgr_dupf_limit); + + qmgr_dup_rcpt_table = + tcache_create(var_qmgr_dupf_limit, mystrdup, myfree); + + qmgr_rcpt_key_flags |= + rcpt_key_style(VAR_DUP_FILTER_STYLE, var_dup_filter_style); + } + + /* qmgr_dup_rcpt_detect - detect duplicate recipient */ + + int qmgr_dup_rcpt_detect(QMGR_MESSAGE *message, const char *transport, + const char *nexthop, RECIPIENT *rcpt) + { + const char *myname = "qmgr_dup_rcpt_detect"; + const char *cached_tid; + const char *key; + + /* + * Legacy queue files have no information to implement recipient + * duplicate detection. + */ + if (message->act_tid == 0 || *message->act_tid == 0 + || message->org_tid == 0 || *message->org_tid == 0) { + if (msg_verbose) + msg_info("%s: keep: %s (no act_tid or no org_tid)", + myname, rcpt->address); + return (0); + } + + /* + * Recipients are distinguished by one or more recipient attributes: + * original invisible Postfix transaction ID, transport, and next-hop. + * This "flat string" representation optimizes for the worst case, which + * is also the most common case: single-recipient messages and no + * internal forwarding. + */ + rcpt_key_format(qmgr_dup_rcpt_table->key_buf, rcpt, qmgr_rcpt_key_flags); + vstring_sprintf_append(qmgr_dup_rcpt_table->key_buf, "%s\n%s\n%s", + message->org_tid, transport, nexthop); + key = STR(qmgr_dup_rcpt_table->key_buf); + + /* + * Enter a non-existent (recipient + invisible original Postfix + * transaction ID + transport + next-hop) into the cache, together with + * its current actual Postfix transaction ID. This can happen when old + * information is flushed from the cache, or when resolver results change + * between repeated delivery attempts (transport map or rewriting rules). + */ + if ((cached_tid = tcache_lookup(qmgr_dup_rcpt_table, key)) == 0) { + TCACHE_ENTER(qmgr_dup_rcpt_table, key, message->act_tid); + if (msg_verbose) + msg_info("%s: enter %s -> %s", myname, key, message->act_tid); + return (0); + } + + /* + * Declare a duplicate when the same (recipient + invisible original + * Postfix transaction ID + transport + next-hop) are already cached + * together with a different invisible actual Postfix transaction ID + * (i.e. a different queue file). + */ + else if (strcmp(cached_tid, message->act_tid) != 0) { + if (msg_verbose) + msg_info("%s: duplicate %s -> %s (not %s)", + myname, key, cached_tid, message->act_tid); + return (1); + } + + /* + * Assume that this is a repeated message delivery attempt when the same + * (recipient + invisible original Postfix transaction ID + transport + + * nexthop) are still cached under the same actual Postfix transaction + * ID). + */ + else { + if (msg_verbose) + msg_info("%s: keep %s -> %s", myname, key, message->act_tid); + return (0); + } + } + + /* qmgr_dup_rcpt_free - memory leak detector aid */ + + void qmgr_dup_rcpt_free(void) + { + tcache_free(qmgr_dup_rcpt_table); + } diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/qmgr/qmgr_message.c ./src/qmgr/qmgr_message.c *** /var/tmp/postfix-2.5-20070501/src/qmgr/qmgr_message.c Tue Jan 16 14:25:10 2007 --- ./src/qmgr/qmgr_message.c Tue May 8 11:01:46 2007 *************** *** 129,135 **** #include #include #include - #include #include #include #include --- 129,134 ---- *************** *** 176,181 **** --- 175,182 ---- message->queue_id = mystrdup(queue_id); message->queue_name = mystrdup(queue_name); message->encoding = 0; + message->act_tid = 0; + message->org_tid = 0; message->sender = 0; message->dsn_envid = 0; message->dsn_ret = 0; *************** *** 615,620 **** --- 616,629 ---- message->redirect_addr = mystrdup(start); continue; } + if (rec_type == REC_TYPE_ATID) { + if (message->act_tid == 0) + message->act_tid = mystrdup(start); + } + if (rec_type == REC_TYPE_OTID) { + if (message->org_tid == 0) + message->org_tid = mystrdup(start); + } if (rec_type == REC_TYPE_FROM) { if (message->sender == 0) { message->sender = mystrdup(start); *************** *** 985,993 **** char **cpp; char *nexthop; ssize_t len; - int status; - DSN dsn; - MSG_STATS stats; DSN *saved_dsn; #define STREQ(x,y) (strcmp(x,y) == 0) --- 994,999 ---- *************** *** 1079,1084 **** --- 1085,1102 ---- } /* + * Optionally skip duplicate recipients that appear in different + * descendants from the same original Postfix transaction. + */ + if (var_qmgr_dupf_limit > 0 + && qmgr_dup_rcpt_detect(message, STR(reply.transport), + STR(reply.nexthop), recipient) != 0) { + qmgr_drop_recipient(message, recipient, + "duplicate recipient discarded"); + continue; + } + + /* * Discard mail to the local double bounce address here, so this * system can run without a local delivery agent. They'd still have * to configure something for mail directed to the local postmaster, *************** *** 1094,1109 **** if (strncasecmp(STR(reply.recipient), var_double_bounce_sender, len) == 0 && !var_double_bounce_sender[len]) { ! status = sent(message->tflags, message->queue_id, ! QMGR_MSG_STATS(&stats, message), recipient, ! "none", DSN_SIMPLE(&dsn, "2.0.0", ! "undeliverable postmaster notification discarded")); ! if (status == 0) { ! deliver_completed(message->fp, recipient->offset); ! msg_warn("%s: undeliverable postmaster notification discarded", ! message->queue_id); ! } else ! message->flags |= status; continue; } } --- 1112,1122 ---- if (strncasecmp(STR(reply.recipient), var_double_bounce_sender, len) == 0 && !var_double_bounce_sender[len]) { ! qmgr_drop_recipient(message, recipient, ! "undeliverable postmaster notification discarded"); ! msg_warn("%s: %s", ! message->queue_id, ! "undeliverable postmaster notification discarded"); continue; } } *************** *** 1357,1362 **** --- 1370,1379 ---- myfree(message->dsn_envid); if (message->encoding) myfree(message->encoding); + if (message->act_tid) + myfree(message->act_tid); + if (message->org_tid) + myfree(message->org_tid); if (message->sender) myfree(message->sender); if (message->verp_delims) diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/util/Makefile.in ./src/util/Makefile.in *** /var/tmp/postfix-2.5-20070501/src/util/Makefile.in Sat Mar 17 13:51:33 2007 --- ./src/util/Makefile.in Mon May 7 20:55:36 2007 *************** *** 30,36 **** username.c valid_hostname.c vbuf.c vbuf_print.c vstream.c \ vstream_popen.c vstring.c vstring_vstream.c watchdog.c writable.c \ write_buf.c write_wait.c sane_basename.c format_tv.c allspace.c \ ! allascii.c load_file.c killme_after.c OBJS = alldig.o allprint.o argv.o argv_split.o attr_clnt.o attr_print0.o \ attr_print64.o attr_print_plain.o attr_scan0.o attr_scan64.o \ attr_scan_plain.o auto_clnt.o base64_code.o basename.o binhash.o \ --- 30,36 ---- username.c valid_hostname.c vbuf.c vbuf_print.c vstream.c \ vstream_popen.c vstring.c vstring_vstream.c watchdog.c writable.c \ write_buf.c write_wait.c sane_basename.c format_tv.c allspace.c \ ! allascii.c load_file.c killme_after.c tcache.c OBJS = alldig.o allprint.o argv.o argv_split.o attr_clnt.o attr_print0.o \ attr_print64.o attr_print_plain.o attr_scan0.o attr_scan64.o \ attr_scan_plain.o auto_clnt.o base64_code.o basename.o binhash.o \ *************** *** 62,68 **** username.o valid_hostname.o vbuf.o vbuf_print.o vstream.o \ vstream_popen.o vstring.o vstring_vstream.o watchdog.o writable.o \ write_buf.o write_wait.o sane_basename.o format_tv.o allspace.o \ ! allascii.o load_file.o killme_after.o HDRS = argv.h attr.h attr_clnt.h auto_clnt.h base64_code.h binhash.h \ chroot_uid.h cidr_match.h clean_env.h connect.h ctable.h dict.h \ dict_cdb.h dict_cidr.h dict_db.h dict_dbm.h dict_env.h dict_ht.h \ --- 62,68 ---- username.o valid_hostname.o vbuf.o vbuf_print.o vstream.o \ vstream_popen.o vstring.o vstring_vstream.o watchdog.o writable.o \ write_buf.o write_wait.o sane_basename.o format_tv.o allspace.o \ ! allascii.o load_file.o killme_after.o tcache.o HDRS = argv.h attr.h attr_clnt.h auto_clnt.h base64_code.h binhash.h \ chroot_uid.h cidr_match.h clean_env.h connect.h ctable.h dict.h \ dict_cdb.h dict_cidr.h dict_db.h dict_dbm.h dict_env.h dict_ht.h \ *************** *** 81,87 **** sigdelay.h sock_addr.h spawn_command.h split_at.h stat_as.h \ stringops.h sys_defs.h timed_connect.h timed_wait.h trigger.h \ username.h valid_hostname.h vbuf.h vbuf_print.h vstream.h vstring.h \ ! vstring_vstream.h watchdog.h format_tv.h load_file.h killme_after.h TESTSRC = fifo_open.c fifo_rdwr_bug.c fifo_rdonly_bug.c select_bug.c \ stream_test.c dup2_pass_on_exec.c DEFS = -I. -D$(SYSTYPE) --- 81,88 ---- sigdelay.h sock_addr.h spawn_command.h split_at.h stat_as.h \ stringops.h sys_defs.h timed_connect.h timed_wait.h trigger.h \ username.h valid_hostname.h vbuf.h vbuf_print.h vstream.h vstring.h \ ! vstring_vstream.h watchdog.h format_tv.h load_file.h killme_after.h \ ! tcache.h TESTSRC = fifo_open.c fifo_rdwr_bug.c fifo_rdonly_bug.c select_bug.c \ stream_test.c dup2_pass_on_exec.c DEFS = -I. -D$(SYSTYPE) *************** *** 1484,1489 **** --- 1485,1497 ---- sys_compat.o: iostuff.h sys_compat.o: sys_compat.c sys_compat.o: sys_defs.h + tcache.o: ctable.h + tcache.o: mymalloc.h + tcache.o: sys_defs.h + tcache.o: tcache.c + tcache.o: tcache.h + tcache.o: vbuf.h + tcache.o: vstring.h timed_connect.o: iostuff.h timed_connect.o: msg.h timed_connect.o: sane_connect.h diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/util/ctable.c ./src/util/ctable.c *** /var/tmp/postfix-2.5-20070501/src/util/ctable.c Thu Jun 15 14:07:16 2006 --- ./src/util/ctable.c Tue May 8 08:16:49 2007 *************** *** 2,8 **** /* NAME /* ctable 3 /* SUMMARY ! /* cache manager /* SYNOPSIS /* #include /* --- 2,8 ---- /* NAME /* ctable 3 /* SUMMARY ! /* push/pull cache manager /* SYNOPSIS /* #include /* *************** *** 12,17 **** --- 12,21 ---- /* void (*delete)(void *value, void *context); /* void *context; /* + /* const void *ctable_enter(cache, key, value) + /* CTABLE *cache; + /* const char *key; + /* /* const void *ctable_locate(cache, key) /* CTABLE *cache; /* const char *key; *************** *** 33,41 **** --- 37,56 ---- /* specify pointers to call-back functions that create a value, given /* a key, and delete a given value, respectively. The context argument /* is passed on to the call-back routines. + /* Specify CTABLE_NOCREATE or CTABLE_NOCREATE to suppress these + /* call-back actions. + /* + /* ctable_enter() stores the specified value into the cache + /* under the specified key. The result value is the value + /* argument. No attempt is made to detect duplicate cache + /* entries. Use ctable_locate() for that. /* /* ctable_locate() looks up or generates the value that corresponds to /* the specified key, and returns that value. + /* If the value is not present in the cache and the create() + /* call-back is not CTABLE_NOCREATE, that function is called + /* to create the requested value. Otherwise the result is a + /* null pointer. /* /* ctable_free() destroys the specified cache, including its contents. /* *************** *** 119,124 **** --- 134,165 ---- return (cache); } + /* ctable_enter - enter something into the cache */ + + const void *ctable_enter(CTABLE *cache, const char *key, void *value) + { + const char *myname = "ctable_enter"; + CTABLE_ENTRY *entry; + + if (cache->used >= cache->limit) { + entry = RING_TO_CTABLE_ENTRY(ring_pred(RING_PTR_OF(cache))); + if (msg_verbose) + msg_info("%s: purge entry key %s", myname, entry->key); + ring_detach(RING_PTR_OF(entry)); + cache->delete(entry->value, cache->context); + htable_delete(cache->table, entry->key, (void (*) (char *)) 0); + } else { + entry = (CTABLE_ENTRY *) mymalloc(sizeof(CTABLE_ENTRY)); + cache->used++; + } + entry->value = value; + entry->key = htable_enter(cache->table, key, (char *) entry)->key; + ring_append(RING_PTR_OF(cache), RING_PTR_OF(entry)); + if (msg_verbose) + msg_info("%s: install entry key %s", myname, entry->key); + return (value); + } + /* ctable_locate - look up or create cache item */ const void *ctable_locate(CTABLE *cache, const char *key) *************** *** 133,154 **** * All this means that the cache never shrinks. */ if ((entry = (CTABLE_ENTRY *) htable_find(cache->table, key)) == 0) { ! if (cache->used >= cache->limit) { ! entry = RING_TO_CTABLE_ENTRY(ring_pred(RING_PTR_OF(cache))); ! if (msg_verbose) ! msg_info("%s: purge entry key %s", myname, entry->key); ! ring_detach(RING_PTR_OF(entry)); ! cache->delete(entry->value, cache->context); ! htable_delete(cache->table, entry->key, (void (*) (char *)) 0); ! } else { ! entry = (CTABLE_ENTRY *) mymalloc(sizeof(CTABLE_ENTRY)); ! cache->used++; ! } ! entry->value = cache->create(key, cache->context); ! entry->key = htable_enter(cache->table, key, (char *) entry)->key; ! ring_append(RING_PTR_OF(cache), RING_PTR_OF(entry)); ! if (msg_verbose) ! msg_info("%s: install entry key %s", myname, entry->key); } else if (entry == RING_TO_CTABLE_ENTRY(ring_succ(RING_PTR_OF(cache)))) { if (msg_verbose) msg_info("%s: leave existing entry key %s", myname, entry->key); --- 174,184 ---- * All this means that the cache never shrinks. */ if ((entry = (CTABLE_ENTRY *) htable_find(cache->table, key)) == 0) { ! if (cache->create == 0) ! return (0); ! else ! return (ctable_enter(cache, key, ! cache->create(key, cache->context))); } else if (entry == RING_TO_CTABLE_ENTRY(ring_succ(RING_PTR_OF(cache)))) { if (msg_verbose) msg_info("%s: leave existing entry key %s", myname, entry->key); diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/util/ctable.h ./src/util/ctable.h *** /var/tmp/postfix-2.5-20070501/src/util/ctable.h Sun Jul 29 14:31:44 2001 --- ./src/util/ctable.h Mon May 7 11:27:30 2007 *************** *** 20,29 **** --- 20,33 ---- typedef void *(*CTABLE_CREATE_FN) (const char *, void *); typedef void (*CTABLE_DELETE_FN) (void *, void *); + #define CTABLE_NOCREATE ((CTABLE_CREATE_FN) 0) + #define CTABLE_NODELETE ((CTABLE_DELETE_FN) 0) + extern CTABLE *ctable_create(int, CTABLE_CREATE_FN, CTABLE_DELETE_FN, void *); extern void ctable_free(CTABLE *); extern void ctable_walk(CTABLE *, void (*) (const char *, const void *)); extern const void *ctable_locate(CTABLE *, const char *); + extern const void *ctable_enter(CTABLE *, const char *, void *); /* LICENSE /* .ad diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/util/ctable.ref ./src/util/ctable.ref *** /var/tmp/postfix-2.5-20070501/src/util/ctable.ref Sun Jul 29 12:15:20 2001 --- ./src/util/ctable.ref Fri May 4 20:13:29 2007 *************** *** 1,60 **** key = a ask: a = 1 ! ./ctable: ctable_locate: install entry key a result: 1 key = b ask: b = 2 ! ./ctable: ctable_locate: install entry key b result: 2 key = c ask: c = 3 ! ./ctable: ctable_locate: install entry key c result: 3 key = d ask: d = 4 ! ./ctable: ctable_locate: install entry key d result: 4 key = e ask: e = 5 ! ./ctable: ctable_locate: install entry key e result: 5 key = f - ./ctable: ctable_locate: purge entry key a ask: f = 6 ! ./ctable: ctable_locate: install entry key f result: 6 key = f ./ctable: ctable_locate: leave existing entry key f result: 6 key = a - ./ctable: ctable_locate: purge entry key b ask: a = 1 ! ./ctable: ctable_locate: install entry key a result: 1 key = b - ./ctable: ctable_locate: purge entry key c ask: b = 2 ! ./ctable: ctable_locate: install entry key b result: 2 key = c - ./ctable: ctable_locate: purge entry key d ask: c = 3 ! ./ctable: ctable_locate: install entry key c result: 3 key = d - ./ctable: ctable_locate: purge entry key e ask: d = 4 ! ./ctable: ctable_locate: install entry key d result: 4 key = e - ./ctable: ctable_locate: purge entry key f ask: e = 5 ! ./ctable: ctable_locate: install entry key e result: 5 key = f - ./ctable: ctable_locate: purge entry key a ask: f = 6 ! ./ctable: ctable_locate: install entry key f result: 6 key = f ./ctable: ctable_locate: leave existing entry key f --- 1,60 ---- key = a ask: a = 1 ! ./ctable: ctable_enter: install entry key a result: 1 key = b ask: b = 2 ! ./ctable: ctable_enter: install entry key b result: 2 key = c ask: c = 3 ! ./ctable: ctable_enter: install entry key c result: 3 key = d ask: d = 4 ! ./ctable: ctable_enter: install entry key d result: 4 key = e ask: e = 5 ! ./ctable: ctable_enter: install entry key e result: 5 key = f ask: f = 6 ! ./ctable: ctable_enter: purge entry key a ! ./ctable: ctable_enter: install entry key f result: 6 key = f ./ctable: ctable_locate: leave existing entry key f result: 6 key = a ask: a = 1 ! ./ctable: ctable_enter: purge entry key b ! ./ctable: ctable_enter: install entry key a result: 1 key = b ask: b = 2 ! ./ctable: ctable_enter: purge entry key c ! ./ctable: ctable_enter: install entry key b result: 2 key = c ask: c = 3 ! ./ctable: ctable_enter: purge entry key d ! ./ctable: ctable_enter: install entry key c result: 3 key = d ask: d = 4 ! ./ctable: ctable_enter: purge entry key e ! ./ctable: ctable_enter: install entry key d result: 4 key = e ask: e = 5 ! ./ctable: ctable_enter: purge entry key f ! ./ctable: ctable_enter: install entry key e result: 5 key = f ask: f = 6 ! ./ctable: ctable_enter: purge entry key a ! ./ctable: ctable_enter: install entry key f result: 6 key = f ./ctable: ctable_locate: leave existing entry key f *************** *** 72,80 **** ./ctable: ctable_locate: move existing entry key b result: 2 key = a - ./ctable: ctable_locate: purge entry key f ask: a = 1 ! ./ctable: ctable_locate: install entry key a result: 1 key = b ./ctable: ctable_locate: move existing entry key b --- 72,80 ---- ./ctable: ctable_locate: move existing entry key b result: 2 key = a ask: a = 1 ! ./ctable: ctable_enter: purge entry key f ! ./ctable: ctable_enter: install entry key a result: 1 key = b ./ctable: ctable_locate: move existing entry key b *************** *** 89,97 **** ./ctable: ctable_locate: move existing entry key e result: 5 key = f - ./ctable: ctable_locate: purge entry key a ask: f = 6 ! ./ctable: ctable_locate: install entry key f result: 6 key = f ./ctable: ctable_locate: leave existing entry key f --- 89,97 ---- ./ctable: ctable_locate: move existing entry key e result: 5 key = f ask: f = 6 ! ./ctable: ctable_enter: purge entry key a ! ./ctable: ctable_enter: install entry key f result: 6 key = f ./ctable: ctable_locate: leave existing entry key f diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/util/tcache.c ./src/util/tcache.c *** /var/tmp/postfix-2.5-20070501/src/util/tcache.c Wed Dec 31 19:00:00 1969 --- ./src/util/tcache.c Mon May 7 11:27:30 2007 *************** *** 0 **** --- 1,123 ---- + /*++ + /* NAME + /* tcache 3 + /* SUMMARY + /* trivial push-mode cache manager + /* SYNOPSIS + /* #include + /* + /* typedef struct TCACHE { + /* VSTRING *key_buf; + /* /* private members... */ + /* } TCACHE; + /* + /* TCACHE *tcache_create(size, save, delete) + /* int size; + /* char *(*save)(const char *value); + /* void (*delete)(char *value); + /* + /* const char *TCACHE_ENTER(tcache, key, value) + /* TCACHE *tcache; + /* const char *key; + /* const char *value; + /* + /* const char *tcache_lookup(tcache, key) + /* TCACHE *tcache; + /* const char *key; + /* + /* void tcache_free(tcache) + /* TCACHE *tcache; + /* DESCRIPTION + /* This module maintains a trivial push-mode cache, where the + /* caller pushes information into the cache and where the cache + /* manager silently throws information away. + /* + /* tcache_create() creates a trivial cache instance. + /* + /* TCACHE_ENTER() enters a value into the cache under the + /* specified key. No duplicate check is done: use tcache_lookup() + /* for that. + /* + /* tcache_lookup() looks up a value that was stored under the + /* specified key, or a null pointer value. + /* + /* tcache_free() destroys a trivial cache instance. + /* + /* Arguments: + /* .IP size + /* The number of cache entries. Old entries are removed when + /* new entries are added while the cache is full. + /* .IP save + /* Null pointer, or pointer to function that saves a copy of + /* its value argument. + /* .IP delete + /* Null pointer, or pointer to function that deletes a saved + /* value. + /* .IP key + /* Lookup key. For convenience, each TCACHE provides a buffer + /* that the application can (but does not have to) use to + /* save the key for the _lookup and _enter operations. + /* .IP delete + /* A null pointer, or pointer to call-back function that + /* receives as arguments a cached value and a pointer to the + /* TCACHE object. + /* BUGS + /* No tcache_delete() or tcache_update() operations. + /* SEE ALSO + /* rcpt_key(3) recipient lookup key formatter. + /* LICENSE + /* .ad + /* .fi + /* The Secure Mailer license must be distributed with this software. + /* AUTHOR(S) + /* Wietse Venema + /* IBM T.J. Watson Research + /* P.O. Box 704 + /* Yorktown Heights, NY 10598, USA + /*--*/ + + /* System library. */ + + #include + + /* Utility library. */ + + #include + + /* Global library. */ + + #include + + /* tcache_delete_cb - delete call-back adapter */ + + static void tcache_delete_cb(void *value, void *context) + { + TCACHE *tp = (TCACHE *) context; + + tp->delete(value); + } + + /* tcache_create - create trivial cache */ + + TCACHE *tcache_create(int size, TCACHE_SAVE_FN save, TCACHE_DELETE_FN delete) + { + TCACHE *tp; + + tp = (TCACHE *) mymalloc(sizeof(*tp)); + tp->key_buf = vstring_alloc(10); + tp->table = ctable_create(size, CTABLE_NOCREATE, delete ? + tcache_delete_cb : CTABLE_NODELETE, + (void *) tp); + tp->save = save; + tp->delete = delete; + return (tp); + } + + /* tcache_free - destroy trivial cache */ + + void tcache_free(TCACHE *tp) + { + vstring_free(tp->key_buf); + ctable_free(tp->table); + myfree((char *) tp); + } diff -cr --new-file --exclude=.indent.pro --exclude=man --exclude=html /var/tmp/postfix-2.5-20070501/src/util/tcache.h ./src/util/tcache.h *** /var/tmp/postfix-2.5-20070501/src/util/tcache.h Wed Dec 31 19:00:00 1969 --- ./src/util/tcache.h Mon May 7 11:27:45 2007 *************** *** 0 **** --- 1,53 ---- + #ifndef _TCACHE_H_INCLUDED_ + #define _TCACHE_H_INCLUDED_ + + /*++ + /* NAME + /* tcache 5 + /* SUMMARY + /* trivial cache manager + /* SYNOPSIS + /* #include + /* DESCRIPTION + /* .nf + + /* + * Utility library. + */ + #include + #include + + /* + * External interface. + */ + typedef char *(*TCACHE_SAVE_FN) (const char *); + typedef void (*TCACHE_DELETE_FN) (char *); + + typedef struct { + VSTRING *key_buf; + CTABLE *table; + TCACHE_SAVE_FN save; + TCACHE_DELETE_FN delete; + } TCACHE; + + extern TCACHE *tcache_create(int, TCACHE_SAVE_FN, TCACHE_DELETE_FN); + extern void tcache_free(TCACHE *); + + #define tcache_lookup(tp, key) \ + ((char *) ctable_locate((tp)->table, (key))) + #define TCACHE_ENTER(tp, key, val) \ + ((char *) ctable_enter((tp)->table, (key), (tp)->save ? \ + (tp)->save(val) : (val))) + + /* LICENSE + /* .ad + /* .fi + /* The Secure Mailer license must be distributed with this software. + /* AUTHOR(S) + /* Wietse Venema + /* IBM T.J. Watson Research + /* P.O. Box 704 + /* Yorktown Heights, NY 10598, USA + /*--*/ + + #endif