Re: POP3 mail fetcher that supports unreliable connections?
On 2003-11-04 20:34:18 -0600, Jacob S. wrote:
> All the Debian installations I've ever done asked me to choose one of
> four choices for configuring the delivery system. And every fetchmail
> installation I've done had the option to tell it which user to deliver
> the mail to, making it nice and straight forward.
Yes, this may be fine from Debian packages, and perhaps I could use
fetchmail with SMTP reliably on Debian; I'm a bit doubtful, though:
for instance, what happens if the local disk is full? I don't want
a mailer-daemon to be sent to the sender of the message, as it is
normally done with SMTP; instead, I want an error to be immediately
reported so that the message isn't deleted from the POP3 server.
But anyway, I want to use the same tools under different Unix
distributions (unfortunately, Debian isn't everywhere): for instance,
I currently use getmail to fetch my mail from POP3 mailboxes, mutt to
read my mail and so on. If you want to know, when I lost my mail, it
was under Solaris, but it was never said in the fetchmail documentation
that fetchmail shouldn't be used under Solaris...
> hmm... I always thought the Debian way was to test things before you
> relied on them. I guess I've been doing too much work.
I've now been warned for several years. However, one can't always
easily test things (in particular the cases where the local disk is
full, the cases where the memory is full and the OOM killer has to
kill some random process, the cases where there are other failures,
and so on).
> > Completely wrong! I hope you don't think that for instance, a MUA
> > should use the local delivery system for copying messages to a
> > different mailbox!
>
> When you're copying mail between mailboxes you're not moving it to a
> separate computer, downloading it off a server, or anything else.
When using IMAP mailboxes, messages are downloaded from a server.
So, with such mailboxes, I don't see the difference.
> You're simply moving data between files or files between folders
> (depending on what kind of mailbox you use). That's not a very good
> example.
Apart the above point about IMAP, I don't see why the fact that the
mail comes from a different machine makes a difference. The important
point IMHO is that the mail has already been delivered to the right
person. (If you say that a single POP3 mailbox can be used by several
people, this is not the primary use, and anyway this requires extended
configuration.)
> But on the other hand, sometimes you do use multiple programs chained
> together for moving things around on your own system - that's why the
> pipe | was invented.
The pipe can't solve all problems and has its limitations. In particular
this is more or less a one-way communication (with very limited error
handling, something very important for data integrity). But I agree that
for mail copying or for POP3, it can be OK, as long as error checking is
properly done (is it for current versions of fetchmail? Old versions had
problems even without any error -- see below). With a protocol with
immediate delete for individual messages, this wouldn't be OK, as an
error could be signaled too late.
> > It is poorly designed as a POP3 mail fetcher tool, as it relies on
> > special support of the local delivery configuration.
>
> I've never heard smtp called "special support" before.
This requires the SMTP server to be configured to support fetchmail
(perhaps the case with Debian, but not the case with every OS or
configuration by system administrators).
> In another post you claim getmail does it the "proper way". That's the
> nice thing about Linux and open source; having multiple programs to do
> the same job, and being able to choose between the two. But I certainly
> wouldn't say that means fetchmail is defective by any means.
Well, data/system privacy and data integrity are the most important
points. Contrary to getmail, fetchmail doesn't ensure data integrity.
On 2003-11-04 21:41:16 -0500, ScruLoose wrote:
> Yup, and I think that what Monique was trying to say tactfully really
> comes down to this: If you're stupid enough to connect fetchmail to a
> misconfigured MTA, that's your problem.
How could I know that it were misconfigured? (I *didn't* configure
it, this was the job of the system administrators.)
> If a non-privileged user wants to use fetchmail and the MTA is
> misconfigured and won't play nice, there's always the -m option to
> skip the MTA and pipe mail straight to procmail (or the MDA of
> choice).
It's a bit too late after losing mail. So, after losing mail, I used
some option to send the messages directly to a file. This worked for
some time, but after a fetchmail upgrade, fetchmail started to corrupt
my mailbox by writing things such as "downloading mail..." messages to
the mailbox instead of the terminal (or stderr), but this is another
story... Then I switched to getmail and I have never had any lost mail
with it.
BTW, I installed fetchmail on another system (SuSE) later, and I again
had lost mail problems (except that I didn't really lose mail since I
had the good idea to keep the mail on the server this time): messages
from one of the POP3 mailboxes weren't delivered (though they had been
retrieved). So, I immediately switched to getmail, again (it was also
faster than fetchmail). If you can read French:
From: Vincent Lefevre <vincent+news@vinc17.org>
Subject: [fetchmail] mails non recus
Newsgroups: fr.comp.mail
Date: 18 Aug 2001 23:21:51 GMT
Message-ID: <20010818225650$2165@vinc17.org>
> Lots of things that an unprivileged user will _use_ need root to
> _configure_ before they work.
But this shouldn't be the case for POP3 fetching.
> I think that if you want to convince anyone that Monique, and fetchmail,
> and its authors, and its many users (and the entire Unix philosophy?)
> are "completely wrong"; you'll need a more solid argument than this
> groping for a dubious metaphor.
It seems that many other people have been convinced. From the getmail
FAQ:
Why did you write getmail? Why not just use fetchmail?
I do not like some of the design choices which were made with
fetchmail. getmail does things a little differently, and for my
purposes, better. In addition, most people find getmail easier to
configure and use than fetchmail. Perhaps most importantly, getmail
goes to great lengths to ensure that mail is never lost, while
fetchmail (in its default configuration) frequently loses mail, causes
mail loops, bounces legitimate messages, and causes many other
problems.
In addition, fetchmail has a long history of security problems:
[snip]
> Besides which, I can easily see people having mutt use procmail to sort
> and move mail to a different mailbox. Maybe I want to take advantage of
> filter rules that weren't there when the mail was delivered the first
> time...
Of course, getmail can do that for instance (with a pipe).
And as you talk about procmail, guess what is the default rule...
store the mail to the user's mailbox! (Not send the mail to the
local delivery agent.) So, why should this be different for a POP3
fetcher?
> Very much the same philosophy as with fetchmail passing to the MTA.
> The MTA's job is to transport mail. It's generally good at it.
No, not the same philosophy. First the MTA's configuration is at system
level, though I want here a configuration at the user level (of course,
the user could still choose to pipe the message to exim or whatever if
he wishes to, but the configuration of what to do with the message is
entirely his choice). Also a MTA has different kinds of rules, which
are not suited to a simple copy (see the comments from the getmail FAQ
for instance).
> Fetchmail's job is to get mail from a remote mailbox. Why rewrite
> MTA functionality?
The goal is *not* to rewrite MTA functionality. The goal is to have
different functionality (much simpler than a MTA).
> Why not just hand off to the existing MTA and let it do its job? If
> the MTA is broken/misconfigured that's hardly fetchmail's fault.
It isn't necessarily broken or misconfigured; it may be configured
for things other than mails forwarded by fetchmail. With antispam
rules, I suppose that this is even more complex. Also, it is
well-known that SMTP isn't a protocol as reliable as file copy.
--
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/> - 100%
validated (X)HTML - Acorn Risc PC, Yellow Pig 17, Championnat International
des Jeux Mathématiques et Logiques, TETRHEX, etc.
Work: CR INRIA - computer arithmetic / SPACES project at LORIA
Reply to: