Mail arrival time puzzle

[bob prohaska]

Yesterday an email exchange ended seemingly without a reply
to my last message. This morning, mutt reported the reply
present, timestamped appropriately in a mix of UTC and PDT
suggesting it actually reached my host at roughly the right time.
But, there had been no "new mail in this inbox" message at least
six hours after the arrival timestamp.

My mail host is FreeBSD using

Mutt 2.0.6 (2021-03-06)
Copyright (C) 1996-2021 Michael R. Elkins and others.
Mutt comes with ABSOLUTELY NO WARRANTY; for details type `mutt -vv'.
Mutt is free software, and you are welcome to redistribute it
under certain conditions; type `mutt -vv' for details.

System: FreeBSD 12.4-STABLE (arm)
ncurses: ncurses 5.9.20140222 (compiled with 5.9)
libiconv: 1.14
libidn2: 2.3.0 (compiled with 2.3.0)
hcache backend: Berkeley DB 5.3.28: (September 9, 2013)

Compiler:
FreeBSD clang version 13.0.0 (***@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
Target: armv7-unknown-freebsd12.4-gnueabihf
Thread model: posix
InstalledDir: /usr/bin

with Sendmail as the transport agent.

The correspondent is using Yahoo Mail via a reasonably recent
Mac OS X installation. Mail from Mac OS X correspondents is
often timestamped with UTC, while mine is timestamped PDT, but
mail is normally announced within a few seconds of receipt. In
this particular case the announcement was delayed by enough to
matter.

Can anybody suggest where to look for the source of the problem?
Initially I thought it had something to do with the mixed timestamps,
but they look correct, even if they're confusing to a human. Clocks
on both computers are set correctly.

Thanks for reading, and any suggestions.

bob prohaska

[Peter Pearson]

Post by bob prohaska
Yesterday an email exchange ended seemingly without a reply
to my last message. This morning, mutt reported the reply
present, timestamped appropriately in a mix of UTC and PDT
suggesting it actually reached my host at roughly the right time.
But, there had been no "new mail in this inbox" message at least
six hours after the arrival timestamp.

[snip]

Post by bob prohaska
with Sendmail as the transport agent.

[snip]

Post by bob prohaska
Can anybody suggest where to look for the source of the problem?

[snip]

Does mail arrive in your computer when some Mail Transport Agent
out there on the internet initiates an SMTP conversation on your
port 25? Or, alternatively, does your computer periodically query
some email service out on the Internet, and use POP or IMAP to
slurp up waiting messages?

Does your incoming email pipeline pass through procmail, which might
make a useful log-file entry? Do you have a /var/log/mail.log ?

[bob prohaska]

Post by Peter Pearson

Post by bob prohaska
Yesterday an email exchange ended seemingly without a reply
to my last message. This morning, mutt reported the reply
present, timestamped appropriately in a mix of UTC and PDT
suggesting it actually reached my host at roughly the right time.
But, there had been no "new mail in this inbox" message at least
six hours after the arrival timestamp.

[snip]

Post by bob prohaska
with Sendmail as the transport agent.

[snip]

Post by bob prohaska
Can anybody suggest where to look for the source of the problem?

[snip]
Does mail arrive in your computer when some Mail Transport Agent
out there on the internet initiates an SMTP conversation on your
port 25?

Truthfully, I've not paid much attention up to now. I start sendmail
and email simply works. If you're asking whether I use any intermediate
mail hosts, the answer is no.

Post by Peter Pearson
Or, alternatively, does your computer periodically query
some email service out on the Internet, and use POP or IMAP to
slurp up waiting messages?

No.

Post by Peter Pearson
Does your incoming email pipeline pass through procmail, which might
make a useful log-file entry?

No.

Post by Peter Pearson
Do you have a /var/log/mail.log ?

There are log files called /var/log/maillog, with numbered, compressed
old versions. I don't see anything recognizable as a failure, but there
are many reports of the form:

Jul 14 11:54:10 www sm-mta[71572]: 36EIsANx071572: 167.255.125.34.bc.googleusercontent.com [34.125.255.167] did not issue MAIL/EX
PN/VRFY/ETRN during connection to IPv4

It appears to me that the mail probably arrived more or less on schedule,
but mutt didn't inform me of its presence until the next morning.

Thanks for writing!

bob prohaska

[Peter Pearson]

Post by bob prohaska

Post by Peter Pearson

Post by bob prohaska
Yesterday an email exchange ended seemingly without a reply
to my last message. This morning, mutt reported the reply
present, timestamped appropriately in a mix of UTC and PDT
suggesting it actually reached my host at roughly the right time.
But, there had been no "new mail in this inbox" message at least
six hours after the arrival timestamp.

[snip]

Post by bob prohaska
with Sendmail as the transport agent.

[snip]

Post by bob prohaska
Can anybody suggest where to look for the source of the problem?

[snip]
Does mail arrive in your computer when some Mail Transport Agent
out there on the internet initiates an SMTP conversation on your
port 25?

Truthfully, I've not paid much attention up to now. I start sendmail
and email simply works. If you're asking whether I use any intermediate
mail hosts, the answer is no.

Post by Peter Pearson
Or, alternatively, does your computer periodically query
some email service out on the Internet, and use POP or IMAP to
slurp up waiting messages?

No.

Post by Peter Pearson
Does your incoming email pipeline pass through procmail, which might
make a useful log-file entry?

No.

I'm trying to get an idea of how email arrives in your computer,
which might help illuminate how mutt is supposed to become aware
of a newly arrived message. I'm not very good at this, 'cause
my notions of how email works were formed about 40 years ago.

One possibility is that your computer has a domain name, such as
bobscomputer.com, and your email address is ***@bobscomputer.com, and
when someone sends email to that address, their mail-transport agent
looks up bobscomputer.com in the domain-name system to get your IP
address, and then sends you the email message by connecting to port 25
at that IP address and conducting an SMTP conversation. This would mean
that your computer has some kind of mail-transport agent constantly
listening for SMTP connections on port 25.

If your email address is not ***@bobscomputer.com, then it's
probably ***@somethingelse.com (maybe somethingelse is GMail),
and there's some arrangement by which either (a) your computer
periodically contacts somethingelse.com to ask about new messages,
or (b) somethingelse.com has some way to notify your computer
when a new message has arrived for you. I use system (a), and
know almost nothing about systems resembling (b).

Does any of this fit your reality?

I'm pretty sure that 95% of the participants in this newsgroup
know a *lot* more about this than I do. Please, guys, won't one
of you jump in?

[bob prohaska]

Post by Peter Pearson

Post by bob prohaska

Post by Peter Pearson

Post by bob prohaska
Yesterday an email exchange ended seemingly without a reply
to my last message. This morning, mutt reported the reply
present, timestamped appropriately in a mix of UTC and PDT
suggesting it actually reached my host at roughly the right time.
But, there had been no "new mail in this inbox" message at least
six hours after the arrival timestamp.

[snip]

Post by bob prohaska
with Sendmail as the transport agent.

[snip]

Post by bob prohaska
Can anybody suggest where to look for the source of the problem?

[snip]
Does mail arrive in your computer when some Mail Transport Agent
out there on the internet initiates an SMTP conversation on your
port 25?

Truthfully, I've not paid much attention up to now. I start sendmail
and email simply works. If you're asking whether I use any intermediate
mail hosts, the answer is no.

Post by Peter Pearson
Or, alternatively, does your computer periodically query
some email service out on the Internet, and use POP or IMAP to
slurp up waiting messages?

No.

Post by Peter Pearson
Does your incoming email pipeline pass through procmail, which might
make a useful log-file entry?

No.

I'm trying to get an idea of how email arrives in your computer,
which might help illuminate how mutt is supposed to become aware
of a newly arrived message. I'm not very good at this, 'cause
my notions of how email works were formed about 40 years ago.
One possibility is that your computer has a domain name, such as
when someone sends email to that address, their mail-transport agent
looks up bobscomputer.com in the domain-name system to get your IP
address, and then sends you the email message by connecting to port 25
at that IP address and conducting an SMTP conversation. This would mean
that your computer has some kind of mail-transport agent constantly
listening for SMTP connections on port 25.

The sender is using Yahoo Mail via a Mac, I think using Safari as
the user interface. It's tempting to blame Yahoo, but I'm not sure
it has anything to do with Yahoo.

Sendmail is my MTA. My computer (FreeBSD) informs me when new mail
has arrived, independent of mutt, with announcements from shell
sessions I have running when they recieve keyboard input. The announcements
aren't instant, but usually show up within a few minutes in the form
"You have new mail."

Whether mutt relies on this mechanism is unclear. If mutt is running, new
messages simply show up in the window mutt is displaying.

The real puzzle is the randomness of the apparent delays. I just did
a few experiments mailing between accounts on the host and messages
were transferred and reported arrived within a few minutes. In the cases
which provoked this question messages were reported to have arrived long
after their arrival timestamps. And, most of the mail from Yahoo shows
up reasonably quick, say within 15 minutes.

Post by Peter Pearson
and there's some arrangement by which either (a) your computer
periodically contacts somethingelse.com to ask about new messages,
or (b) somethingelse.com has some way to notify your computer
when a new message has arrived for you. I use system (a), and
know almost nothing about systems resembling (b).
Does any of this fit your reality?
I'm pretty sure that 95% of the participants in this newsgroup
know a *lot* more about this than I do. Please, guys, won't one
of you jump in?

When knowledgeable folks ignore a question it usually means the
question is ill-formed, trivial or they don't know any answer.
I'm frequently guilty on all counts 8-) and the random nature
of this problem makes testing difficult.

Thanks for replying,

bob prohaska

[Eric Pozharski]

[ *SKIP* ]

Post by bob prohaska
Can anybody suggest where to look for the source of the problem?

You ought to read the headers first. And, basically, grep logs from
there. Sure, by default mutt hides most of them (or weeds them,
mutt-speak) but there's magic key-press that unveils them...

Problem is -- it's all spoonfeeding. You already know all of this,
don't you?

[ *SKIP* ]

I'm pretty sure that 95% of the participants in this newsgroup know a
*lot* more about this than I do. Please, guys, won't one of you jump
in?

My guess is -- people are fed up guessing. Just wait for it, here comes
drama.

[John D Groenveld]

Post by bob prohaska
My mail host is FreeBSD using
Mutt 2.0.6 (2021-03-06)

You might try upgrading to the latest mail/mutt
from ports which has 2.2.10:
<URL:http://www.mutt.org/>

John
***@acm.org

[bob prohaska]

Post by John D Groenveld

Post by bob prohaska
My mail host is FreeBSD using
Mutt 2.0.6 (2021-03-06)

You might try upgrading to the latest mail/mutt
<URL:http://www.mutt.org/>

That's something I didn't think of at all.

Thank you!

bob prohaska