I'm a bit baffled by this problem. Maybe there's a sendmail guru out there who can help me out here. We have some end-users who need to receive system-generated mail that originates from a java-based application on our network. The java app sends the mail through our sendmail cluster, which then sends the email on to the end-user over the Internet. The size of the emails can range from a few kb up through around 2Mb in size.
The mail logs show that these emails are typically delayed for up to 24 hours, sometimes longer. It appears that several attempts to deliver the email are made, with each unsuccessful one showing "Deferred" status in the maillog. After several hours of failed attempts the mail finally goes through successfully. Here is a typical entry (customer name redacted):
exmx1 : Dec 14 05:28:39 exmx1 sendmail[31574]: mBDIYhEa031574: to=< customer@example.com>, delay=00:00:00, xde lay=00:00:00, mailer=esmtp, pri=41874, relay=mail.example.com., dsn=4.0.0, stat=Deferred
Unfortunately there is no reason given for the "Deferred" status such as "Server unavailable", etc.
Do you have any ideas how I can begin to troubleshoot this?
"I'm a bit baffled by this problem. Maybe there's a sendmail guru out there who can help me out here. We have some end-users who need to receive system-generated mail that originates from a java-based application on our network. The java app sends the mail through our sendmail cluster, which then sends the email on to the end-user over the Internet. The size of the emails can range from a few kb up through around 2Mb in size."
I should probably add that each of these emails has an attached file, which accounts for the large size of the emails. Also, sorry if my previous email was sent in HTML format, I think I might have had "rich text" turned on in Gmail.
Sean Carolan wrote:
"I'm a bit baffled by this problem. Maybe there's a sendmail guru out there who can help me out here. We have some end-users who need to receive system-generated mail that originates from a java-based application on our network. The java app sends the mail through our sendmail cluster, which then sends the email on to the end-user over the Internet. The size of the emails can range from a few kb up through around 2Mb in size."
I should probably add that each of these emails has an attached file, which accounts for the large size of the emails. Also, sorry if my previous email was sent in HTML format, I think I might have had "rich text" turned on in Gmail.
just a WAG from notta guru . . .
Perhaps someone has a greylist filter running & your outbound emails look different enough to trigger it every time. They could whitelist you as a fix.
Sean Carolan wrote on Mon, 15 Dec 2008 10:15:15 -0600:
Do you have any ideas how I can begin to troubleshoot this?
sendmail -v -q (will flush the mail queue, just wait and look). First reason I would think of: greylisting.
Kai
On Mon, Dec 15, 2008 at 10:15:15AM -0600, Sean Carolan wrote:
exmx1 : Dec 14 05:28:39 exmx1 sendmail[31574]: mBDIYhEa031574: to=< customer@example.com>, delay=00:00:00, xde lay=00:00:00, mailer=esmtp, pri=41874, relay=mail.example.com., dsn=4.0.0, stat=Deferred
Unfortunately there is no reason given for the "Deferred" status such as "Server unavailable", etc.
Do you have any ideas how I can begin to troubleshoot this?
Hi Sean, Two suggestions, #1 - turn your sendmail logging/debugging setting up as high as it will go for just long enough to capture some of these events. (then turn it back to its previous setting)
#2 - try using script and then telnet to capture an SMTP session (Done by hand) with the MTA at the receiving end. This can be a little tricky and requires a good understanding of how SMTP transactions are done so you can emulate it by hand.
I would focus on hat the receiving end does when you send it a length that is > 1 MB.
Jeff Kinz
#1 - turn your sendmail logging/debugging setting up as high as it will go for just long enough to capture some of these events. (then turn it back to its previous setting)
#2 - try using script and then telnet to capture an SMTP session (Done by hand) with the MTA at the receiving end. This can be a little tricky and requires a good understanding of how SMTP transactions are done so you can emulate it by hand.
Thank you all for the suggestions, I think we may have pinned down the problem. This list is a great resource, props to all the experienced users who put in time answering questions here. :)
==================================== The mail logs show that these emails are typically delayed for up to 24 hours, sometimes longer. It appears that several attempts to deliver the email are made, with each unsuccessful one showing "Deferred" status in the maillog. After several hours of failed attempts the mail finally goes through successfully. Here is a typical entry (customer name redacted): exmx1 : Dec 14 05:28:39 exmx1 sendmail[31574]: mBDIYhEa031574: to=customer@example.com, delay=00:00:00, xde lay=00:00:00, mailer=esmtp, pri=41874, relay=mail.example.com., dsn=4.0.0, stat=Deferred Unfortunately there is no reason given for the "Deferred" status such as "Server unavailable", etc. ==================================== I would imagine this is a hotmail or yahoo account or something along those lines. Deferred does not usually mean bounced or turned into spam, it is more of a way of making the user wait for their mail. Yahoo can hold mail for a long time, minutes to hours, before they deliver.
If you keep resending the deferred mail, you may end up getting auto junked though. In my minds eye, they are just trying to get people to upgrade as then as if by magic, mail to them does not get deferred.
But in my experience, all deferred mail is usually sent, as far as I know. Just 'deferred' for a bit.
on 12-15-2008 12:01 PM Bob Hoffman spake the following:
==================================== The mail logs show that these emails are typically delayed for up to 24 hours, sometimes longer. It appears that several attempts to deliver the email are made, with each unsuccessful one showing "Deferred" status in the maillog. After several hours of failed attempts the mail finally goes through successfully. Here is a typical entry (customer name redacted):
exmx1 : Dec 14 05:28:39 exmx1 sendmail[31574]: mBDIYhEa031574: to=customer@example.com, delay=00:00:00, xde lay=00:00:00, mailer=esmtp, pri=41874, relay=mail.example.com., dsn=4.0.0, stat=Deferred
Unfortunately there is no reason given for the "Deferred" status such as "Server unavailable", etc. ==================================== I would imagine this is a hotmail or yahoo account or something along those lines.
Not necessarily
Deferred does not usually mean bounced or turned into spam, it is more of a way of making the user wait for their mail. Yahoo can hold mail for a long time, minutes to hours, before they deliver.
Any MTA can defer mail. It doesn't need to be a big bulk provider.
If you keep resending the deferred mail, you may end up getting auto junked though. In my minds eye, they are just trying to get people to upgrade as then as if by magic, mail to them does not get deferred.
Like I said, not necessarily...
But in my experience, all deferred mail is usually sent, as far as I know. Just 'deferred' for a bit.
Servers defer mail for many reasons. The message could exceed a size limit that makes it wait until a certain time.
The server could be over a set load average. It will need to flush its queue and wait until the load comes down.
It could be greylisting, but I have never seen greylisting set this high.
It could be under a DDOS attack, which makes the server stop listening for a while.
A log entry with deferred with no other message is probably connection related. Maybe the mail server is on a small pipe with most of the bandwitth being used for other reasons at that moment.
There are more reasons than a conspiracy to get you to pay a subscription...
There are more reasons than a conspiracy to get you to pay a subscription...
True, but in most or almost all, it is not your sendmail or your server that has anything to do with it. A bounce would indicate a problem or other things...but 'defer' could mean anything. I get a lot from yahoo, both with the stupid ID things and one server without them. They can defer for a while.
I seldom get a defer from anywhere else, but a mailout of a large list could get some.
But I see nothing you can do at your end if you are sending it other than sending a letter to the deferring agent if you have the email, you know who it is. If it is a big company like yahoo or hotmail, I would just send a letter to the recipient and tell them about the issue so they can expect a delay.
Anyone notice how little junk snail mail they are getting now? Or how many snuggie blanket ads they are receiving via email....strange world out there.
on 12-15-2008 2:12 PM Bob Hoffman spake the following:
There are more reasons than a conspiracy to get you to pay a subscription...
True, but in most or almost all, it is not your sendmail or your server that has anything to do with it. A bounce would indicate a problem or other things...but 'defer' could mean anything. I get a lot from yahoo, both with the stupid ID things and one server without them. They can defer for a while.
I seldom get a defer from anywhere else, but a mailout of a large list could get some.
But I see nothing you can do at your end if you are sending it other than sending a letter to the deferring agent if you have the email, you know who it is. If it is a big company like yahoo or hotmail, I would just send a letter to the recipient and tell them about the issue so they can expect a delay.
Anyone notice how little junk snail mail they are getting now? Or how many snuggie blanket ads they are receiving via email....strange world out there.
Bulk mail is still expensive, and it is easier for the scammers to send e-mail then to try and get a bulk mail piece past the postal inspectors.
Looking at my stats for the year, At the beginning of 2008 I was rejecting (at MTA) less than half a million messages a month. Now it is over 2 million a month. That is pre-processing rejections on a system with less than a hundred people on it. I can imagine what you ISP level guys are dumping!
But I see nothing you can do at your end if you are sending it other than sending a letter to the deferring agent if you have the email, you know who it is. If it is a big company like yahoo or hotmail, I would just send a letter to the recipient and tell them about the issue so they can expect a delay.
It's not a big webmail provider, just a smaller company. I suspect it might have to do with the large attachments, or their mail servers getting overloaded.
It's not a big webmail provider, just a smaller company. I suspect it might have to do with the large attachments, or their mail servers getting overloaded.
I really wish they would come up with a few things..
1- A real all in one really well programmed email system for linux and windows. Something highly customizable and does sending/receive/pop/transfer/etc without all the nightmares. Not having a degree in all the mail programs makes it hard to not make mistakes the more stuff you try to do.
2- As part of that, finally do away with the whole apache@myserver thing for web apps. Along with a nice way for php and asp to access it safely instead of having to use smtp via some OOP thingee.
3- paid program option with id like sslmail would be awesome. Especially to get through the junk that many big free mail companies and isps put in our way, as well as making their life easier. Since it is ssl cert., they should be able to come after you if you are a spammer or phisher. Also, it would get rid of the senderID,spf, etc junk they are trying to push on us.
4- so, santa, that is what I would wish for. I know...not gonna happen.
Ever notice the bulk of spam mail is either gibberish or hawking a product like a drug or well known name brand? Me thinks maybe that big business has it all set up..flood with gibberish and flood with their name brand junk.. Makes it look like lots of spammers when it is probably just a few large conglomerates.
(uh oh, they might have heard me....<ducks>)
on 12-15-2008 2:55 PM Bob Hoffman spake the following:
It's not a big webmail provider, just a smaller company. I suspect it might have to do with the large attachments, or their mail servers getting overloaded.
I really wish they would come up with a few things..
1- A real all in one really well programmed email system for linux and windows. Something highly customizable and does sending/receive/pop/transfer/etc without all the nightmares. Not having a degree in all the mail programs makes it hard to not make mistakes the more stuff you try to do.
2- As part of that, finally do away with the whole apache@myserver thing for web apps. Along with a nice way for php and asp to access it safely instead of having to use smtp via some OOP thingee.
3- paid program option with id like sslmail would be awesome. Especially to get through the junk that many big free mail companies and isps put in our way, as well as making their life easier. Since it is ssl cert., they should be able to come after you if you are a spammer or phisher. Also, it would get rid of the senderID,spf, etc junk they are trying to push on us.
4- so, santa, that is what I would wish for. I know...not gonna happen.
Ever notice the bulk of spam mail is either gibberish or hawking a product like a drug or well known name brand? Me thinks maybe that big business has it all set up..flood with gibberish and flood with their name brand junk.. Makes it look like lots of spammers when it is probably just a few large conglomerates.
(uh oh, they might have heard me....<ducks>)
The big conglomerates spam you with TV and print ads, and junk in your postal mail box. I would suspect most of the spam is actually fake or illegal products or knockoffs that they couldn't legitimately sell in any traditional retail system, or attempts to get your credit card numbers for illegal uses.
The retail police have been dispatched to your residence because of your subversive messages. Have a nice day!! ;-P
Bob Hoffman wrote:
It's not a big webmail provider, just a smaller company. I suspect it might have to do with the large attachments, or their mail servers getting overloaded.
I really wish they would come up with a few things..
1- A real all in one really well programmed email system for linux and windows. Something highly customizable and does sending/receive/pop/transfer/etc without all the nightmares. Not having a degree in all the mail programs makes it hard to not make mistakes the more stuff you try to do.
Look at SME server from http://www.contribs.org. Answer a few questions during the install, add users with a simple web form, and it works. It has webmail too.
2- As part of that, finally do away with the whole apache@myserver thing for web apps. Along with a nice way for php and asp to access it safely instead of having to use smtp via some OOP thingee.
SME server has some of that too - and there are an assortment of contributed add-ins. However the extent that it is already customized makes it harder to make your own local changes if you do need any.
on 12-15-2008 8:15 AM Sean Carolan spake the following:
I'm a bit baffled by this problem. Maybe there's a sendmail guru out there who can help me out here. We have some end-users who need to receive system-generated mail that originates from a java-based application on our network. The java app sends the mail through our sendmail cluster, which then sends the email on to the end-user over the Internet. The size of the emails can range from a few kb up through around 2Mb in size.
The mail logs show that these emails are typically delayed for up to 24 hours, sometimes longer. It appears that several attempts to deliver the email are made, with each unsuccessful one showing "Deferred" status in the maillog. After several hours of failed attempts the mail finally goes through successfully. Here is a typical entry (customer name redacted):
exmx1 : Dec 14 05:28:39 exmx1 sendmail[31574]: mBDIYhEa031574: to=<customer@example.com mailto:customer@example.com>, delay=00:00:00, xde lay=00:00:00, mailer=esmtp, pri=41874, relay=mail.example.com http://mail.example.com., dsn=4.0.0, stat=Deferred
Unfortunately there is no reason given for the "Deferred" status such as "Server unavailable", etc.
Do you have any ideas how I can begin to troubleshoot this?
Does the machine have good dns resolution? Are the clocks synced to a proper time.. (IE... local time is +8 but the clock is actually -8). GMT is OK as long as the box knows that is its timezone.
-
Bob Hoffman -
jkinz＠kinz.org -
Kai Schaetzl -
Les Mikesell -
Scott Silva -
Sean Carolan -
Toby Bluhm