Can one send mail from the host server rather than an email account hosted on an SMTP server?

[jjjjp]

I have had intermittent problems in the past sending out multiple emails (in a loop, up to 50 emails) using the Send Mail script step. Some of the emails can fail to send. I was put in touch with an IT support person in the university where I work, and he thought that the problem might be solved by sending via an SMTP server intended specifically for applications and therefore equipped to handle the high volumes and fast speeds of the multiple emails produced by a looping set of send instructions.  The prefix for the SMTP server is appsmtp rather than smtp. I gave him the IP address for my version of FM Server 14, and he created the necessary firewall exception. However, if this solution is to work, the emails must go out directly from the server rather than from an email address that is hosted by the server. Even though I provide the new server address (smtp.utoronto.ca), and the correct port, my attempts to send test emails have failed. The IT person did not see correct the IP address at his end. He saw the IP address associated with my Mac.

Is what I am trying to do possible? And if it is, what do I need to do to make it work? Maybe automatic emails (that happen overnight) will work but not emails initiated by a client?

If it's not possible to do what I'm trying to do, please provide any insight into why emails in a loop might intermittently fail. Would it help to provide a time buffer between sends via a pause?

[OlgerDiekstra]

Sounds like you're using the send mail step using the local client, not the server. In the "Send Mail" script step there's a "send via" option. You need to have that set to "SMTP server". And then make sure every "Send Mail" script step in your solution uses the same approach (where applicable). I have emails going out from the server day and night. Hundreds at a time. No problems.

If your IT person is seeing the IP of your Mac on the SMTP server, your not using the "send via SMTP server". Not sure what "appsmtp" is meant to be, probably something specific to Macs? You can query the SMTP server from the FMserver using a telnet session ("telnet smtp.utoronto.ca 25") and enter the "helo" command. If your IT person has set it up correctly, then it should respond with a banner, or some sort of response. If that works, you can try a test script from the FMserver with the send mail script step, sending a test message via the SMTP server. If that also works, you have to work out why your emails are not going through the server.

Edited August 31, 2015 by OlgerDiekstra

[jjjjp]

Thanks for your prompt response.

I definitely did use Send via SMTP server. As for the IP address the IT person is seeing at his end, I meet with him on Thursday, and so will try to investigate further and will get back to this thread with any new information about that. I have no trouble sending through the smtp rather than the appsmtp—except for the very intermittent and unpredictable failures! Do you have a sense of whether the fact that the sends are happening in a loop could be at play?

[OlgerDiekstra]

Do you have a sense of whether the fact that the sends are happening in a loop could be at play?

Very well possible. Get the IT guy to check whether they have throttling enabled on the SMTP server. Essentially, this restricts the amount of emails you can send per time frame. Ie, only 10 emails per minute. Unfortunately, FM doesn't give you much clues as to why the delivery failed. You'd have to look in the SMTP server logs for that.

[jjjjp]

I spoke to my tech person and have a better sense of what's going on and what the limitations are. The "appsmtp" server is just an SMTP server that was designed and configured at our university to handle input from apps. The "smtp" server I've used before can also handle smtp requests from apps, but the IT person thought that the appstmtp server might solve the problem of intermittent send failures (FM error code 1506). Unlike the smtp server, however, the appstmtp server has a firewall, and sending mail from an FM client would require a firewall exception already to have been set up for that client's IP address. Even though the email would not be sent out from the client, the request to send the email would. So I can really only use the appsmtp server for the server side scripts that get invoked in my database once a day at 7 in the morning. It isn't practical to create exception for the IP addresses of all possible users.

I gather that if one is sending mail through the client, as part of an interactive process in which the client determines the exact content and destination of the emails, then a server side script is out of the question. To be sure, I could create a kind of queue of records representing the emails the client intended to send, and then an hourly server-side script could send out the emails in the queue. But this is too complicated a solution for me to contemplate now, especially since I don't even know yet whether the appsmtp server is going to solve my intermittent send problems. And it's desirable that the emails go out exactly when the client completes the interactive process of sending an email batch.)

By the way, the tech person said that throttling isn't enabled on the smtp server. So the failures wouldn't likely be a result of the smtp requests being too close together in time. He thinks that the problem must be on the coding end. But I can run the same script five minutes later after nothing has changed and not get the same random failure in one of the emails in the batch that I had experience five minutes later.

At least now I will be able to observe if the intermittent send failures happens for our overnight server-side scripts after I switch those to the appsmtp server. The tech person will be monitoring his log for failures from our FM Server.

Thanks very much for your help. Any further insights are of course welcome.

[OlgerDiekstra]

I gather that if one is sending mail through the client, as part of an interactive process in which the client determines the exact content and destination of the emails, then a server side script is out of the question. To be sure, I could create a kind of queue of records representing the emails the client intended to send, and then an hourly server-side script could send out the emails in the queue. But this is too complicated a solution for me to contemplate now, especially since I don't even know yet whether the appsmtp server is going to solve my intermittent send problems. And it's desirable that the emails go out exactly when the client completes the interactive process of sending an email batch.)

That's what I've done. Create an Email Queue. It's serviced every 5 minutes by the server. it sends out bulk emails to customers a few times a day. Some bulk emails are send as actual emails, other bulk sets got to a SMS gateway. This is hundreds a time. And then throughout the day staff submit emails to be send. Because the queue gets serviced every 5 minutes, its realtime enough. Email isn't meant to be instantenous anyway. The load on the server is insignificant.

The biggest downside of serverside emails (without plugins) is that its plaintext email only. Attachments are possible but only one. I resorted to zipping all files that need to be attached and attaching a single zipfile.

If you want formatted emails, then you have to use either a plugin or get creative with php for instance. I did that for a while which worked well for emails that were always formatted the same way.

By the way, the tech person said that throttling isn't enabled on the smtp server. So the failures wouldn't likely be a result of the smtp requests being too close together in time. He thinks that the problem must be on the coding end. But I can run the same script five minutes later after nothing has changed and not get the same random failure in one of the emails in the batch that I had experience five minutes later.

At least now I will be able to observe if the intermittent send failures happens for our overnight server-side scripts after I switch those to the appsmtp server. The tech person will be monitoring his log for failures from our FM Server.

If emails are usually going through fine, then it's either a server process that gets in the way (antivirus?) or something on the SMTP server. Even if it's the FMserver, the SMTP server should be able to provide a hint. If there's lines in the logs stating "connection from client lost" or something similar, then the FMserver stopped communicating to the SMTP server, or something interfered outside the SMTP service/daemon with the traffic (which could also be antivirus on the SMTP server). One thing you could do if this is a server run script, is wait a minute when an error 1506 occurs, then try it again. And repeat for 3 or 4 times if necessary.

Goya's Base Elements (http://www.goya.com.au/blog) has recently been updated (just noticed) and now supports SMTP, which I'll have to check out to see if it adds significant benefits. I use BE for other purposes already. It's free.

[Josh Ormond]

See if something like this might help.  http://timdietrich.me/blog/filemaker-mandrill-send-emails-and-text-messages/

[jjjjp]

Thank you, both Josh and OlgerDiekstra. These are excellent suggestions. I think I will, asap, switch all of the server-side sends to the appstmtp server and see over the next while whether that solves the random failures that occurred on the smtp server. Once it begins to look that way, I will implement an alternative to the current smtp server that sends requests from the client rather than FM Server. Probably I will go with a queue to keep everything within Filemaker and the university, but I will also consider the Mandrill option.

[Josh Ormond]

Just as a side note. It's a bit more advanced, but Mandrill gives you some options to track the emails as well. Things like seeing how many were opened, links that were clicked in the email, etc. Very useful if you need know if your emails are being put to use, or are just wasting server space.