Having recently been troubleshooting a customers hybrid Exchange environment I thought I’d write about my experience.
The customer had an Exchange 2010 deployment and the Hybrid Configuration Wizard had already been run but they were having mail flow problems from their On-Prem to Online mailboxes.
First I decided to perform the basic SMTP connectivity test to Exchange Online Protection to make sure their address was allowed to send mail through. To do this from a command prompt I ran:
telnet [tenantname].mail.protection.outlook.com 25 ehlo mail from: onprem.mailbox@domain.com rcpt to: cloud.mailbox@domain.com
Now, normally you should get a “250 2.1.5 Recipient OK” message back however we received:
Our IP address is blacklisted, opps! So we proceeded to resolve the blacklist issue, upon investigation I discovered that the Exchange servers were actually connecting externally as the default IP address on their internet connection, which client devices also used.
I always recommend that mail servers should be presented externally via an IP address that isn’t used by other services just to prevent issues like this.
Once removed from the blacklist we were able to perform the test again successfully, all good, or so we thought!
Email was still not flowing and building up in the queue, at this point I turned on verbose logging on the outbound SMTP connector and went to check the SmtpSend folder for the logs.
Upon opening the logs I found the following, repeated
Now, I like to play spot the difference with logs from working deployments to try and spot things that are not working or incorrect, so I got a log from a working deployment:
As you can see, after the STARTTLS we should see the “220 2.0.0 SMTP server ready” response followed by the establishment of the TLS connection, but in our case we weren’t and presumably the “Remote” entry was to say it had timed out.
Now I had to find a way to manually establish a TLS SMTP connection, I’m not aware of a way of using the standard telnet based approach to do this so had to find an alternative.
That alternative was to use openSSL to establish the connection, to do this you can use the following command:
openssl s_client -starttls smtp -crlf -connect [tenantname].mail.protection.outlook.com:25
When run work a working machine the connection should establish and look similar to the below (full output is much longer):
However from the customers Exchange server it would not connect
The customer raised it with their firewall team who advised that there was some “app awareness configured and it only allowed smtp over tls” not sure how it worked from a telnet client or what they changed to resolve it but that’s the dark art of firewall appliances!
[This is an edited version of a post that was originally published at ucsteve.com]
