Author Topic: Outlook getting Stuck/disconnected occassionally with Exchange  (Read 35636 times)

haze

  • Full Member
  • ***
  • Posts: 155
  • Karma: +0/-0
    • View Profile
Hi,
I am aware that similar case has been posted before but just wanted to check if there is any new updates/solutions.

Got 2 ISG-1000 running as A/P cluster with ScreenOS 6.0r5.
Got problem of outlook getting stuck/disconnections when connected to exchange. The firewall is between the users and exchange & domain controllers. I noticed that i could replicate problem easily when clicking on address book or new mail and clicking on "TO". I found that outlook is connecting to DC(global catalog) to retrieve address book entries at the time of the problems. I used packet sniffer on the computer and found that at most time a session is established between outlook and DC and at the time of problem, there is retransmission from PC to DC. Clicking on reconnect(on outlook icon) or closing and opening outlook solves the problem. I also noticed at time of problem that the outlook has a session to  port 1025 of DC and it is in this session that retransmission occurs.

Following are changes i already made,
1) ALG MSRPC is disabled.
2) Service timouts have been modifed as following
" set service MS-EXCHANGE-DATABASE timeout 60
  set service MS-EXCHANGE-DIRECTORY timeout 60
  set service MS-EXCHANGE-INFO-STORE timeout 60
  set service MS-EXCHANGE-MTA timeout 60
  set service MS-EXCHANGE-STORE timeout 60
  set service MS-EXCHANGE-SYSATD timeout 60
  set service MS-RPC-EPM timeout 60"
3) Seperate policy is created from users to server vlan with all these services and it is on TOP
4) There is no IDP on policy.

I still keep getting the problem.
Do i need to change service timeout value of MS-Netlogon and MS-RPC-ANY (currently 1 minute)? Help appreciated :)
Thank you.


spingineer

  • Full Member
  • ***
  • Posts: 143
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #1 on: August 07, 2008, 06:59:31 am »
Try changing the timeout for ms-netlogon.  In a similar case, we took a debug flow, and cross referenced the uuid, and saw that it matched ms-netlogon, which happen to have a timeout of 1 minute.

martianism

  • Newbie
  • *
  • Posts: 33
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #2 on: August 08, 2008, 12:32:19 pm »
Hi Haze,

I have experienced a very similar issue with using ScreenOS 6.0r5.... specifically the problems were with GC connections to the domain controller when using Exchange 2007 / Windows AD 2003. If the same problem then there is a fix :-)

What was evident in our case when using a packet sniffer was the following sequence of events.

1 - The client sends the DC a MS-RPC-EPM request for service UUID - 1544f5e0-613c-11d1-93df-00c04fd7bd09)

2 - The client receives back a RPC-EPM response consisting of 2 Tower arrays.  The first array tells it to use some random high port (or if you are setting your ports statically in the registry on the DC, which I had to do to get round the problem initially, gives whichever port you set).  The second tower tells it to use 1025/TCP.

3 - The client attempts to connect to the DC on the first port... but gets no response because the firewall is blocking the packets.

4 - The client gives up and moves on to use port 1025 instead, which the ALG has allowed through the firewall.

This process sometimes gets repeated during a single use of outlook and it seems it sometimes just gives up altogether when it cannot use the first port (hence having to kick outlook).


After rather significant investigation to persuade the JTAC that this really was a case of the ALG not implementing the protocol properly (seems to always be an issue with responses which include multiple towers on this particular ALG) they produced a engineering patch, which will be rolled into 6.0r7 (still waiting for confirmation from JTAC as to when that will actually arrive though).   

The patch that I am currently running is 6.0.0r5-ea9.... not sure if that would be available from JTAC or not, its certainly not general release at this point judging by the patch name.   I should have more info soon as to when R7 is being released, or I guess you could prod JTAC to provide a version of this patch for your platform (I had it on SSG320 and 550). 

I can probably provide more detailed account + sanitized dbug flow data if that would help for comparison..

Hope it helps

--
Martin

haze

  • Full Member
  • ***
  • Posts: 155
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #3 on: August 09, 2008, 04:06:48 am »
Hi Spingineer,
Thanks for your reply.
I have changed the ms-netlogon to 30 min and observing behavior.

Hi Martian,
I have already disabled MS-RPC ALG. So would the same case apply here?

haze

  • Full Member
  • ***
  • Posts: 155
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #4 on: August 10, 2008, 12:00:53 am »
Hi Martin,
Could you tell me what exactly is a UUID? is it something relevant to MS-RPC? I am doing a debug rpc map. Do you need me to post it so that you could have a look through it?

What is  see through wireshark is is that there is a session from outlook pc to port 1025 of the DC(global Cat). Everything is working fine at this point. And it keeps working ok until at some point when user clicks on address book and i see retransmissions for the same session. I have to click "cancel server connections" and click on reconnect. If i click on address book continously i seems to be opening instantly. But if i leave outlook idle for some-time and i see the problem. Please help :(

haze

  • Full Member
  • ***
  • Posts: 155
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #5 on: August 10, 2008, 01:31:55 am »
Hi Martin,
Here is how the session goes just to give you an idea.
PC <-> Server TCP session established to port 1025(blackjack)
PC <-> Server (Bind,Bind ACK,AlterContect and AlterContext Response)
PC <-> Server (NSPI UNbind, NSPI Unbind response, ACK)
PC <-> Server (NSPI Bind request,NSPI Bind Response,ACK)
PC <-> Server (NSPI Unbind, NSPI Ubind response, ACK)
PC -> Server (NSPI Query rows request) - This is when i click on address book and it is about to get stuck
PC -> Server (TCP Retransmission NSPI Query request) - This happens until i cancel outlook server connections.
Hope this helps.


haze

  • Full Member
  • ***
  • Posts: 155
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #6 on: August 10, 2008, 05:45:10 am »
Hey guys,
I found the session that is being established on firewall between pc and server using the
"get session src-ip a.b.c.b dst-port 1025"
i found that the session is counting down and it is going through policy with all services set to 60 min. But the session was counting down from 60 secinds which was wierd. So i created a custom service for tcp port 1025 with timeout of 60 minutes. Then i can see for new sessions that it is counting down from 60 minutes. Now i am observing the behaviour of the firewall. The problem does not seem to be appearing as quickly as before but i will have to wait and see.
What i do not understand is why does not outlook send any keepalives to Domain Controller. There are users who keep the Outlook open the whole day without

martianism

  • Newbie
  • *
  • Posts: 33
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #7 on: August 11, 2008, 04:50:46 am »
Hi Haze,

Sorry for the delay..... loooong (rough) weekend :-(

UUID's are used by MS-RPC-EPM to identify what service it is you are wanting to talk to. 

eg (simplified)

Global Catalog's UUID could be 12 and Martin's Mystical RPC Service could be 14

Because you have to talk to the End Point Mapper on port 135 for any MS RPC service, you give the UUID of the service you would like to access in you initial request, so I would send a packet to port 135 asking to talk to service 12 if I wanted to use GC.

The server can then lookup with UUID what that service corrisponds to, if I am allowed to access it and what port range to use.  It will then reply with a packet which says 'Service 12 is on port 1025'

In the case of our AD config it was returning two possible ports, which is fairly normal.  It should first return a high random port that it has chosen and then 1025 as a last resort.

The problem we had was that the ALG was correctly correctly parsing the UUID as being for Global Catalog and allowing the request through, but when the reply came back it was only opening the second (less favorable) port for the communication, so when the client tried to talk to the server on the specified port, it was denied by the firewall.

I guess the key to seeing if it is the same issue from a packet sniff would be to look at the EPM request and response for the UUID I gave earlier.  If the response has two tower arrays (giving two different ports) and you then see SYN's from the client to the server but no reply, then it would definatly seem to be the same problem... otherwise, might not be.

Hope that makes some kind of sense..... caffeine required for makin brain work good  :-o

svishwakarma

  • Newbie
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #8 on: August 11, 2008, 01:56:40 pm »
Hi Haze, are you using Network Connect or Web Rewriting while using Outlook?

Thanks!

martianism

  • Newbie
  • *
  • Posts: 33
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #9 on: August 12, 2008, 06:34:36 am »
Hi Svishwakarma,

I think Haze is trying to pass through a firewall whereas Net Connect and Web Rewriting would apply to an IVE type solution afaik

Haze,

Just read your post.

I assume from this you are using a service with a static port definition in the policy?  By default Global Catalog (and the other AD services) use dynamic ports, so you should normally need to use the MS-RPC-EPM ALG coupled with the appropriate service, in this instance the MS-EXCHANGE-DIRECTORY service, this defines the UUID's for the required service/s.

Could you post a sanitized config used for this section of policy?  Might be useful if you could post a pcap session also, although that might be harder to sanitize.

--
Martin

martianism

  • Newbie
  • *
  • Posts: 33
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #10 on: August 12, 2008, 07:33:24 am »
Incidentally, JTAC have just confirmed that the patch that fixed the issue for us will definitely be rolled into 6.0.0R7 which will be released on 12/09/08 apparently.

svishwakarma

  • Newbie
  • *
  • Posts: 2
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #11 on: August 22, 2008, 12:24:01 pm »
Hi Haze, try creating a WSAM profile for outlook in resource profile and specify the entire subnet range which is having your exhange servers. Seems to have resolved the issue for me, atleast for now. Pls let me know if you need more info on this.

haze

  • Full Member
  • ***
  • Posts: 155
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #12 on: September 02, 2008, 07:43:11 am »
Hi Martin,
So the fix i did for this issue is create policy in following order from top
Users -> DC allow MS-RPC-Any with timeout of 60 minutes
Users -> Exchange allow customer port 1200 with timeout of 60 minutes.
This has resolved the issue. However if there is inactivity of more than 60 minute, still timeout will occur. There has to be some tweak from Microsoft/exchange side, because i do not see any keep alives from outlook to keep the session up. I do not have expertise on Microsoft so i have left it with the above config and it is working.

Hi svishwakarma,
The problem i was facing is not for SSL box. It is for Juniper Firewall.

Thanks Everyone
 :-D

damianh

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #13 on: September 04, 2008, 01:37:06 pm »
We had similar problems in my previous job. At the end we had to do RPC over HTTP.

http://office.microsoft.com/en-gb/help/HA011402731033.aspx


ArturPhk

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #14 on: October 06, 2008, 08:19:45 am »
Hi there everyone.

I would like to know updates about this problem, i'm having it with two ISG2000 with ScreenOS 6.0r5 and this is becoming a big problem...  :x

Has anyone been able to fix this?
Does the upgrade (to 6.0r7) fix the problem? Has anyone done it?

Thanks

Cheers,
Phk

haze

  • Full Member
  • ***
  • Posts: 155
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #15 on: October 08, 2008, 08:40:37 am »
Hi Arthur,
If it is the same problem that i had, just to recap, outlook client will have connections to 2 network resource , the AD and the Exchange. The issue was that outlook was not sending keep alive to both, so firewall would kill the session. So outlook fails sometimes.
I am currently using 6.0r5 and the workaround created is to
1)create a new service port 1200 and set timeout to 60 minute or more.
2)change timeout of rpc-any to 60 minute or more.
Create new policy from users to servers with both these services allowed and PUT IT ON TOP.
I am not getting any complaints from users any more.
If you set the timeout to 60minute and if the users keep the outlook open for more than 60 minute without doing anything like send/receive the session would have timed-out and the problem will appear. So at my site, i have set the session timeout to little less than the working hours(ie. 6 hours is timeout) so that i do not get bugged by users who leave outlook open for hours and not touching it. The only disadvantage i can think of is that the firewall has to maintain sessions and this uses up memory, but i am not facing any issues so far.

The way i see it, the only real fix can be either,
1) Outlook should send keepalives, at-least in our environment it does not and the exchange guys have no idea about these things.
(or)2) juniper should release some fix in the next release but i still feel firewall is doing the normal thing of killing a session if there in no activity after timeout.



haze

  • Full Member
  • ***
  • Posts: 155
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #16 on: December 24, 2008, 07:39:18 am »
Incidentally, JTAC have just confirmed that the patch that fixed the issue for us will definitely be rolled into 6.0.0R7 which will be released on 12/09/08 apparently.

Hi, just an update on this thread
I was going through release notes of 6.0.0R7 and found the following under fixes.
"276077óNon-RPC MS Exchange traffic is dropped due to incorrect
timeout."
I hope this is the fix that we were waiting for. Anyone with problems tried it?

ArturPhk

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #17 on: December 25, 2008, 10:10:21 am »
Incidentally, JTAC have just confirmed that the patch that fixed the issue for us will definitely be rolled into 6.0.0R7 which will be released on 12/09/08 apparently.

Hi, just an update on this thread
I was going through release notes of 6.0.0R7 and found the following under fixes.
"276077óNon-RPC MS Exchange traffic is dropped due to incorrect
timeout."
I hope this is the fix that we were waiting for. Anyone with problems tried it?

Greetings,

Yes, the 6.0.0r7 fixed our problems, Outlook timeouts stopped happening.

Merry Christmas for Everyone :)

Phk

csavoy

  • Newbie
  • *
  • Posts: 1
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #18 on: April 30, 2009, 08:22:11 am »
By default, there is no keep-alive for Outlook to keep the session open with the Exchange server.

I asked Microsoft. The solution is to create and/or change the value of the following registry key :

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\KeepAliveTime

Do this on the Exchange server and it will send keep-alive to the Outlook clients to keep the sessions open on the firewall. No more timeout tuning on the firewall.

I tried it on my Exchange server and it works.

A related article here : http://www.outlookpower.com/issuesprint/issue200402/00001228.html

Best Regards,

Christophe




ArturPhk

  • Newbie
  • *
  • Posts: 9
  • Karma: +0/-0
    • View Profile
Re: Outlook getting Stuck/disconnected occassionally with Exchange
« Reply #19 on: April 30, 2009, 08:27:40 am »
By default, there is no keep-alive for Outlook to keep the session open with the Exchange server.

I asked Microsoft. The solution is to create and/or change the value of the following registry key :

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\KeepAliveTime

Do this on the Exchange server and it will send keep-alive to the Outlook clients to keep the sessions open on the firewall. No more timeout tuning on the firewall.

I tried it on my Exchange server and it works.

A related article here : http://www.outlookpower.com/issuesprint/issue200402/00001228.html

Best Regards,

Christophe






Excellent :)

Thank you