Slow LDAP group retrieval

[How to run the NSM GUI under Mac OS X]

How to run the NSM GUI under Mac OS X
How to setup a route-based VPN on a NetScreen
How to make your PPTP VPN client work through a NetScreen/SSG
Setting up a dial-up VPN and NetScreen Remote
How the Global Zone Works (It's probably not what you think)
How to configure NetScreen/SSG devices to work with the built in Mac OS X VPN client
How to Configure Dial-up VPN with route-based tunnels, XAuth, and use IPSecuritas

[matfa]

Hey Guys, hoping that someone can shed some light on this for me. I updated my cluster of SA4000s to 6.4R3 in October and ever since then, users have been complaining about slow speed logging in. I don't know if the OS upgrade was the cause of this as of yet. I myself never noticed this since this issue only seems to affect certain users. At the time, these logins were taking 10 -15 seconds longer than normal.
 
This week, all hell broke loose and this issue is becoming more prevalent with more users being affected everyday. As of right now, I could try logging in 4 times before actually being signed in. When it fails, after about 3 minutes of waiting after entering my pin from my token (2nd factor),I get thrown back to the login screen which advises me that my login credentials are incorrect. Back when we were on OS 6.2R5, the host checker screen would show 1-3 seconds after entering my token number.
 
I have been doing policy traces which show the primary (AD) and secondary (Entrust IDG) authentications pass and occur pretty quickly. Then it goes into the role mapping stage that seems to start off quickly (100th of a ms per query) but starts slowing down after that; sometimes to a second and a half per group query.
 
I don't think that its our AD servers not responding quick enough; we have a good number of them for the amount of users they service nd I have tried authenticating against each one with the same results. A few of them are only 2 hops away as well.
 
I read through the forums and the only issue I could find that resembled mine was a guy that had his network domain suffix different than the suffix in his LDAP settings; I made sure both of mine were the same.
 
I opened a ticket with JTAC and they took my policy traces, TCPDumps and user logs. They came back and said they seen winbind authentication failing. To fix this, I gave my service account domain admin privileges which stopped the error but didn't remediate the speed issue.
 
The weird thing is, if I take off the second factor auth (Entrust), logins are fast! This makes no sense to me because when I look at a policy trace from a two factor login, the Entrust portion usually finishes within 2-4 seconds and then passes authorization over to LDAP which is where it slows down.
 
I've already made our nested group searches 0 and killed off the reverse 'memberOf' search in the LDAP settings. I've also created a new realm with only one role mapping that seemed to take a long time.
 
I did update the OS on Friday from 6.4R3 to 6.4R4.1. This seemed to make the issue worse, so I rolled back. One thing I think I might try is to install 6.2R5 again and see if it kills our issue. Do you know if I can just upload an older OS the same way I upload a new OS, or does 6.4R3 need to be backwards compatible with 6.2R5? It should be since I updated to 6.4R3 from 6.2R5.
 
Does anyone have any suggestions, or know where to look next?

[spacyfreak]

Too much text, man!


Use IAS Radius instead and configure Radius Server on IVE.
Easy to implement, fast, stable, wonderfull.

[matfa]

Yeah I know, lol, but an issue like this requires that much text!

So you are saying to replace LDAP with IAS for authorization? Or to use it instead of AD for authentication?

Im planning on using IAS for 802.1x so that may be an option.

[spacyfreak]

Winbind is way to complicated, i dont trust this function.
I implemented for a customer. Works - but what when you do update of IVE?

Radius is an very stable protocol, and IVE does not have to be member server in your windows domain.
You can configure Windows Groups for Rolemapping very easy with IAS RAS Policies.
I use return attributes (attribute 25 class) which returns to IVE some value, for example a number like "1" to IVE when auth is successfull.
Then IVE can map a user who logs in to the proper role on Realm level.
Configuration is done in an hour, and no more headaches when you do IVE update.

You can install IAS on Domaincontroller(s) with some clicks, no add. license necessary.
Plus, when you use Radius Proxy, it scales very good and you can include any future domains into your IVE auth processing.


I do this for many years, and worked with all kind of auth methods IVE provides, but radius is best, try out and trust.

[spacyfreak]

Little Howto:

IAS Configuration:
1. Install IAS on member server or domaincontroller
2. on IAS add your IVE IP(s) as Radius Clients, with a radius secret
3. on IAS delete the default ras policies
4. confiugre ras policy with "windows group"
5. choose your windows group where users are member who should use IVE
6. on ras policy profile settings, advanced, delete default attributes
7. add attribute class (25) and give it a value like "vpn users"
8. add attribute "Ignore Dial-In Users properties" and give it value "true"


on IVE

1. create auth server radius
2. add your IAS Server IP and your radius secret
3. add backup IAS Server IP if you have one
4. on realm role mapping, configure
"if user attribute is class (25) with value "vpn users", assign role "vpn users"

Thats it.


Notes:

1. allways when you edit your ias configuration, restart ias radius service
2. use a long radius secret as it is included in encryption of user passwords
3. if you can not use domain local or global groups, create universal group
4. if you use several ras policies on ias, each policy should also include the ip of your ive system so this rule only matches when ive sends a radius request
5. use netsh command to backup or restore your radius config on ias
6. you can easily troubleshoot auth issues with radiu when you watch window server system eventlog


Enjoy!

[spacyfreak]

So you are saying to replace LDAP with IAS for authorization? Or to use it instead of AD for authentication?

Im planning on using IAS for 802.1x so that may be an option.

Yes, radius can do authentication AND autorization in one go.
You autorize the user with radius return attribute, for example class (25).
You can give on IAS any value to this attribute, like a text string or a number.
On IVE the value of this attribute when it comes to IVE while a user i logging in can map a user to a specific IVE user role.

Means..

1. User tries to log in with his credentials
2. IVE sends user credentials via radius to IAS server, password is encrypted
3. IAS does a ldap query and checks username, password and group membership
4. If credentials match, and the user is in the proper group, IAS sends "radius accept" packet to ive. Plus, in this packet radius sends attribute class with value "vpn users" to ive
5. IVE receives the accept message and the class value.
6. If the class value matches the value you configured on ive realm role mapping, user is mapped to the proper role
7. If user credentials are wrong or he is not in the proper windows group, radius sends "radus reject" message, and user is denied to log in

[matfa]

Im just looking at this now and I think IAS might be a headache for me. I have around 200 role mapping rules and it looks as if I would need to manually create each of these rules on the IAS server, is that correct? I would like to speed up the login process but I would also like to keep manual configuration to a minimum. However, I still may try this out if I'm not able to figure something out.

Have you ever tried installing an older operating system? At this point, my rollback wont go back to the OS that I want it to, so I am going to try installing 6.2R5 on top of 6.4R3, will this work?

[matfa]

I figured out how to downgrade the operating system and unfortunately, it's not pretty. I have to go into the upgrade/downgrade tab and upload the older OS image and click on the button that says "DELETES all system and user configuration". So it looks like if I do end up reverting the OS image, I will need to take a snapshot, upload the image and delete the configuration, then upload the settings once again.

[spacyfreak]

Is your AD Authentication slow - or is it the secondary auth that slows up the login?

200 role mapping rulez! BEHAVE!
I provide ive service for 800 users here and have 10 role mapping rulez.
What about STANDARDIZATION?
Most users use the "standard policy set" with full vpn connect with corporate client device (client certificates required), and for "special needs" i have another realm with "granualr policies for each requirement".
So my number of role mapping rulez is not so high.

How you do rolemapping? with usernames or what?
I think i would redesign your concept before going on this way..

You can book me, i am cheap!

[spacyfreak]

You could also evaluate to use variables to reduce your admin overhead, in rolemapping rules.

Why do you need so many rolemapping rulez? I am curious..

[matfa]

Yeah I know 200 is high but thats the best way I could figure it out. What I do is check a users group memberships to see what department they are a member of and that is how they I map their department drive to their homepage. If I had to do it by username, I think I would be here all week.  

How do you perform department specific mappings?

For the user drives, I scaled it down to three role mappings depending on which file server clusters the users mapped their user drive to. In the role mapping, I used a path like: \\fileserver\filepath\<username>

Is there some variable that would allow me to pull the users OU from AD, like: \\fileserver\departments\<department>

?  


Believe me, I would LOVE to get rid of all of these roles and standardize!

[matfa]

One other thing I forgot. Since I use dynamic VLANning internally, I assign user groups an internal virtual port so that the filtering at the firewall resembles their access they have while at work. I would say around 80% of the user groups get stuck in one default users group, because they do not have any special access. This could make it easier to group them all together, provided I figure something out for the department name variable.

I just created an exact replica of my SA4000s on an SA2000 so that I can screw with it. Wierd thing is that the issue doesnt seem to be as bad today. When i first got here it was taking up to a minute to get to the host checker screen, now its taking 10 seconds. I hate intermittent issues!

Spaceyfreak, I would hire you but I'm sure you would want more money. lol

[spacyfreak]

Oh, you use hostchecker.
I am happy that i did not implement.
I love it simple and stable.

Well ok, your work with variables, but 200 rulez is still a lot...

Well, i dont do ANY mappings to networkdrives via ive webportal, as the users have full vpn and can map their drives on windows exporer.

 the users have the option to map their drives via ive webportal as an additional option, if they want, if they dont want to map, i dont care. Muhahahaha.

But we have some thousand users with hundreds of fileserers and a dozend domains, i dont have the time to care of that, as long as company is not interested in standardization, i am not, too. hehe.

I use clientcertificates attributes and radius attributes mostly for rolemapping.
depending on CN in clientcert, the user is mapped to some role to get some special setup like proxy or different dns server on ive tunnel if needed.

[matfa]

Well I fixed it, somewhat. I didn't fix the slowness issue between the SA and LDAP but I killed off 120 role mappings to cut down on the amount of traffic to be processed by LDAP.

I used the userAttr.department variable to pull a users department name which allowed me to map a users department drive using only two different role mapping rules.

I would get rid of more roles, however I still need roles for departments that require a different virtual port than the "standard users" virtual port. Once I figure out a way to get this cut down, I should be able to cut it down to around 50 roles.

To cut it down even further, I think I am going to implement a second sign in page for any users that are not employees. This will cut it down at least another 15 - 20 roles.

The issue may still exist but if I don't need to pass that many packets out to the LDAP server, I don't think I need to worry about it.


You don't use host checker? You crazy mang! I need to know that an user coming into my network is using a secure PC that won't be spreading trojans/worms/malware/etc...
I don't have any issues with host checker per se; the only times there are issues, is when a new A/V suite comes out that is not in the latest ESAP file. Cache cleaner on the other hand is a pain in the ass. I don't know how many times I have had the helpdesk call me because a user went to the web page and the login boxes weren't there and the site had an error saying "You are not authorized to login, please contact your administrator".

Anyway, thanks for the info Spacyfreak.

[spacyfreak]

Well its a question of "philosophy".
Into  the network via network connect only come corporate client devices, they have to authenticate with clientcertificate, have restricted user rights on their notebooks, and a centralized antivirus domain solution.

So in my eyes chance to get an infect via remote access is not "bigger" then in the inside network itself, where 50000 people from all over the world "could" bring their private notebooks from home and connect them to the LAN "from inside".

[matfa]

Looks like I spoke too soon. I tried getting in on Saturday and I had to try five times before I finally was logged in. I called Juniper and dealt with their second teir of support who forwarded the ticket to their advanced team.

While doing some testing it became clear to me that limiting the number of roles is not going to fix the issue. I have a second realm that only has one role mapping that says "if username = user1, map role1". This mapping doesnt even need to do an LDAP query and yet it still times out and doesnt allow the user to log in. Based on this information, it seems to be that LDAP isn't the issue at all.

I still don't see how the issue disappears when we disable our secondary authentication server. When this is enabled, it only takes 5-10 seconds to authenticate the user and then passes the authentication over to the AD server which then goes slow. Doesn't make any sense.

[spacyfreak]

What happens when you do a new  test realm, and activate your second. auth solution there as primary auth?`
I bet its still slow.
so your issue is the slow process on your second auth solution!

[matfa]

Yepper, it is still slow after using a test realm. This is on both my production cluster and my test SA2000 (which is a direct copy of my prod boxes).

I would also think that it's the Entrust servers causing the issue but when I look at a policy trace, there are times when the Entrust server authenticates a user right away and then passes the auth back to the SA which then times out during LDAP retrieval.

This is too weird, dont know if its the SA or the Etrust server now. Guess I'll wait and see what the JTAC advanced team says.

[matfa]

Hey guys, just thought I would reply to my thread to let you know how this was fixed in case someone else encounters this.

The actual issue appears to be that the LDAP query coming from the IVE was no "memberOf" even though I had selected memberOf in the LDAP authentication server setup. I was performing some captures from the firewall and noticed that for every login, there was 43000bytes of data going back and forth between the IVE and the DCs. Looking deeper into the packets, I could see that the IVE was doing one request for each role mapping and since I had 180 roles when this started, the IVE would time out the login.

I updated my test SA2000 to 6.5R3.1 and noticed that a login only produced 3700bytes of data because it was using memberOf which caused the DCs to send back one packet with all of the users group memberships in AD. I updated our prod SA4000 cluster and voila! This not only sped up logins but also general browsing on the IVE page.

Thanks for the suggestions Spacey.

•      Forum/Home
•      Store
•      Articles
•        Tech Articles
•        Industry News
•        Site News
•      Knowledgebase
•        JUNOS/Routing
•        NS/ISG/SSG/NSM
•        SSL VPN
•      Downloads

Please consider donating if we've saved you time or money. It helps pay for the bandwidth, equipment, and hosting charges to keep this site running

Submit Article/KB - Do not submit questions here.

[Today]

• Multicast routing enigma.... by Di4bLo [Today at 03:31:19 AM]
• Switching and forwarding ... by rafibd71 [Yesterday at 10:51:57 PM]
• Juniper consultant needed... by wiredinoc [Yesterday at 05:49:10 PM]
• SSG 140 unable to Remote ... by hrz [Yesterday at 05:40:18 PM]
• Changing PBR in case of f... by hrz [Yesterday at 05:32:38 PM]
• Lost management - all mgm... by hrz [Yesterday at 05:28:31 PM]
• Netscreen 5xp by timeshadowrider [Yesterday at 03:11:08 PM]
• Dynamic VPN cannot ever c... by drozymandias [Yesterday at 03:08:05 PM]
• SNMP on SSG20 by archigos [Yesterday at 02:44:47 PM]
• Limit Access to folder th... by Segment [Yesterday at 12:51:22 PM]

Members

• Total Members: 20493
• Latest: prabhakar_bansode

Stats

• Total Posts: 38020
• Total Topics: 9820
• Online Today: 51
• Online Ever: 393
• (August 06, 2008, 07:40:57 AM)

Users Online

Users: 7
Guests: 43
Total: 50

Pascal
AceCandies
Snok
rafibd71
Budak
prabhakar_bansode
dominic.crawford