LDAP support on Rancher v0.35.0

Hello,

When I try to setup the LDAP authentication (on port TCP/389 without TLS) and test it in rancher I obtain the following error :

2015-09-01 16:34:52,058 INFO  [:] [] [] [] [tp1873792729-14] [i.a.a.i.l.LdapIdentitySearchProvider] Failed to login to ldap user:mydomain.com\rancher javax.naming.AuthenticationException: [LDAP: error code 49 - 80090308: LdapErr: DSID-0C0903A9, comment: AcceptSecurityContext error, data 52e, v1db1]
at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3087) ~[na:1.7.0_79]
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3033) ~[na:1.7.0_79]
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2835) ~[na:1.7.0_79]

This error ([LDAP: error code 49 - 80090308: LdapErr: DSID-0C0903A9, comment: AcceptSecurityContext error, data 52e, v1db1]) seem indicate the bond credential is not good, but I’ve verified from another program (Apache Directory Studio) without any issue.

How to fix this ?

Based on the error, it looks like it’s invalid credentials.

https://confluence.atlassian.com/display/JIRAKB/LDAP+Error+Code+49

Can you try again?

That’s what I’ve found, but I’ve double checked my credentials and that’s ok.
I’ve also tried two forms for “Service Account Username”:

  • CN=rancher,CN=Users,DC=mydomain,DC=com
  • rancher

But same result with both of them.
For info, for “Search base” I have enter : CN=Users,DC=mydomain,DC=com

Please find the full stack trace if this could help :

2015-09-01 16:34:52,058 INFO  [:] [] [] [] [tp1873792729-14] [i.a.a.i.l.LdapIdentitySearchProvider] Failed to login to ldap user:mydomain.com\rancher javax.naming.AuthenticationException: [LDAP: error code 49 - 80090308: LdapErr: DSID-0C0903A9, comment: AcceptSecurityContext error, data 52e, v1db1]
at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3087) ~[na:1.7.0_79]
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3033) ~[na:1.7.0_79]
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2835) ~[na:1.7.0_79]
at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2749) ~[na:1.7.0_79]
at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:316) ~[na:1.7.0_79]
at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:193) ~[na:1.7.0_79]
at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:211) ~[na:1.7.0_79]
at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:154) ~[na:1.7.0_79]
at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:84) ~[na:1.7.0_79]
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684) ~[na:1.7.0_79]
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307) ~[na:1.7.0_79]
at javax.naming.InitialContext.init(InitialContext.java:242) ~[na:1.7.0_79]
at javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:153) ~[na:1.7.0_79]
at io.cattle.platform.iaas.api.auth.integration.ldap.LdapIdentitySearchProvider.login(LdapIdentitySearchProvider.java:151) [cattle-iaas-auth-logic-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.iaas.api.auth.integration.ldap.LdapIdentitySearchProvider.getIdentities(LdapIdentitySearchProvider.java:223) [cattle-iaas-auth-logic-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.iaas.api.auth.integration.ldap.LdapTokenCreator.getLdapToken(LdapTokenCreator.java:49) [cattle-iaas-auth-logic-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.iaas.api.auth.integration.ldap.LdapTokenCreator.getToken(LdapTokenCreator.java:64) [cattle-iaas-auth-logic-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.iaas.api.auth.identity.TokenResourceManager.createToken(TokenResourceManager.java:65) [cattle-iaas-auth-logic-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.iaas.api.auth.identity.TokenResourceManager.createInternal(TokenResourceManager.java:54) [cattle-iaas-auth-logic-0.5.0-SNAPSHOT.jar:na]
at io.github.ibuildthecloud.gdapi.request.resource.impl.AbstractBaseResourceManager.create(AbstractBaseResourceManager.java:115) [cattle-framework-java-server-0.5.0-SNAPSHOT.jar:na]
at io.github.ibuildthecloud.gdapi.request.handler.ResourceManagerRequestHandler.generate(ResourceManagerRequestHandler.java:39) [cattle-framework-java-server-0.5.0-SNAPSHOT.jar:na]
at io.github.ibuildthecloud.gdapi.request.handler.AbstractResponseGenerator.handle(AbstractResponseGenerator.java:14) [cattle-framework-java-server-0.5.0-SNAPSHOT.jar:na]
at io.github.ibuildthecloud.gdapi.request.handler.write.DefaultReadWriteApiDelegate.handle(DefaultReadWriteApiDelegate.java:27) [cattle-framework-java-server-0.5.0-SNAPSHOT.jar:na]
at io.github.ibuildthecloud.gdapi.request.handler.write.DefaultReadWriteApiDelegate.write(DefaultReadWriteApiDelegate.java:22) [cattle-framework-java-server-0.5.0-SNAPSHOT.jar:na]
at sun.reflect.GeneratedMethodAccessor469.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_79]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) [spring-aop-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) [spring-aop-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) [spring-aop-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:96) [spring-tx-3.2.4.RELEASE.jar:3.2.4.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:260) [spring-tx-3.2.4.RELEASE.jar:3.2.4.RELEASE]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94) [spring-tx-3.2.4.RELEASE.jar:3.2.4.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91) [spring-aop-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) [spring-aop-3.2.5.RELEASE.jar:3.2.5.RELEASE]
at com.sun.proxy.$Proxy41.write(Unknown Source) [na:na]
at io.github.ibuildthecloud.gdapi.request.handler.write.ReadWriteApiHandler.handle(ReadWriteApiHandler.java:19) [cattle-framework-java-server-0.5.0-SNAPSHOT.jar:na]
at io.github.ibuildthecloud.gdapi.servlet.ApiRequestFilterDelegate.doFilter(ApiRequestFilterDelegate.java:86) [cattle-framework-java-server-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.api.servlet.ApiRequestFilter$1.runInContext(ApiRequestFilter.java:60) [cattle-framework-api-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:108) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) [cattle-framework-managed-context-0.5.0-SNAPSHOT.jar:na]
at io.cattle.platform.api.servlet.ApiRequestFilter.doFilter(ApiRequestFilter.java:53) [cattle-framework-api-0.5.0-SNAPSHOT.jar:na]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83) [jetty-servlets-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300) [jetty-servlets-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) [jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) [jetty-security-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) [jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.Server.handle(Server.java:499) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) [jetty-io-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]

Is mydomain.com (or the other one in the first stack trace, which you may want to remove if you were trying to anonymize it :slight_smile: ) the actual name of the LDAP domain? As in if you were logging into a PC, you’d type mydomain.com\rancher as the username?

If so you want:

  • Default Login Domain: mydomain.com
  • Search Base: dc=mydomain,dc=com (you can restrict this further later)
  • Service Account Username: rancher , or anotherdomain\rancher if it is not in mydomain.com (the default login domain)
  • Your Username: yourusername, or anotherdomain\yourusername if you are not in mydomain.com.

The rest of the settings aren’t relevant until authenticating works.

Of course “my domain.com” is not the real domain name :smile:
I’ve replaced in the stack trace and message.

From any windows session, yes we log in by using “mydomain.com\username”.

rancher (LDAP binding account) is also member of mydomain.com

@wizardofmath Can you try a domain with a dot in it, that seems like the obvious thing that might be special about @Bastien_Vigneron’s setup.

Ok, by analyzing the network traffic between ranger_server and active directory server, it seem ranger have a small issue : they don’t use the binding info before trying to authenticate user.

Our AD server is configured to deny any unbinded LDAP request, then we got an “invalidCredentials” error each time.

The issue is, the informations provided is section “1. Configure an LDAP server” especially “SERVICE ACCOUNT USERNAME*” and “SEARCH BASE*” is never used during the authentication request to the AD server.

Rancher try to just use the credentials provided in section “3. Test and enable authentication”.

The service account is used for various things like searching for users (when editing Environments) or to lookup what groups a user is a member of when using an API key (vs direct login through the UI). Basically any time we need something from LDAP other than login, because we don’t save the logged-in user’s credentials anywhere after login.

You are correct that they are not used when verifying a user login (which is what happens when you click that button before we turn access control enforcing on and potentially lock you out). That should be opening a connection and then immediately binding with the “your username” and password. So I’m not sure what is being requested un-bind-ed, but we will look into that.

Thank you Vincent.
Most Active Directory servers in enterprise are configured to reject “non binded” LDAP requests, even if this is just an authentication request.
That’s a part of traditional security policies.
That’s why all softwares (non Kerberos compatible) we use in my company establish a primary LDAP binding connection (with the service account) and then could authenticate potential users.

Anyway, keep me posted :wink:

FYI I added this to Github issues so we don’t lose track of it… https://github.com/rancher/rancher/issues/1952

Thank you Vincent, will keep one eye on that.

@Bastien_Vigneron @wizardofmath is looking into this, but do you know how we can configure AD to configure it the way you’re talking about? We don’t really know what we’re doing with Windows administration. :smile:

To clarify, are you saying we need to bind as the service account user, then bind again inside the same connection as the regular person user? The first request in all the LDAP connections is binding right now, but there’s one request using the service account and a separate one using the user account.

Hello Vincent, Which version of Windows Server did you use it ?

You can find a detailed documentation here : https://technet.microsoft.com/en-us/library/cc755809(v=ws.10).aspx

Since Windows 2008, only LDAP v3 is supported with default config.
On LDAP v3, the referral mechanism is activated by default.

We (@wizardofmath) setup a single Windows Server 2012 VM running AD with basically all the default config and that’s what was developed against.

I know forests and domain controllers and replication and all that exist as concepts but we are pretty much all Linux/Mac people and know really nothing about Windows server administration, so I don’t know how realistic of a test case our 1 VM is, or how to make a more realistic setup.

The only thing I see in that doc that sounds maybe relevant is:

If no bind is performed on the originating query server, or if a simple (plaintext) bind is performed on the originating server without the use of SSL encryption to protect the passwords, the LDAP client performs an anonymous bind to the referral server. The latter behavior protects passwords from being sent inadvertently over the network in plaintext as the result of chasing a referral.

We’re definitely doing simple bind, and SSL is optional… Is there a referral server involved on your side? Does enabling SSL change anything?

We are in a Forest configuration with multiples Windows 2008 Servers.
Then referal is in place.

Has this been resolved? I cannot get my Rancher setup to authenticate with our LDAP using the same settings that work for other tools such as JIRA, GitLab and Jenkins.

This also seems to be a problem still for still… now running v1.1.0-dev3

domain has ‘.’ and failing to auth.

domain functional level is 2008r2.

Is there an easy way to debug this?