Issue #350 still incorrectly states that you can't do start_tls over ldapi

There is no meaningful way to do TLS over LDAPI. Either handshake fails or TLS connection cannot be verified.

With 389-DS:

>> import ldap
>>> conn = ldap.initialize("ldapi://%2Frun%2Fslapd-IPA-TEST.socket")
>>> conn.start_tls_s()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.9/site-packages/ldap/ldapobject.py", line 882, in start_tls_s
    return self._ldap_call(self._l.start_tls_s)
  File "/usr/lib64/python3.9/site-packages/ldap/ldapobject.py", line 340, in _ldap_call
    reraise(exc_type, exc_value, exc_traceback)
  File "/usr/lib64/python3.9/site-packages/ldap/compat.py", line 46, in reraise
    raise exc_value
  File "/usr/lib64/python3.9/site-packages/ldap/ldapobject.py", line 324, in _ldap_call
    result = func(*args,**kwargs)
ldap.CONNECT_ERROR: {'result': -11, 'desc': 'Connect error', 'ctrls': []}

With OpenLDAP 2.4.57

>>> l = self.ldap_object_class(self.server.ldapi_uri)
>>> l.set_option(ldap.OPT_X_TLS_CACERTFILE, self.server.cafile)
>>> l.set_option(ldap.OPT_X_TLS_NEWCTX, 0)
>>> l.start_tls_s()
ldap.CONNECT_ERROR: {'result': -11, 'desc': 'Connect error', 'ctrls': [], 'info': 'TLS: hostname does not match name in peer certificate'}

With OpenLDAP 2.4.57 and custom LDAP_OPT_HOST_NAME:

...
>>> l.set_option(ldap.OPT_HOST_NAME, self.server.hostname)
>>> l.start_tls_s()
ldap.SERVER_DOWN: {'result': -1, 'desc': "Can't contact LDAP server", 'errno': 107, 'ctrls': [], 'info': 'Transport endpoint is not connected'}

There is no meaningful way to do TLS over LDAPI. Either handshake fails or TLS connection cannot be verified.

This is literally something that's tested as a part of the OpenLDAP test suite as I noted in the issue.

It's entirely possible to do it with other APIs as well, such as Net::LDAP which doesn't even use the OpenLDAP libraries. Example:

cat starttls-over-ldapi.pl
#!/usr/bin/perl

use strict;
use Net::LDAP;

my $ldap = Net::LDAP->new('ldapi://%2fvar%2fsymas%2frun%2fldapi/') or die "$@";

my $mesg = $ldap->start_tls(
                                        verify => 'require',
                                        cafile => "/opt/symas/ssl/cacert.pem",
                                        sslserver => "ldap.symas.net",
                                        ) or die "start_tls: $@";

From the slapd log file when this is executed:

Sep 17 18:40:32 solsv01mpc7 slapd[1276]: conn=344841 fd=31 ACCEPT from PATH=/var/symas/run/ldapi (PATH=/var/symas/run/ldapi)
Sep 17 18:40:32 solsv01mpc7 slapd[1276]: conn=344841 op=0 EXT oid=1.3.6.1.4.1.1466.20037
Sep 17 18:40:32 solsv01mpc7 slapd[1276]: conn=344841 op=0 STARTTLS
Sep 17 18:40:32 solsv01mpc7 slapd[1276]: conn=344841 op=0 RESULT oid= err=0 duration=0.138ms text=
Sep 17 18:40:32 solsv01mpc7 slapd[1276]: conn=344841 fd=31 TLS established tls_ssf=256 ssf=256

So yes, it is possible to do startTLS over ldapi if the client is configured correctly.

It's entirely possible to do it with other APIs as well, such as Net::LDAP which doesn't even use the OpenLDAP libraries.
So yes, it is possible to do startTLS over ldapi if the client is configured correctly.

Would you care to explain how to do TLS with cert validation over LDAPI using OpenLDAP client libraries and python-ldap?

It's entirely possible to do it with other APIs as well, such as Net::LDAP which doesn't even use the OpenLDAP libraries.
So yes, it is possible to do startTLS over ldapi if the client is configured correctly.

Would you care to explain how to do TLS with cert validation over LDAPI using OpenLDAP client libraries and python-ldap?

So I realized my initial comment wasn't exactly clear. The OpenLDAP client libraries already support this, and they are tested via the OpenLDAP test suite via the use of the command line tools (ldapwhoami, etc) which make use of the OpenLDAP client libraries. I'll see if I can dig into what's necessary for python-ldap to correctly utilize the libraries.

@tiran it works fine for me:

>>> import ldap
>>> url="ldapi://%2fvar%2fsymas%2frun%2fldapi/"
>>> conn = ldap.initialize(url)
>>> conn.set_option(ldap.OPT_X_TLS_CACERTFILE, "/opt/symas/ssl/CA/certs/testsuiteCA.crt")
>>> conn.set_option(ldap.OPT_HOST_NAME, "ub18.quanah.org")
>>> conn.set_option(ldap.OPT_X_TLS_NEWCTX, 0)
>>> conn.start_tls_s()

Slapd logs:

Sep 21 15:33:05 ub18 slapd[5618]: conn=1004 fd=13 ACCEPT from IP=192.168.1.24:53512 (IP=0.0.0.0:389)
Sep 21 15:33:05 ub18 slapd[5618]: conn=1004 op=0 EXT oid=1.3.6.1.4.1.1466.20037
Sep 21 15:33:05 ub18 slapd[5618]: conn=1004 op=0 STARTTLS
Sep 21 15:33:05 ub18 slapd[5618]: conn=1004 op=0 RESULT oid= err=0 qtime=0.000087 etime=0.001012 text=
Sep 21 15:33:05 ub18 slapd[5618]: conn=1004 fd=13 TLS established tls_ssf=256 ssf=256 tls_proto=TLSv1.3 tls_cipher=TLS_AES_256_GCM_SHA384

@quanah Nope, fails on my machine and on all our CI systems, #433

@tiran

I've expanded my script, still works even after setting demand. I don't believe the second test case will ever succeed, since the OPT_HOST_NAME value must be set to a DNS: value of subjectAltName in the server certificate.

I can only speculate that there's an issue with the ldap libraries caused by some type of custom patch that's not present in stock OpenLDAP (which is what I'm testing against), or that some of the values in your test are not what they should be. For example, if the value of self.server.hostname does not match anything in the subjectAltName DNS field of the certificate on the server.

For the cert I'm using:

        Subject: C = US, ST = CA, O = OpenLDAP Foundation, OU = OpenLDAP Test Suite, CN = ub18.quanah.org
            X509v3 Subject Alternative Name:
                DNS:localhost, DNS:ub18.quanah.org, IP Address:127.0.0.1, IP Address:0:0:0:0:0:0:0:1

Here's my updated script based off your test suite, which continues to work:

>>> import ldap
>>> url="ldapi://%2fvar%2fsymas%2frun%2fldapi/"
>>> conn = ldap.initialize(url)
>>> conn.set_option(ldap.OPT_X_TLS_CACERTFILE, "/opt/symas/ssl/CA/certs/testsuiteCA.crt")
>>> conn.set_option(ldap.OPT_HOST_NAME, "ub18.quanah.org")
>>> conn.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_DEMAND)
>>> conn.set_option(ldap.OPT_X_TLS_NEWCTX, 0)
>>> conn.start_tls_s()
>>> conn.whoami_s()
''
>>> conn.unbind()
>>> quit()

The test case fails on both Debian/Ubuntu and on Fedora. We cannot support a feature that we are not able to test and that does not work for majority of our user base. Most of our users are either on Debian-based systems (including Ubuntu) or on Fedora-based systems (including CentOS and RHEL) with OpenLDAP 2.4.57 or older.

We also don't have resources to support the full feature set of libldap. python-ldap is an opinionated wrapper that supports a popular subset of OpenLDAP's features. So far there hasn't been any request for TLS over LDAPI except from you. I suggest that you open a feature request for TLS over LDAPI so the issue doesn't get lost.

@tiran As I've noted multiple times now, there is literally nothing special about this functionality. It's not new, it's been around for years, and it's fully supported by libldap. Not only that, as I've repeatedly demonstrated, it's already fully supported by python-ldap as well.

I spent time this morning setting up an centos8 system using the python3-ldap shipped with centos8 for python3.6, and was trivially able to confirm that the ability to do startTLS + ldapi is currently supported with the system libldap shipped in centos8.

I spent time this morning also setting up an ubuntu20 system using the python3-ldap shipped with ubuntu20 for python3.8, and was trivially able to confirm that the ability to do startTLS + ldapi is currently supported with the system libldap shipped in Ubuntu20.

CentOS8:

>>> import ldap
>>> url="ldapi://%2fvar%2fsymas%2frun%2fldapi/"
>>> conn = ldap.initialize(url)
>>> conn.set_option(ldap.OPT_X_TLS_CACERTFILE, "/opt/symas/ssl/CA/testsuiteCA.crt")
>>> conn.set_option(ldap.OPT_HOST_NAME, "127.0.0.1")
>>> conn.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_DEMAND)
>>> conn.set_option(ldap.OPT_X_TLS_NEWCTX, 0)
>>> conn.start_tls_s()
>>> conn.whoami_s()
''
>>> conn.unbind()

Ubuntu20:

>>> import ldap
>>> url="ldapi://%2fvar%2fsymas%2frun%2fldapi/"
>>> conn = ldap.initialize(url)
>>> conn.set_option(ldap.OPT_X_TLS_CACERTFILE, "/opt/symas/ssl/CA/testsuiteCA.crt")
>>> conn.set_option(ldap.OPT_HOST_NAME, "127.0.0.1")
>>> conn.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_DEMAND)
>>> conn.set_option(ldap.OPT_X_TLS_NEWCTX, 0)
>>> conn.start_tls_s()
>>> conn.whoami_s()
''
>>> conn.unbind()
>>>
>>> quit()

As you can see, no issues on either server. Since this was inside a container that had no valid host, I had to set the hostname to the localhost IP, which is in the subjectAltName value of the cert I previously listed.

So, again, this clearly works across the board, and the issue being hit with the test suite is specific to the test suite setup. I've no visibility into the cert being used by the test suite or the way in which slapd is configured to run via the test suite, so I can't help further there. But there is zero issue with python-ldap doing startTLS + ldapi.

the 3 PRs are postponed to the next version.