postfix -> dovecot via lmtp using xoauth2 can't find user reddit / selfhosted ---------------------------------------------------------------------- Torxed · 2023-10-18 09:58 Hi! I am not sure why but I'm sure it's something stupidly easy once you spot it. But I've been going on for days now with this and hoping someone had a similar issue. Setup is: * postfix * Using `dovecot` as auth service * Using `dovecot` via lmtp for mail delivery * dovecot * Using `keycloak` as auth backend Authentication directly via IMAP appears to be working. Mail is accepted on port 25 to the configured domain, but handover and actual delivery to the user does not work. I'm not sure what configurations are in play here, so I'll post the ones I think is relevant. And here is the issue: ``` mail-postfix-1 | Oct 18 10:07:24 domain postfix/smtpd[109]: connect from _gateway[172.26.0.1] mail-postfix-1 | Oct 18 10:07:26 domain postfix/smtpd[109]: 1E85829FFFD: client=_gateway[172.26.0.1] mail-postfix-1 | Oct 18 10:07:27 domain postfix/cleanup[114]: 1E85829FFFD: message-id= mail-postfix-1 | Oct 18 10:07:27 domain postfix/qmgr[80]: 1E85829FFFD: from=, size=341, nrcpt=1 (queue active) mail-postfix-1 | Oct 18 10:07:27 domain postfix/local[124]: 1E85829FFFD: passing to transport=lmtp mail-dovecot-1 | Oct 18 10:07:27 lmtp(15): Info: Connect from 172.26.0.5 mail-dovecot-1 | Oct 18 10:07:27 auth: Error: All password databases were skipped mail-dovecot-1 | Oct 18 10:07:27 lmtp(torxed@domain.se): Error: auth-master: userdb lookup(torxed@domain.se): Auth USER lookup failed mail-dovecot-1 | Oct 18 10:07:27 lmtp(15): Error: lmtp-server: conn 172.26.0.5:54040 [1]: rcpt torxed@domain.se: Failed to lookup user torxed@domain.se: Internal error occurred. Refer to server log for more information. mail-dovecot-1 | Oct 18 10:07:27 auth: Debug: master in: USER 1 torxed@domain.se service=lmtp lip=172.26.0.2 lport=24 rip=172.26.0.5 rport=54040 mail-dovecot-1 | Oct 18 10:07:27 auth: Debug: static(torxed,172.26.0.5): Performing userdb lookup mail-dovecot-1 | Oct 18 10:07:27 auth: Debug: skipping passdb: mechanism filtered mail-dovecot-1 | Oct 18 10:07:27 auth: Debug: skipping passdb: mechanism filtered mail-dovecot-1 | Oct 18 10:07:27 auth: Debug: static(torxed,172.26.0.5): Finished userdb lookup mail-dovecot-1 | Oct 18 10:07:27 auth: Debug: userdb out: FAIL 1 mail-postfix-1 | Oct 18 10:07:27 domain postfix/lmtp[125]: 1E85829FFFD: to=, relay=dovecot[172.26.0.2]:24, delay=2.6, delays=2.5/0.01/0.02/0.01, dsn=4.3.0, status=deferred (host dovecot[172.26.0.2] said: 451 4.3.0 Temporary internal error (in reply to RCPT TO command)) mail-dovecot-1 | Oct 18 10:07:27 lmtp(15): Info: Disconnect from 172.26.0.5: Logged out (state=READY) mail-postfix-1 | Oct 18 10:07:29 domain postfix/smtpd[109]: disconnect from _gateway[172.26.0.1] ehlo=1 mail=1 rcpt=1 data=1 quit=1 commands=5 ``` My postfix `master.cf`: ``` maillog_file = /dev/stdout always_add_missing_headers = yes local_header_rewrite_clients = permit_inet_interfaces virtual_transport = lmtp:dovecot:24 mailbox_transport = lmtp:dovecot:24 smtpd_tls_protocols = !SSLv2,!SSLv3,!TLSv1,!TLSv1.1 compatibility_level = 3.7 smtp_tls_security_level = may smtpd_use_tls = yes smtpd_tls_cert_file = /etc/postfix/fullchain.pem smtpd_tls_key_file = /etc/postfix/privkey.pem #smtp_header_checks = regexp:/etc/postfix/smtp_header_checks #mime_header_checks = regexp:/etc/postfix/smtp_header_checks #header_checks = regexp:/etc/postfix/smtp_header_checks #policy-spf_time_limit = 60s smtpd_recipient_restrictions = permit_sasl_authenticated reject_unauth_destination permit_mynetworks # check_policy_service unix:private/policy-spf #non_smtpd_milters=unix:/run/dkimpy-milter/milter.socket #smtpd_milters=unix:/run/dkimpy-milter/milter.socket queue_directory = /var/spool/postfix command_directory = /usr/bin daemon_directory = /usr/lib/postfix/bin data_directory = /var/lib/postfix mail_owner = postfix myhostname = domain.se mydomain = domain.se myorigin = localhost inet_interfaces = all mydestination = domain.se unknown_local_recipient_reject_code = 450 alias_maps = hash:/etc/postfix/aliases virtual_alias_maps = hash:/etc/postfix/virtual virtual_alias_domains = localhost alias_database = $alias_maps home_mailbox = Maildir/ smtpd_banner = $myhostname ESMTP debug_peer_level = 2 debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5 sendmail_path = /usr/bin/sendmail newaliases_path = /usr/bin/newaliases mailq_path = /usr/bin/mailq setgid_group = postdrop html_directory = no manpage_directory = /usr/share/man sample_directory = /etc/postfix readme_directory = /usr/share/doc/postfix inet_protocols = ipv4 shlib_directory = /usr/lib/postfix meta_directory = /etc/postfix disable_vrfy_command = yes smtpd_helo_required=yes smtp_tls_loglevel=1 smtpd_tls_loglevel=1 ``` `master.cf`: ``` smtp inet n - n - - smtpd #policy-spf unix - n n - 0 spawn # user=nobody argv=spfcheck sender smtps inet n - n - - smtpd -o syslog_name=postfix/smtps -o smtpd_tls_security_level=encrypt -o smtpd_sasl_type=dovecot -o smtpd_sasl_path=inet:dovecot:12345 -o smtpd_sasl_security_options=noanonymous -o smtpd_sasl_local_domain=$myhostname -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes -o smtpd_reject_unlisted_recipient=no -o smtpd_client_restrictions= -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_relay_restrictions= -o smtpd_recipient_restrictions=reject_non_fqdn_recipient,reject_unk nown_recipient_domain -o smtpd_relay_restrictions=permit_sasl_authenticated,reject -o milter_macro_daemon_name=ORIGINATING ``` My main dovecot files, `10-maildir.conf`: ``` mail_location = maildir:%h/Maildir namespace inbox { inbox = yes } ``` `10-lmtp.conf`: ``` service lmtp { user = vmail inet_listener lmtp { port = 24 } } protocol lmtp { #mail_plugins = $mail_plugins sieve mail_plugins = $mail_plugins } ``` `10-auth.conf`: ``` auth_mechanisms = $auth_mechanisms oauthbearer xoauth2 auth_realms = domain.se auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@+ auth_username_format = %Ln auth_worker_max_count = 10 auth_failure_delay = 2 secs auth_ssl_require_client_cert = no auth_ssl_username_from_cert = no auth_mechanisms = plain auth_debug = yes auth_verbose = yes passdb { driver = oauth2 mechanisms = oauthbearer xoauth2 args = /etc/dovecot/dovecot-oauth2.token.conf.ext } passdb { driver = oauth2 mechanisms = plain login args = /etc/dovecot/dovecot-oauth2.plain.conf.ext } userdb { driver = static args = uid=vmail gid=vmail home=/home/vmail/%u } ``` Then I have two `.ext` files for dovecot governing the `xoauth2`: `dovecot-oauth2.plain.conf.ext`: ``` openid_configuration_url = http://sso.domain.se:8080/realms/domain/.well-known/openid- configuration grant_url = http://sso.domain.se:8080/realms/domain/protocol/openid- connect/token client_id = dovecot client_secret = a-secret introspection_url = http://sso.domain.se:8080/realms/domain/protocol/openid- connect/token/introspect introspection_mode = post use_grant_password = yes debug = yes username_attribute = email pass_attrs = pass=%{oauth2:access_token} # %Lu # Lowercases the username # # %n # Drops the domain if one was supplied # # %n-AT-%d # Changes the “@” symbol into “-AT-” before lookup username_format = %Ln # username_format = %Lu active_attribute = active force_introspection = yes active_value = true rawlog_dir = /tmp/oauth2 ``` `dovecot-oauth2.token.conf.ext`: ``` openid_configuration_url = http://sso.domain.se:8080/realms/domain/.well-known/openid- configuration grant_url = http://sso.domain.se:8080/realms/domain/protocol/openid- connect/token client_id = dovecot client_secret = a-secret tokeninfo_url = http://sso.domain.se:8080/realms/domain/protocol/openid- connect/userinfo introspection_url = http://sso.domain.se:8080/realms/domain/protocol/openid- connect/token/introspect introspection_mode = post use_grant_password = no debug = yes username_attribute = email pass_attrs = pass=%{oauth2:access_token} username_format = %Ln active_attribute = active force_introspection = yes active_value = true rawlog_dir = /tmp/oauth2 ``` I've configured keycloak with a domain, client and a test user for that domain. ---------------------------------------------------------------------- Torxed · 2023-10-19 14:22 Good enough for my test case, but surely not reliable for real world scenarios as this opens up delivery to any given destination.. but adding `allow_all_users=yes` to the userdb at least enables me to debug the thing I setup this to debug. Ideally `dovecot` would have support for `xauth` for lmtp as well, but they currently don't: https://doc.dovecot.org/configuration_manual/aut hentication/user_databases_userdb/ > ## Currently supported user databases are: > * Passwd: System users (NSS, /etc/passwd, or similar). See Passwd. > * Passwd-file: /etc/passwd-like file in specified location. See Passwd-file. > * NSS: Name Service Switch. See NSS. > * LDAP: Lightweight Directory Access Protocol. See LDAP > * SQL: SQL database (PostgreSQL, MySQL, SQLite). See SQL > * Dict: Dict key-value database (Redis, memcached, etc.). See Key- value authentication (dict) database. > * Static: Userdb information generated from a given template. See Static User Database. > * VPopMail: External software used to handle virtual domains. > * Prefetch: This assumes that the passdb already returned also all the required user database information. See Prefetch User Database > * Lua: Lua script for authentication. See Lua based authentication. > New in version v2.3.0. Until they do, `lua` would be the closest I guess to manually query the keycloak/OAuth2 for user-info. But again, the following ended up being the solution to get going with the real tests that I wanted to do: ``` userdb { driver = static args = uid=vmail gid=vmail home=/home/vmail/%u allow_all_users=yes } ```