Page 1 of 1
Client certificates revoked after upgrade to 2.11
Posted: Wed Aug 24, 2022 8:01 am
by chort1
After our last server patching (RHEL8), Access server was upgraded to version 2.11. After this, several users are getting error message "Client certificate has been revoked" when trying to log in via the client.
Checking the admin panel, I can see that 70-80% of the user profiles are gone. If the users log on to the web GUI and create a new profile, they can log in with the client again, using the new profile, though that's far from ideal.
Is this a known issue? It coincides quite heavily with the upgrade, but if there's any type of troubleshooting that might prove otherwise, I'd very much like to investigate.
Edit: I can add that this is a standalone server, but uses MariaDB (running locally) as backend.
Re: Client certificates revoked after upgrade to 2.11
Posted: Wed Aug 24, 2022 8:09 am
by openvpn_inc
Hello chort1,
Sounds like something in your environment went wrong. This is definitely not a known issue, it shouldn't happen. But there may be something in the database backend going wrong, but without more details I simply don't know. Since you rely on an external database backend I hope you have backups there. If so you could consider a rollback and restoring an older copy of the database, so that you can then have everyone connect again without issues, and then set up a test environment to run the upgrade there and see if any issues show up there and discuss them with our technical support guys on our support ticket system.
Kind regards,
Johan
Re: Client certificates revoked after upgrade to 2.11
Posted: Wed Aug 24, 2022 9:39 am
by chort1
Hi Johan, and thanks for the quick reply.
Am I not correct in assuming that MySQL/MariaDB backend is a supported configuration? It's been working fine up until now, and it seems strange that a faulty db backend would remove only some of the users... I guess we can dump the as_certs database and restore it, though users can fix the issue by creating new profiles for themselves, if this is a one time occurrence. We have a patch window for another (similar) Access server installation tonight, so we'll see how that goes.
Re: Client certificates revoked after upgrade to 2.11
Posted: Wed Aug 24, 2022 11:15 am
by chort1
Some more information, after a bit of troubleshooting
It seems that after this incident, some users have had the serial number on their profiles changed.
Example:
Yesterday Bob's profile was number 108, and today it was 44.
Yesterday Alice's profile was number 44, and today it was gone
When Alice tries to sign on, it fails, and we get a log entry that says:
Code: Select all
username-only match fail, client username=\alice\', DB username=\'bob\'
Re: Client certificates revoked after upgrade to 2.11
Posted: Wed Aug 24, 2022 12:21 pm
by openvpn_inc
Hi,
What benefit do you see from using mariadb over sqlite3? I would not think there is any. It gives you another moving part which can break.
What was the version you upgraded from, was it 2.8.x or earlier? We had a schema change in 2.9 (same schema through 2.11).
Support ticket is a good idea. We do provide support for free tier users as time permits; not limited to paying customers.
regards, rob0
Re: Client certificates revoked after upgrade to 2.11
Posted: Wed Aug 24, 2022 12:38 pm
by chort1
Hi rob0
The choice of mariadb was a request from operations, who were more comfortable maintaining and troubleshooting this than sqlite.
We upgraded via yum from 2.10.3_c47a813c-1 to 2.11.0_794ab41d-1 on RHEL8.6
After some further troubleshooting it looks like all profiles over a certain limit (41 being the highest) were removed. The renumbering was likely a consquence of users requesting new profiles and being assigned the newly freed-up serial numbers
In other words, it looks like the database was somehow reverted to an earlier stage. Could be related to the older sqlite-files still being present in /usr/local/etc/db ...?
Will consider a support ticket. We have a subscription, by the way.
Re: Client certificates revoked after upgrade to 2.11
Posted: Wed Aug 24, 2022 12:57 pm
by openvpn_inc
Hi,
If as.conf points to mysql, the sqlite3 databases are not consulted.
As for "maintaining" the sqlite3 databases, Access Server does that without human help. Best to think of sqlite3 as a file type, rather than as a RDBMS. And troubleshooting with a command line tool, sqlite3(1), can be done just as with mariadb/mysql(1). (Granted, there are a few quirks of mysql's SQL implementation, which don't apply with sqlite3, but for the most part it is very similar, and querying the databases can't break anything.)
2.10 to 2.11 would use the same database schema. So I am puzzled here. At what point was it migrated to mysql?
If you do open a ticket let me know. I'm not in Support, but I will look in when I can.
regards, rob0
Re: Client certificates revoked after upgrade to 2.11
Posted: Wed Aug 24, 2022 1:11 pm
by chort1
I see your point about sqlite, but when you run a cluster, mysql is a requirement, so I imagined it would still be a supported configuration for a standalone...?
I see that in /usr/local/openvpn_as/etc/db/backup there are automatically created folders whose dates coincide with patch times, including one for yesterday. When I check the certs.db files in them, they seem unchanged from one folder to the next.
The migration to sql was done in April, and the certs.db-files contain only entries dated back then. 6 entries, as opposed to 41 that were left after upgrading the previous server.
If I make a ticket, I'll let you know. I think we'll see how patching the remaining server turns out tonight
Re: Client certificates revoked after upgrade to 2.11
Posted: Thu Aug 25, 2022 6:35 am
by chort1
Hi again, the problem has been resolved, and it was NOT an issue with the upgrade itself
At some point we moved /var/lib/mysql to a new partition and forgot to remove the old fstab entry, so we were left with two entries for the same path. After a reboot, the path was mounted on the old volume, containing SQL data from before the move.
Thank you for all replies, and sorry for wasting your time
Re: Client certificates revoked after upgrade to 2.11
Posted: Thu Aug 25, 2022 1:27 pm
by openvpn_inc
Hi chort1,
So my initial guess about it being a database backend issue turned out to be true, nice. Glad you figured it out.
Thanks for reporting back.
Have a nice day,
Johan
Re: Client certificates revoked after upgrade to 2.11
Posted: Sat Apr 13, 2024 7:09 am
by n_malavde@yahoo.com
Hi Everyone,
Below steps resolved my issue.
Open VPN Access Server Version: 2.8.5, MFA Enabled, User accounts synched with Security Group on the Server via LDAP.
Open VPN Connect Client Version: 3.4.4, TAP-Windows: 9.24.2 (Download location:
https://build.openvpn.net/downloads/releases/)
Resolution:
Open VPN Admin Console:
1. Delete the user: User Management >> User Permissions.
2. Revoke Certificate: User Management >> Revoke Certificates
Windows Active Directory Server:
1. Remove the user from the security group >> Apply >> OK,
2. Add the user in the same security group >> Apply >> OK.
Client or Workstation:
1. Delete any previous profiles for the same connection beforehand.
2. Log on to the user console via a web browser to generate the MFA using the authenticator app. Download 'Yourself (user-locked profile)' , client.ovpn file and import the same file in the Open VPN Connect client.
3. Connect the VPN using the Open VPN Connect Client with username, password, and the MFA code.
That is all folks.
Thanking you and warm regards.
Ninad.