Page 1 of 1

Custom algorithm as cipher option

Posted: Thu Jun 18, 2020 2:38 pm
by rafael-at
Suppose i want to work with a different algorithm in the symmetrical communication between client and server in a openvpn tunnel. The page about changing encryption cipher in access server gives a list of allowed ciphers, but, in the case that i want to use a non-standard custom made algorithm to encrypt and decrypt messages sent through the VPN tunnel, what would be the steps to be done to make such algorithm available on the openvpn?

I initially thought about 2 options:
- Create an application to simulate a cryptographic token and insert my custom made algorithm in the PKCS11 functions, managing also the certificate and key stored on the server/client and use the generated ".so" as a pkcs11 token to feed the server and client configuration files.
- Modify openSSL library to integrate my custom cipher, such as described here, so my algorithm shows as an option for --cipher in server and client configuration files.

I am aware of the benefits of using an algorithm such as AES to manage the encryption, but would any of the 2 options above work to include a custom algorithm?

Thanks in advance.

Re: Custom algorithm as cipher option

Posted: Thu Jun 18, 2020 5:20 pm
by TinCanTech
All encryption in OpenVPN is provided by the SSL library.

One of:
  • OpenSSL
  • PolarSSL
  • LibreSSL
  • WolfSSL
So if you want to write your own cipher then that is where you start. Your second option above..

And [ oconf ] is a BBCode for OpenVPN configuration files which removes private user keys.

It does not work in a subject field ... :roll:

Looking forward to hearing more about your new cipher ..

Re: Custom algorithm as cipher option

Posted: Fri Jun 19, 2020 12:21 pm
by rafael-at
Thank you for the reply, sorry i got a little confused by the forum rules.

Re: Custom algorithm as cipher option

Posted: Tue Mar 09, 2021 6:09 am
by techsmith
i have configured my custom cipher in openssl, according to its architecture.
Custom Cipher is being used at secure connection between virtual client and server while establishing secure connection between them.( verified by wireshark).
But my custom cipher is not supported by openvpn.
how can i enlist my custom cipher from openssl into openvpn?