Custom algorithm as cipher option

How to customize and extend your OpenVPN installation.

Moderators: TinCanTech, TinCanTech, TinCanTech, TinCanTech, TinCanTech, TinCanTech

Post Reply
rafael-at
OpenVpn Newbie
Posts: 2
Joined: Thu Jun 18, 2020 2:24 pm

Custom algorithm as cipher option

Post by rafael-at » Thu Jun 18, 2020 2:38 pm

Suppose i want to work with a different algorithm in the symmetrical communication between client and server in a openvpn tunnel. The page about changing encryption cipher in access server gives a list of allowed ciphers, but, in the case that i want to use a non-standard custom made algorithm to encrypt and decrypt messages sent through the VPN tunnel, what would be the steps to be done to make such algorithm available on the openvpn?

I initially thought about 2 options:
- Create an application to simulate a cryptographic token and insert my custom made algorithm in the PKCS11 functions, managing also the certificate and key stored on the server/client and use the generated ".so" as a pkcs11 token to feed the server and client configuration files.
- Modify openSSL library to integrate my custom cipher, such as described here, so my algorithm shows as an option for --cipher in server and client configuration files.

I am aware of the benefits of using an algorithm such as AES to manage the encryption, but would any of the 2 options above work to include a custom algorithm?

Thanks in advance.
Last edited by Pippin on Thu Jun 18, 2020 2:52 pm, edited 1 time in total.
Reason: Fix topic title

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: Custom algorithm as cipher option

Post by TinCanTech » Thu Jun 18, 2020 5:20 pm

All encryption in OpenVPN is provided by the SSL library.

One of:
  • OpenSSL
  • PolarSSL
  • LibreSSL
  • WolfSSL
So if you want to write your own cipher then that is where you start. Your second option above..

And [ oconf ] is a BBCode for OpenVPN configuration files which removes private user keys.

It does not work in a subject field ... :roll:

Looking forward to hearing more about your new cipher ..

rafael-at
OpenVpn Newbie
Posts: 2
Joined: Thu Jun 18, 2020 2:24 pm

Re: Custom algorithm as cipher option

Post by rafael-at » Fri Jun 19, 2020 12:21 pm

Thank you for the reply, sorry i got a little confused by the forum rules.

techsmith
OpenVpn Newbie
Posts: 1
Joined: Tue Mar 09, 2021 6:07 am

Re: Custom algorithm as cipher option

Post by techsmith » Tue Mar 09, 2021 6:09 am

i have configured my custom cipher in openssl, according to its architecture.
Custom Cipher is being used at secure connection between virtual client and server while establishing secure connection between them.( verified by wireshark).
But my custom cipher is not supported by openvpn.
how can i enlist my custom cipher from openssl into openvpn?

hastenf
OpenVpn Newbie
Posts: 1
Joined: Wed May 10, 2023 10:06 am

Re: Custom algorithm as cipher option

Post by hastenf » Wed May 10, 2023 10:13 am

I am also struggling with same issue. Did you find anyway to custom cipher to work with openvpn.? Or any way to add your cipher to know ciphers list?

Post Reply