DES-X Encryption slow / Help needed

[eychei]

Hi everyone,

I did write a small vb.net program to encrypt and decrypt DESX.
Its actually nearly the same as DES but with an additional input and output XOR with two extra keys.

wo8Bl.png

So when I decrypt files it easy really fast taking around 10sec for 100mb.
This is because I can decrypt the complete file first without needing to go from one block to the next.

But when I encrypt files I need to chain 8 byte sized chunks in serial.


My vb.net code looks like this:

 'DES-X Start
            Dim xor_in_array As Byte() = XORarrayValues(input_whitener_array, ByteToEncrypt)
            Dim ciphertext As Byte() = New Byte(ByteToEncrypt.Length - 1) {}
            Dim ms As New MemoryStream
            Dim cs As New CryptoStream(ms, des.CreateEncryptor(), CryptoStreamMode.Write)
            Dim cut_bytes() As Byte = New Byte(8 - 1) {}
            For i = 0 To (xor_in_array.Length / 8) - 1

                Array.Copy(xor_in_array, CInt(i) * 8, cut_bytes, 0, 8)
                If i = 0 Then
                    Dim xor_iv = XORarrayValues(cut_bytes, Var.iv)
                    ms.Position = 0
                    cs.Write(xor_iv, 0, xor_iv.Length)
                    Dim des_xor_iv = ms.ToArray()
                    Dim xor_out = XORarrayValues(Var.output_whitener, des_xor_iv)
                    Array.Copy(xor_out, 0, ciphertext, CInt(i) * 8, 8)

                Else
                    Dim ciphertext_used() As Byte = New Byte(8 - 1) {}
                    Array.Copy(ciphertext, CInt(i - 1) * 8, ciphertext_used, 0, 8)
                    Dim xor_ciphertext_used = XORarrayValues(cut_bytes, ciphertext_used)
                    ms.Position = 0
                    cs.Write(xor_ciphertext_used, 0, xor_ciphertext_used.Length)
                    Dim des_xor_out = ms.ToArray()
                    Dim xor_out = XORarrayValues(Var.output_whitener, des_xor_out)
                    For j = 0 To xor_out.Length - 1
                        ciphertext(i * 8 + j) = xor_out(j)

                    Next
                    'Array.Copy(xor_out, 0, ciphertext, CInt(i) * 8, 8)
                    'ciphertext = ciphertext.Concat(xor_out).ToArray()



                End If

So I tried different approaches in appending the ciphertext at the end to a bte array so i can write it to file afterwards.
It does not change in speed if I use array.copy / concat or for loop copy.
I also tried with creating 4-8 parallel threads. While this reduces the time needed for encryption, it is still slow for very big files.


How can I increase speed for the encryption algorithm? Hope someone has a solution.

Thx.

-e

[chants]

XORarrayValues being immutable is certainly hurting perf. You should consider making it mutable where it stores the result in one of the inouy buffers or reuse an output buffer.

Memory allocation is expensive and you overdo it. Also the ToArray calls should be avoided if there is any other way to reference the bytes needed as a view or what have you.

[Gregory Morse]

Originally Posted by eychei

Hi everyone,

I did write a small vb.net program to encrypt and decrypt DESX.
Its actually nearly the same as DES but with an additional input and output XOR with two extra keys.

Attachment 10479

So when I decrypt files it easy really fast taking around 10sec for 100mb.
This is because I can decrypt the complete file first without needing to go from one block to the next.

But when I encrypt files I need to chain 8 byte sized chunks in serial.



So I tried different approaches in appending the ciphertext at the end to a bte array so i can write it to file afterwards.
It does not change in speed if I use array.copy / concat or for loop copy.
I also tried with creating 4-8 parallel threads. While this reduces the time needed for encryption, it is still slow for very big files.


How can I increase speed for the encryption algorithm? Hope someone has a solution.

Thx.

-e

You can write native code instead of the .Net code. Will be much faster. At least, write the time consuming parts of the code in native. You can use the pre-written libraries available on internet. No need to create your own. Much faster.

[chants]

Originally Posted by Gregory Morse

You can write native code instead of the .Net code. Will be much faster. At least, write the time consuming parts of the code in native. You can use the pre-written libraries available on internet. No need to create your own. Much faster.

In the future if you are going to troll with annoying advice, consider not posting. He clearly stated that decryption functioned in a reasonable time. To throw out generic canned nonsense advice like using native code was clearly not what was asked. Although you've been doing this for a while now so it's not surprising.

[dyers eve]

Originally Posted by chants

In the future if you are going to troll with annoying advice, consider not posting. He clearly stated that decryption functioned in a reasonable time. To throw out generic canned nonsense advice like using native code was clearly not what was asked. Although you've been doing this for a while now so it's not surprising.

You are not the only one who knows everything, @chants.
Whether they are right or wrong, is not for you to decide.
While you may have a better answer, if someone else also wants to post their opinions, let them do so!

@eychie It is better to avoid managed frameworks like .Net when you want to optimise for performance. It is also preferable to code the most compute-intensive parts of the encryption in C++ or better yet, in a memory-safe language like Rust.
Is there a good reason why you chose VB.Net instead of C?
I cannot comment further since I cannot access your attachment due to my low rank. I can help with the code if you provide more information, if you need.

[chants]

The picture is just the generic CBC encryption mode. So I'm unclear how you can parallelize it as CBC has dependencies requiring it processed in a line order. But if you are happy with the performance in one direction, then VB.NET can deliver adequate enough performance in the opposite direction if you performance profile and refactoring to make good reuse of memory and avoid unnecessary copies.

Don't worry about the guy with ~15 accounts telling you to abandon ship. Though such advice is fine in general, most of us can read and developed abstract thinking during childhood so we are capable of inferring contextual clues. Unlike this smooth brained individual, yielding too many accounts.

[chants]

By the way, if you want to parallelize this, you should consider a different cipher mode of operation, preferably Galois Counter Mode (GCM) as the ECB mode has various attack vectors and security risks, which is probably why you are using CBC, but it doesn't scale well for very large data sizes.

[eychei]

Thanks chants.

Originally Posted by chants

By the way, if you want to parallelize this, you should consider a different cipher mode of operation, preferably Galois Counter Mode (GCM) as the ECB mode has various attack vectors and security risks, which is probably why you are using CBC, but it doesn't scale well for very large data sizes.

I am using CBC because the encryption scheme is not done by me it is the software I am trying to recreate.

[eychei]

I am using vb.net because I do not know much C.
I also wrote the program first in python but it is also very slow.
Would this be much faster in C?

So to sum it up, there is not much to optimize in the vb.net code speed vise?

How do I make the array mutable?

I will also try to remove the ToArray() copies. Maybe that helps in speed.


-e

[chants]

.NET JIT is quite good, doing things unsafe is closer to C but there will only be some relatively small increase if you make the VB.NET code highly efficient. XORarrayValues should write directly into one if the two input arguments (and ^= mutable variant used, not x=x^y) or pass a third output buffer and reuse it. Also consider running the VS profiler to see line by line where most time is spent. If you still aren't happy after these basic steps, you can consider whether to spend the time rewriting in C.

Are you sure this isn't just CBC DES and directly decryptable extremely fast with Windows builtin libraries via .NETs System.Security.Cryptography? Ah nevermind, seems there is no way to inject the XORs between blocks. ECB mode and doing manually seems fine.