I want a "Red Teaming"
Michael Schneider
This is how Symmetric Encryption really works
It would appear that people are all too quick to place their trust in information provided by third parties, which goes hand-in-hand with a certain amount of risk. They naturally jump at the chance to choose the simplest solution and are building up more trust than knowledge on encryption as a subject.
Particularly when new claims are made regarding the strength of encryption solutions, there may be unfounded pressure to migrate to what is currently the strongest encryption technology, even though this might not be a vital requirement at the time in question. Having said that, there’s nothing wrong with heeding third-party information and opting for the strongest encryption solution. It’s decidedly better than simply sweeping the issue of encryption under the rug. However, you do need to be familiar with some encryption concepts so you can base your understanding of the issue’s constantly changing situation on slightly more conscious choices as opposed to trust.
If you have specific questions about encryption algorithms’ or implementation solutions’ cryptographic strengths and weaknesses in the likes of environments with limited resources, you should consult experienced cryptologists beforehand. And another golden rule before we get into the nitty-gritty: you should only seek to implement your own cryptographic methods for practice purposes. Cryptographic methods in productive systems should only be implemented by experienced and recognized cryptologists.
This article will use some current examples to describe various terms and concepts of symmetric encryption. While the concepts mentioned aren’t to be construed as recommendations, they should effectively introduce a few terms, including the likes of AE, MAC and mode of operation.
The mathematics behind cryptographic methods and how they are effectively implemented are only touched on slightly or left out entirely. Some of the concepts presented here are greatly simplified. The illustrations used should help readers to understand the concepts better; a key for the symbols used in the illustrations can be found at the end of the article.
In principle, the modern symmetric encryption methods are technical further developments of classic mechanisms. The Caesar cipher (which dates back to approximately 50 years B.C.) is a simple substitution method and a solution that is often cited as one of the classic methods.
Scholars suggest that stone tablets from Mesopotamia recording a craftsman’s formula for pottery glaze, dating back to 2000 years B.C. and written in cuneiform script, are the first signs of encryption being used to protect information. It is believed that the tablets were encrypted to protect the potter’s formula from being stolen for commercial reasons.
The symmetry is derived from the fact that the same secret key K
is used for encryption and decryption, as shown in the figure below. (A key of the symbols used can be found attached to this article.)
At present, symmetric methods are still the first choice for encrypting larger volumes of data, due to the simple fact that they usually deliver better results. Generally speaking, a distinction is also made between hardware-based and software-based encryption, but in most cases the differences in performance become progressively smaller.
However, since the same secret key is used for encryption and decryption, a pure symmetric method is unsuitable for encrypted communication. After all, how should the sender disclose the secret key to the recipient in the first place (key distribution problem)? While messengers and sealed letters could still be used in the past, this solution didn’t scale up well.
And that’s why asymmetric methods (typically used to exchange the secret key for symmetric methods, resulting in a hybrid method) were introduced.
Up to a defined block size, the plaintext block P
is encrypted with the key K
and block cipher BC
to produce ciphertext C
, as shown in the figure below. In the case of the AES, the block size is 128 bits, for example.
Blockwise processing requires another component to be functional; the additional high-level issues with this are:
The answer to the above questions is what is known as the “mode of operation” for block ciphers. This is why the claim that something is encrypted with AES is usually incomplete – the mode of operation should be specified too. Why? Because the mode of operation isn’t just a function for processing any size of plaintext, it incorporates security features too.
The phrase Everybody knows ECB mode is bad because we can see the penguin is an impressive example of security features that should be “added” “outside” of the actual encryption algorithm.
The ECB mode of operation, which is the simplest of the modes of operation, ensures that any length of plaintext can be encrypted with a block cipher. The plaintext is formatted to the required block size and each block is encrypted one after the other using a BC algorithm, as shown above.
The figure below illustrates the penguin problem, recreated with the scip logo: The scip logo is encrypted with the 128-bit AES-ECB method; AES-CBC is also shown by way of comparison.
The conclusion that can be drawn from the penguin problem is that identical plaintext blocks produce identical ciphertext blocks. So, if P1
and P3
are identical, then C1
and C3
are identical too. Certain information can thus be extracted from the ciphertext. A degree of randomness is needed to overcome this issue.
The encryption algorithm makes transformation secure, but only exactly for a 128-bit block in the case of AES with a 128-bit block size and without a mode of operation.
On the one hand, the basic modes are intended to ensure that any length of plaintext can be encrypted and, to put it in simple terms, that no patterns can be identified in the ciphertext. Even if the same plaintext is encrypted several times over with the same key, a “different” ciphertext should “be created” each time.
Randomization methods are used to prevent patterns from emerging, making an encryption algorithm like the AES cryptographically stronger when used in combination with a mode of operation (except ECB). As shown in the example above, ECB (the electronic codebook) should no longer be used.
CBC (cipher block chaining mode), shown in the figure below, works in a similar way to ECB, but chains each block to the previous one. For the first block, what is known as a “randomized initialization vector” (IV) is used to add a degree of randomization. This ensures that identical plaintext blocks are not encrypted into identical ciphertext blocks. Even if the encryption is invoked twice for the same plaintext and key, the ciphertext will be different each time due to the randomized IV and chaining.
Another method known as “padding” is necessary for the CBC mode. If the plaintext is not a multiple of the block size, e.g. if the plaintext is 132 bits long, a first 128-bit block can be generated, but a 128-bit block is also required for the remaining 4 bits of plaintext, so 124 bits must be added to the 4 bits to produce 128 bits. One effect of padding is that the ciphertext is always slightly longer than the plaintext.
Another method used to create the required block size is CTS. While it may well be slightly more complicated, it makes sure that ciphertext and plaintext are the same length. In simple terms, this is done by stealing the number of bits needed for padding purposes from the penultimate cipher block and filling up the last plaintext block before it is encrypted with the amount of bits needed.
CTR (counter mode), shown the figure below, practically turns a block cipher into a stream cipher. A counter (each block of a message must not have the same counter value) and nonce (number used only once; doesn’t have to be random, but unique by key K/Message) as IV are encrypted by block ciphers to form a kind of key stream (KS). The key stream is then encrypted one bit at a time via XOR, with plaintext turning into ciphertext (more on this under “Stream ciphers”). This eliminates the need for padding methods for block ciphers with CTS mode.
GCM (Galois/counter mode) is essentially based on CTR, but integrates the message authentication code (MAC) function. This is to ensure that, during decryption, it can be checked whether the data has been deliberately or unintentionally modified while encrypted (data authenticity and integrity), resulting in authenticated encryption (AE).
The authenticity of unencrypted data, such as header data (associated data), is also authenticated with the GCM’s MAC function, resulting in authenticated encryption with associated data (AEAD).
The Wegman-Carter MAC method, which is based on the hashing method GHASH, is used to verify whether data has changed. After encryption (encrypt-then-authenticate), the MAC method generates an authentication tag, which can also be calculated during decryption. If they match, it can be assumed that the data has not been changed. The GCM can also be used to authenticate unencrypted data; this is known as the GMAC (Galois message authentication code).
As its name implies, CCM (counter mode with CBC-MAC) is based on the CTR mode and uses CBC as the MAC method. Unlike GCM, authentication is performed first, followed by encryption (authenticate-then-encrypt).
XTS (XEX-based tweaked codebook mode with ciphertext stealing) is a mode designed to encrypt data on a disk. Disk encryption increases the demands placed on a secure encryption concept. Subdivision into sectors on the disks in particular has a major influence here, since each sector is to be encrypted and decrypted securely, individually and at a high performance level. The term “tweaked” or “tweakable encryption” (TE) comes into play here.
TE is used in disk encryption to include the sector in the encryption process. This enables encryption of an entire disk with the same key, but a sector-specific value is used as a kind of additional key (the tweak). Stream ciphers (as well as block ciphers with counter mode) are unsuitable for disk encryption for specific reasons.
The basic mode for XTS is XEX (XOR-encrypt-XOR), which uses three keys for encryption, so one of these keys can be the tweak value. In an XEX implementation without a tweak, this is achieved with key whitening, or the keys K1
and K2
are derived from the key K
.
As its name implies, the stream cipher encrypts a stream of data, i.e. plaintext one bit at a time. This is done by encrypting each bit from the plaintext P
with one bit from what is known as the “key stream” KS
to create the ciphertext C
. The key stream is generated from a secret key K
and a nonce N
using stream ciphers SC
; see the figure below. To be specific, a pseudo-randomized stream of bits is generated for the key and the nonce using a stream cipher algorithm.
Historically, stream ciphers were considered to deliver better results in hardware-based encryption, because they used fewer resources (number of logic gates). Today, the comparison with block ciphers has put this into perspective to a certain extent, since nowadays block ciphers also have lean hardware implementations and, not least, because the available resources in the hardware chips are constantly increasing.
Stream ciphers should not use the same KS
twice (or the same nonce with the same key). What’s more, the KS
itself should be prevented from repeating after a certain period.
The length of the KS
(a type of KS
block size) is therefore fixed, e.g. ChaCha20; this is 512 bit. So, if a plaintext larger than 512 bit is to be encrypted, it must be possible to invoke a new instance of the encryption algorithm. A new invocation is to be randomized, since the same key is used for it.
Stream ciphers can be roughly divided into two types. A stateful stream cipher (e.g. RC4) generates a secret internal state using a key, a nonce and an algorithm. The state is used as the starting point for creating the KS
. An update function is used to change the state over time so as to pseudo-randomize KS
generation.
A counter-based stream cipher (e.g. Slasa20/ChaCha) rectifies randomization in a relatively simple way: with a counter for each KS period, similar to the counter mode for block ciphers.
What about AE in stream ciphers? Stream ciphers that are recommended at present do not come with an integrated AE function, as is the case with the block cipher modes of operation. However, a general MAC method, such as ChaCha20, Poly1305 (encrypt-then-MAC) can naturally be implemented too.
The following summary can be provided to give a basic understanding of symmetric encryption. The symmetric block cipher algorithms consist of an encryption algorithm and a secret key, which can be used to securely transform plaintext with a fixed block size into a ciphertext of the same block size. By using a mode of operation, any size of plaintext can be encrypted by invoking the algorithm several times. To ensure that no patterns are created if the algorithm is invoked repeatedly, the mode of operation should create a degree of randomness (randomized IV, nonce IV, chaining, etc.). Furthermore, modes of operation can include MAC methods (AE, AEAD) to verify data integrity and thus incorporate additional security features.
Symmetric stream ciphers consist of an encryption algorithm, a nonce IV and a secret key, which is used to generate a key stream. The key stream is linked to form a ciphertext using XOR and plaintext. The key stream can only be a certain length. This is why the process is invoked several times, even for stream ciphers, to encrypt plaintext longer than a key stream. The mode of operation necessary for this already forms part of the stream cipher. While currently recommended stream ciphers (BSI, ECRYPT) do not offer integrated AE, stream ciphers can be combined with general MAC methods.
If you’d like to go one step further, Serious Cryptography by Jean-Philippe Aumasson (ISBN 978-1-59327-826-7) is well worth a read.
Symbol | Description |
---|---|
K | Secret key |
K1–K2 | Derived from K; key parts |
P | Plaintext |
P1–P3 | Plaintext blocks |
C | Ciphertext |
C1–C3 | Ciphertext blocks |
BC | Block cipher |
SC | Stream cipher |
IV | Initialization vector |
N | Nonce; number only used once |
Ctr | Counter |
T | Tag; authentication tag |
KS | Key stream |
⊕ | XOR gate |
Our experts will get in contact with you!
Michael Schneider
Marisa Tschopp
Michèle Trebo
Andrea Covello
Our experts will get in contact with you!