Over the past 20 years, a laudable standardization has taken hold in the cybersecurity sector. The US-American non-profit organization MITRE is one of the leader in this field. This is supported by various departments of the US government and has been able to establish standards such as CVE (clear identification of vulnerabilities), CWE (vulnerability classes) and CPE (normalized identification of products). ATT&CK is a rather young standardization, which documents tactics and techniques of attackers. This article critically examines the approach of the standard.

We use ATT&CK in the context of our penetration tests, in order to give our customers and coworkers a means into the hand, in order to be able to talk as possible without misunderstandings with one another about attack methods. We describe these with the corresponding unique identifiers and graphically depict the attack path.

Since December 2020, the vulnerability database VulDB has also made mapping with ATT&CK possible. We have accompanied its introduction. Our many years of experience with various standards, their approaches, structures and interoperabilities allowed us to identify potential weaknesses, deficiencies and problems in ATT&CK relatively quickly.

We have tried to implement a pattern-based parsing to identify the corresponding ATT&CK techniques based on existing descriptions of vulnerabilities. For example, if the expression bruteforce occurs in a text, then T1110 (Brute Force) is assigned. We have also implemented a mapping from CWE to ATT&CK techniques. CWE-307 (Improper Restriction of Excessive Authentication Attempts) then also becomes T1110.

No real Hierarchy

The basic problem of ATT&CK is that hierarchical structures are missing or inconsistent. The techniques cannot be assigned exclusively to individual tactics. Techniques can often be used by multiple tactics and across multiple phases of an attack.

The identifiers of both tactics and techniques are also not traceable. They lack linearity, grouping or hierarchy. The tactic Reconnaissance is usually approached at the beginning of an attack and only after that the Initial Access follows. If you look at the identifiers, they are called TA0043 and TA0001. They are (by chance) not descending order and also not neighboring numbers or neighboring ranges. TA0001 and TA0002 or TA1000 and TA2000 would have been much more appropriate.

This makes it extremely difficult to understand and work with these entries. Identifiers have to be constantly searched for and compared with each other. CWE has this problem much less, at least in the lower ranges. For example, CWE-119 is used for memory violations, CWE-120 for classic buffer overflow issues, CWE-121 for stack-based and CWE-122 for heap-based buffer overflow vulnerabilities. Only in higher regions do inconsistent numerical ranges become apparent, for example CWE-415 for double-free and CWE-416 for use-after-free.

Inconsistent Distribution of Techniques

T1555 deals with Credentials from Password Stores. The description is easy to understand:

Adversaries may search for common password storage locations to obtain user credentials. Passwords are stored in several places on a system, depending on the operating system or application holding the credentials.

However, the sub-techniques are then limited to Keychain (T1555.001), Securityd (T1555.002) and Credentials from Web Browsers (T1555.003).

This does not refer to /etc/passwd or /etc/shadow as it is the case with Unix-based systems. Only with T1003.008 the corresponding sub-technology is introduced. A real difference in terms of the concept and the shown decoupling cannot be understood.

A similar problem arises if you want to assign Cross Site Scripting as a vulnerability. On one hand, T1059.007 is a good choice because it deals with the execution of Javascript. On the other hand, it is always a drive-by attack, which is described in T1189 and also gives the example of such XSS.

Ambiguous Definitions

The task of standardizing is sometimes to create an uniform and comprehensible nomenclature. Unluckily, T1498, which deals with Network Denial of Service, behaves not in this respect. In the description this class is specified as follows:

Network DoS can be performed by exhausting the network bandwidth services rely on. (…) A Network DoS will occur when the bandwidth capacity of the network connection to a system is exhausted due to the volume of malicious traffic directed at the resource or the network connections and network devices the resource relies on.

This specifically focuses on bandwidth overload, either due to a high number of accesses or a large amount of data. This makes it difficult to assign network-based attacks that take advantage of peculiarities in network protocols. These include IP fragmentation (e.g. Ping of Death) and spoofed unreachable errors with ICMP Type 3, Code 1-3.

Overlaps and Duplicates

In a specific case, a duplicate must be mentioned. T1537 describes Transfer Data to Cloud Account with the following words:

Adversaries may exfiltrate data by transferring the data, including backups of cloud environments, to another cloud account they control on the same service to avoid typical file transfers/downloads and network-based exfiltration detection.

Similarly, T1567.002 with the title Exfiltration Over Web Service: Exfiltration to Cloud Storage states:

Adversaries may exfiltrate data to a cloud storage service rather than over their primary command and control channel. Cloud storage services allow for the storage, edit, and retrieval of data from a remote cloud storage server over the Internet.

Only on closer inspection does the context reveal that in T1537 the compromised source must already be a cloud system. Accordingly, AWS CloudTrail logs, Azure activity logs, Stackdriver logs are recorded as Data Sources. However, T1567.002 are exfiltrations via web services, which mainly affect end user platforms such as Linux, Windows, and macOS.

Specific but Incomplete Sub-Techniques

T1021 deals with Remote Services, which can be used to implement remote access. Thus then also as sub-techniques RDP (T1021.001), SSH (T1021.004) and VNC (T1021.005) is available.

However, this list is far from being complete. Traditionally, the unencrypted Telnet or even rlogin is missing. The same problem can be found elsewhere, for example at T1213 (e.g. wikis or ticketing systems are missing).

Inconsistent Sub-Techniques

Previously, the example of T1021 illustrated that lists of sub-techniques can be incomplete.

In addition, the list is also inconsistent in this specific case. RDP, SSH and VNC are indeed classical mechanisms for remote maintenance. SMB/Windows Admin Shares (T1021.002) and Distributed Component Object Model (T1021.003) work differently, however. SMB is much more similar to NFS and DCOM is closer to RPC.

Free and commercial solutions such as TeamViewer and Chrome Remote Desktop should be mentioned here instead, since they are more suitable in terms of concept.

This can also be observed at T1499, which documents the so-called Endpoint Denial of Service. In the sub-categories mainly flooding techniques are mentioned (T1499.001 against operating system, T1499.002 against services, T1499.003 against applications). Only T1499.004 also introduces another DoS technique, which is consolidated however then in contrast to the before mentioned ones regarding operating system and applications.

Too narrowly defined Sub-Techniques

In T1037.001 describes scripts, which are executed during boot or login. Sub-technique T1037.001 deals with Windows and T1037.002 with Mac. In this case we will take a closer look at the Windows aspect. In the description you may find:

Adversaries may use Windows logon scripts automatically executed at logon initialization to establish persistence. Windows allows logon scripts to be run whenever a specific user or group of users log into a system. This is done via adding a path to a script to the HKCU\Environment\UserInitMprLogonScript Registry key.

This definition does not go far enough. In the history of MS-DOS and Windows, there are about a dozen ways how components can be started automatically at boot time or after a login. In MS-DOS these were typically autoexec.bat and config.sys. And since Windows NT, additional registry keys could be made available for automatic execution.

T1037.001 thus shows that it is too limited and completely ignores historical aspects. It may be debated whether techniques on operating systems from more than 20 years ago are still of high importance. But experience shows that it is often highly critical components that are operated with such archaic software.

Sub-Techniques too short

T1052 discusses the exfiltration of data through a physical medium. In the description different media are listed in a non-conclusive list:

Such media could be an external hard drive, USB drive, cellular phone, MP3 player, or other removable storage and processing device.

However, for this technology with T1052.001 only a specific sub-technology exists, which is limited to USB as media. The previously mentioned media are not available as dedicated sub-technologies.

A similar problem can be found with T1222, which deals with the rights of files and directories. As sub-techniques the two operating system families Windows (T1222.001) and Linux/Mac (T1222.002) are listed. Alternative operating systems are not mentioned. In this specific case it is generally questionable whether this should be differentiated after operating system instead of after file system or application types (e.g. Web server, FTP server, file server).

Verified Incidents only

In April 2019 we submitted a proposal for the NAC Bypass technique in the tactic Defense Evasion. The technique refers to the lecture of Alva Lease Skip Duckwall IV at Defcon 19, and refers to the tool NACKered as well as our improvement nac_bypass.

One of the criteria for the inclusion of new techniques is the existence of data from verified incidents. NAC bypass is mostly used by internal attackers and in the context of penetration tests or red team assessments, about which there are rarely public reports. The technique was not included in the framework with the remark that there are no verified incidents yet.

As a result, the framework does not compile all possible techniques, but rather summarizes them in a retrospective review. This reactive behavior prevents interested readers from perceiving potential and foreseeable threats as such. Only when a first incident is revealed will information about the techniques used be documented.

Conclusion

ATT&CK is an interesting and also important standard. It helps to catalog different tactics and techniques. However, in many points this has not always been successful.

The described shortcomings make it very difficult to understand and handle the standard. Techniques and sub-techniques are sometimes difficult to find, cannot be understood easily, are incomplete or wrong.

The maturity of ATT&CK is not comparable to that of other MITRE projects, such as CWE or CPE. There is an urgent need for improvement in order to develop a solid and sustainable construct that can have a positive impact on the cybersecurity industry.

We have communicated the points summarized in this paper to the project team and hope that they will be taken into account in future decisions and developments.

About the Authors

Marc Ruef has been working in information security since the late 1990s. He is well-known for his many publications and books. The last one called The Art of Penetration Testing is discussing security testing in detail. He is a lecturer at several faculties, like ETH, HWZ, HSLU and IKF. (ORCID 0000-0002-1328-6357)

Michael Schneider has been in IT since 2000. Since 2010 he is focused on information security. He is an expert at penetration testing, hardening and the detection of vulnerabilities in operating systems. He is well-known for a variety of tools written in PowerShell to find, exploit, and mitigate weaknesses. (ORCID 0000-0003-0772-9761)

Links

• https://attack.mitre.org
• https://attack.mitre.org/resources/changelog.html
• https://attack.mitre.org/tactics/TA0001
• https://attack.mitre.org/tactics/TA0043
• https://attack.mitre.org/techniques/T1003/008/
• https://attack.mitre.org/techniques/T1021/
• https://attack.mitre.org/techniques/T1021/002/
• https://attack.mitre.org/techniques/T1021/003/
• https://attack.mitre.org/techniques/T1021/004/
• https://attack.mitre.org/techniques/T1021/005/
• https://attack.mitre.org/techniques/T1037/
• https://attack.mitre.org/techniques/T1037/001/
• https://attack.mitre.org/techniques/T1037/002/
• https://attack.mitre.org/techniques/T1052/
• https://attack.mitre.org/techniques/T1052/001/
• https://attack.mitre.org/techniques/T1059/007/
• https://attack.mitre.org/techniques/T1110/
• https://attack.mitre.org/techniques/T1189/
• https://attack.mitre.org/techniques/T1213/
• https://attack.mitre.org/techniques/T1222/
• https://attack.mitre.org/techniques/T1222/001/
• https://attack.mitre.org/techniques/T1222/002/
• https://attack.mitre.org/techniques/T1498/
• https://attack.mitre.org/techniques/T1499/
• https://attack.mitre.org/techniques/T1499/001/
• https://attack.mitre.org/techniques/T1499/002/
• https://attack.mitre.org/techniques/T1499/003/
• https://attack.mitre.org/techniques/T1499/004/
• https://attack.mitre.org/techniques/T1537/
• https://attack.mitre.org/techniques/T1555/
• https://attack.mitre.org/techniques/T1555/001/
• https://attack.mitre.org/techniques/T1555/002/
• https://attack.mitre.org/techniques/T1555/003/
• https://attack.mitre.org/techniques/T1567/002/
• https://cpe.mitre.org/
• https://cve.mitre.org/
• https://cwe.mitre.org/
• https://cwe.mitre.org/data/definitions/119.html
• https://cwe.mitre.org/data/definitions/120.html
• https://cwe.mitre.org/data/definitions/121.html
• https://cwe.mitre.org/data/definitions/122.html
• https://cwe.mitre.org/data/definitions/307.html
• https://cwe.mitre.org/data/definitions/415.html
• https://cwe.mitre.org/data/definitions/416.html
• https://github.com/p292/NACKered
• https://github.com/scipag/nac_bypass
• https://insecure.org/sploits/ping-o-death.html
• https://remotedesktop.google.com/
• https://tools.ietf.org/html/rfc792
• https://vuldb.com
• https://www.defcon.org/images/defcon-19/dc-19-presentations/Duckwall/DEFCON-19-Duckwall-Bridge-Too-Far.pdf
• https://www.mitre.org
• https://www.teamviewer.com/