In some environments, it is mandatory to keep track of every user activity on a specific system. Since in most cases the systems are certified, we searched for a solution running on a RHEL 6, no patched kernel, and no unsupported hack.

We evaluated some solutions, but not one is really satisfying.

Problem

The problem is pretty simple: a legitimate user logs into a system using ssh or console and starts a shell; we want to keep track of all the commands he executes on the system. It should be impossible for the unprivileged user to bypass the audit mechanism. It should be hard, or at least there must be a notice for the superuser to bypass or tamper the audit mechanism.

Solutions

Some solutions (in form of scripts, bash functions, pipes and so on) can be dducked and may function in some way, but all have some limitations; specifically, the solutions using bash functions or rewritten PROMPT may be manipulated or unset by the unprivileged user.

After some testing, we decided to take a closer look at following tools:

• auditd
• rootsh
• pam_tty_audit.so
• sudo

auditd

auditd is the userspace component to the Linux Auditing System. It’s responsible for writing audit records to the disk. Of particular interest is the option -S (track a SYSCALL), specifically the syscall EXECVE. We added two rules to auditd.conf, tagging unprivileged user and privileged user activities differently.

auditd policy:

# logging _bash_ commands
-a exit,always -F arch=b64 -F euid=0 -S execve -k rootact
-a exit,always -F arch=b32 -F euid=0 -S execve -k rootact
-a exit,always -F arch=b64 -F euid>=1000 -S execve -k useract
-a exit,always -F arch=b32 -F euid>=1000 -S execve -k useract

auditd is not really a tool to track shell activity, it is very verbose, logs a log of details and must be carefully configured since impacts hard on system performance.

The biggest issues are the command parameters logged on a separate line:

• SYSCALL line for the user parameters
• EXECVE for the command and command parameters

On the other side, we can audit user activity very well and use standard components (aureport, to search and report the audit log. As example, if a user executes a command:

[admin@mos ~]$ ls -lisa

The corresponding auditd log shows:

[root@mos pam.d]# aureport --tty -ts today
...
type=SYSCALL msg=audit(1432238948.640:38103): arch=c000003e syscall=59 success=yes exit=0 a0=1124430 a1=11289c0 a2=112de40 a3=7fffcf153270 items=2 ppid=13513 pid=16214 auid=1000 uid=1000 gid=1000 euid=1000 suid=1000 fsuid=1000 egid=1000 sgid=1000 fsgid=1000 tty=pts3 ses=19 comm="ls" exe="/bin/ls" subj=unconfined_u:unconfineer:unconfined_t:s0-s0:c0.c1023 key="useract"
type=EXECVE msg=audit(1432238948.640:38103): argc=3 a0="ls" a1="--color=auto" a2="-lisa"
...

rootsh

rootsh is a logging wrapper for shells. It starts a shell with logging of input/output. You can run rootsh as a standalone application if you only want to log your own user’s session. If you call rootsh with additional commands, these will be passed to the shell.

We set the unprivileged user shell to rootsh in /etc/passwd, so for the unprivileged user is really hard to jump out of rootsh.

The unprivileged user can start a sh or a bash, but rootsh will also log the activity, input and output. The tool logs activity very well, it is really simple to check the executed commands and the relative output, but it lacks of the basic auditing feature: the timestamp.

We can follow the user’s activity, but its limited by the shell starting time. All commands and outputs do not have the relative timestamp, so we cannot determine at which time the user has executed something.

The logging type is basically a dump of the terminal: what you see on your shell is what is written in the log.

pam_tty_audit.so

A specific components of the audit system uses the pam_tty_audit.so PAM module to enable or disable auditing of TTY input for specified users. When the user logs in, pam_tty_audit.so records the exact keystrokes the user makes into the /var/log/audit/audit.log file. The module is part of auditd framework, so make sure it is enabled before configuring pam_tty_audit.so. It can be enabled in general, for some users or can be excluded for some users, making it pretty flexible.

The module logs keystrokes, so if you press DEL or BACKSPACE, in the audit.log you will see a lot of , , etc. this is a little bit uncomfortable, but good enough to reconstruct user activity.

Since this is a auditd component, we can use the aureport with the -tty parameters to report all record logged by the module.

The logging does not always appear in real time: the strokes are buffered, if the user started a new sh, the log will be written when the user exits the sh.

[root@mos pam.d]# cat /etc/pam.d/system-auth
...
session	  required   pam_tty_audit.so enable=*
...

The resulting logfile looks like:

[root@mos pam.d]# aureport --tty -ts today
...
15. 05/25/2015 16:57:34 106664 1000 ? 51 _bash_ <ret>
16. 05/25/2015 16:57:39 106666 1000 ? 51 _bash_ "su oper",<ret>
20. 05/25/2015 16:58:17 107808 1000 ? 50 _bash_ <up>,<ret>
21. 05/25/2015 16:58:24 107811 1000 ? 51 _bash_ "ls .lisa",<backspace>,<backspace>,<backspace>,<backspace>,<backspace>,<backspace>," -lisa",<ret>,"cd",<ret>,<ret>,<ret>,"find / -name test",<ret>,"exit",<ret>
...

sudo

sudo allows a permitted user to execute a command as privileged user or another user, as specified by the security policy. sudo supports logging a command’s input and output streams. I/O logging is not on by default but can be enabled using the LOG_INPUT and LOG_OUTPUT flags.

For example, we can log all activities of admins and operators using the following configuration directives:

...
%admin ALL = (ALL) NOPASSWD: LOG_INPUT: LOG_OUTPUT: ALL
%oper ALL = NOPASSWD: LOG_INPUT: LOG_OUTPUT: SOFTWARE, SERVICES, PROCESSES
...

The log is pretty cryptic: for each user a directory with several files containing the input/outputs is created.

This solution is useful to have as reference, but be prepared to invest time in order to reconstruct the user activity.

[root@mos ~]# cd /var/log/sudo-io/
[root@mos ~]# find .
.
./00
./00/00
./00/00/02
./00/00/02/timing
./00/00/02/stderr
./00/00/02/log
./00/00/02/ttyin
./00/00/02/ttyout
./00/00/02/stdin
./00/00/02/stdout

Summary

We did not find the perfect solution. To summarize the pros and contras of each solution:

Each of them has strengths and weaknesses. Since auditd and sudo are a standard part of every secure system, we ended up tuning this two tools.

About the Author

Rocco Gagliardi has been working in IT since the 1980s and specialized in IT security in the 1990s. His main focus lies in security frameworks, network routing, firewalling and log management.

Links

• http://linux.die.net/man/1/rootsh
• http://linux.die.net/man/8/auditd
• http://linux.die.net/man/8/aureport
• http://linux.die.net/man/8/pam_tty_audit
• http://www.sudo.ws
• https://duckduckgo.com/?q=log+_bash_+commands&ia=qa