Protecting your Linux server against security exploits with AppArmor

AppArmor is a security feature available in the Linux kernel which adds Mandatory Access Control (MAC) to your system. Mandatory Access Control defines a security policy which the administrator of the system defines and cannot be overridden by the user. You use AppArmor to harden your system against all kinds of attacks, protecting your system against known or unknown zero-day exploits. It does so by restricting the applications on your system. You define an AppArmor profile for security sensitive applications which confines them to access only specific files, network access and other capabilities. This way it will become much harder or even impossible for an attacker to abuse a security hole in a service for which you have defined an AppArmor profile. SuSE sometimes calls this immunization.

AppArmor is similar to SELinux (Security-Enhanced Linux), the other well-known Linux security module which is used by default on Fedora and Red Hat Enterprise Linux. Debian and Ubuntu have opted for AppArmor by default. AppArmor profiles are a bit easier to manage than SELinux.

Setting up Apparmor on Debian

First we install all the AppArmor utilities and all default profiles:

# apt install apparmor apparmor-profiles apparmor-utils apparmor-profiles-extra libpam-apparmor auditd

With the aa-status command you can check the AppArmor status:

# aa-status
43 profiles are loaded.
20 profiles are in enforce mode.
   /usr/bin/freshclam
   /usr/bin/man
   /usr/bin/pidgin
   /usr/bin/pidgin//sanitized_helper
   /usr/bin/totem
   /usr/bin/totem-audio-preview
   /usr/bin/totem-video-thumbnailer
   /usr/bin/totem//sanitized_helper
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/NetworkManager/nm-dhcp-helper
   /usr/lib/connman/scripts/dhclient-script
   /usr/sbin/clamd
   /{,usr/}sbin/dhclient
   apt-cacher-ng
   lsb_release
   man_filter
   man_groff
   nvidia_modprobe
   nvidia_modprobe//kmod
   tcpdump
23 profiles are in complain mode.
   /usr/bin/irssi
   /usr/sbin/sssd
   avahi-daemon
   dnsmasq
   dnsmasq//libvirt_leaseshelper
   identd
   klogd
   mdnsd
   nmbd
   nscd
   php-fpm
   ping
   samba-bgqd
   samba-dcerpcd
   samba-rpcd
   samba-rpcd-classic
   samba-rpcd-spoolss
   smbd
   smbldap-useradd
   smbldap-useradd///etc/init.d/nscd
   syslog-ng
   syslogd
   traceroute
0 profiles are in kill mode.
0 profiles are in unconfined mode.
17 processes have profiles defined.
0 processes are in enforce mode.
14 processes are in complain mode.
   /usr/sbin/php-fpm8.2 (834763) php-fpm
   /usr/sbin/php-fpm8.2 (834764) php-fpm
   /usr/sbin/php-fpm8.2 (834765) php-fpm
   /usr/sbin/php-fpm8.2 (834766) php-fpm
   /usr/sbin/php-fpm8.2 (834767) php-fpm
   /usr/sbin/php-fpm8.2 (834768) php-fpm
   /usr/sbin/php-fpm8.2 (834769) php-fpm
   /usr/sbin/php-fpm8.2 (834770) php-fpm
   /usr/sbin/php-fpm8.2 (834771) php-fpm
   /usr/sbin/php-fpm8.2 (834772) php-fpm
   /usr/sbin/php-fpm8.2 (834773) php-fpm
   /usr/sbin/php-fpm8.2 (834774) php-fpm
   /usr/sbin/php-fpm8.2 (834775) php-fpm
   /usr/sbin/php-fpm8.2 (834826) php-fpm
3 processes are unconfined but have a profile defined.
   /usr/bin/freshclam (797288)
   /usr/sbin/clamd (662585)
   /usr/sbin/sssd (647439)
0 processes are in mixed mode.
0 processes are in kill mode.

We see that AppArmor is enabled and that it has 32 profiles of which 16 are being enforced and another 16 are in complain mode, which means that a warning will be logged when they violate the policy, but they will not be blocked. Currently 1 running process is confined in enforce mode.

All the profiles are installed in /etc/apparmor.d. The text files, usually named after the full path to the binary with the slashes replaced by dots.

To completely disable an Apparmor profile you use the aa-disable command. For example:

# aa-disable /usr/sbin/haveged

To enable a profile in complain mode, use the aa-complain command:

# aa-complain /usr/sbin/haveged

Use the aa-enforce command to set enforce an Apparmor profile:

# aa-enforce /usr/sbin/haveged

If you make a modification to one of the profiles in /etc/apparmor.d, you will need to make the kernel reload the profile with the apparmor_parser command:

# apparmor_parser -r /etc/apparmor.d/usr.sbin.haveged

If you want to remove a profile currently loaded in the kernel, you can use this command:

# apparmor_parser -R /etc/apparmor.d/usr.sbin.haveged

When you have done this, you can remove the file if you don’t need it any more.

Creating Apparmor profiles with aa-genprof

I recommend creating an AppArmor profile for every service which is accessible via the network end/or deals with data from the outside world.

You can run the aa-unconfined command to find applications which listen on a TCP or UDP port and do not have an AppArmor profile loaded. These are excellent candidates to create a profile for.

Example 1: Creating an AppArmor profile for Knot Resolver

One of the unconfined processes indicated by aa-unconfined, is /usr/sbin/kresd which is Knot Resolver, the caching DNS resolver. I will generate a profile with aa-genprof. I need two shells for that: one where I run aa-genprof, and another one where I exercise the functionality of Knot Resolver. aa-genprof creates an empty profile and puts it in complain mode, so that all events generated by the process will be logged. aa-genprof will then propose rules for any of them.

So in the first shell I run:

# aa-genprof /usr/sbin/kresd
Updating AppArmor profiles in /etc/apparmor.d.

Before you begin, you may wish to check if a
profile already exists for the application you
wish to confine. See the following wiki page for
more information:
https://gitlab.com/apparmor/apparmor/wikis/Profiles

Profiling: /usr/sbin/kresd

Please start the application to be profiled in
another window and exercise its functionality now.

Once completed, select the "Scan" option below in
order to scan the system logs for AppArmor events.

For each AppArmor event, you will be given the
opportunity to choose whether the access should be
allowed or denied.

[(S)can system log for AppArmor events] / (F)inish

Now in the second shell we need to exercise the functionality of kresd. Let’s start by stopping, starting and restarting kresd:

# systemctl stop system-kresd.slice && systemctl start kresd.target && systemctl restart system-kresd.slice

I also recommend reloading the service, but the kresd service does not support this, so this is of no use here. Now we exercise kresd by doing some DNS lookups, for example with the host and the kdig commands:

$ host example.com
$ kdig -t mx debian.org

Wait for some time, and then you go back to the shell where aa-genprof is running and press S to scan the audit log for any generated events.

Profile:    /usr/sbin/kresd
Capability: net_bind_service
Severity:   8

 [1 - include <abstractions/nis>]
  2 - capability net_bind_service,
(A)llow / [(D)eny] / (I)gnore / Audi(t) / Abo(r)t / (F)inish

Here kresd requests the net_bind_service capability. This is needed to listen to a privileged port (port number < 1024). aa-genprof proposes 2 possible rules. The first option is to include abstractions/nis which contains capability net_bind_service. Files in /etc/apparmor.d/abstractions contain common rules which may be included in multiple different profiles. Often these abstractions are too broad and in case of doubt I recommend to choose the more specific option, which is option 2 here. I press 2 select this option, then press A to allow this.

Then Knot Resolver checks whether we have transparent hugepages enabled by reading /sys/kernel/mm/transparent_hugepage/enabled. This is because Knot Resolve uses jemalloc which reads this file.

Profile:  /usr/sbin/kresd
Path:     /sys/kernel/mm/transparent_hugepage/enabled
New Mode: r
Severity: 4

 [1 - /sys/kernel/mm/transparent_hugepage/enabled r,]
(A)llow / [(D)eny] / (I)gnore / (G)lob / Glob with (E)xtension / (N)ew / Audi(t) / Abo(r)t / (F)inish

I press A to allow this.

Knot Resolver uses lua for plug-ins, and so wants to read lua code:

Profile:  /usr/sbin/kresd
Path:     /usr/share/lua/5.1/cqueues/socket.lua
New Mode: r
Severity: unknown

 [1 - include <abstractions/totem>]
  2 - /usr/share/lua/5.1/cqueues/socket.lua r,
(A)llow / [(D)eny] / (I)gnore / (G)lob / Glob with (E)xtension / (N)ew / Audi(t) / Abo(r)t / (F)inish

Now I want to allow reading of all lua files in /usr/share/lua, independent of the lua version, so that when Knot Resolver starts using a newer lua version in the future, the rule is still valid. Press 2 select that rule and then press E for glob with extension multiple times, until you get option 5 - /usr/share/lua/**.lua r,. This matches all files which name end with .lua anywhere within /usr/share/lua. Make sure this options is selected and then press A to allow this.

kresd creates a control file:

Profile:  /usr/sbin/kresd
Path:     /run/knot-resolver/control/1
New Mode: owner w
Severity: unknown

 [1 - owner /run/knot-resolver/control/1 w,]
(A)llow / [(D)eny] / (I)gnore / (G)lob / Glob with (E)xtension / (N)ew / Audi(t) / (O)wner permissions off / Abo(r)t / (F)inish

The mode owner w means the permission to write the file only if the user as which the writing process is running is also the owner of the file.

If you are running multiple kresd processes, each process will create its own control file, so we want to use a wildcard here. Press G for glob so that you get the option 2 – owner /run/knot-resolver/control/* w, and press A to accept this.

kresd tries to take an exclusive lock on /var/cache/knot-resolver/lock.mdb:

Profile:  /usr/sbin/kresd
Path:     /var/cache/knot-resolver/lock.mdb
New Mode: owner rwk
Severity: unknown

 [1 - owner /var/cache/knot-resolver/lock.mdb rwk,]
(A)llow / [(D)eny] / (I)gnore / (G)lob / Glob with (E)xtension / (N)ew / Audi(t) / (O)wner permissions off / Abo(r)t / (F)inish

Notice the mode rwk: k is the permission, and in combination with w this is an exclusive lock. You can find the explanation of all file permissions in the AppArmor Core Policy Reference. Press A to allow this.

I’m using rpz-downloader to download RPZ files containing malicious domains I want to block. Knot Resolver wants to read these RPZ files:

Profile:  /usr/sbin/kresd
Path:     /var/lib/rpz-downloader/urlhaus.abuse.ch.rpz
New Mode: r
Severity: unknown

 [1 - /var/lib/rpz-downloader/urlhaus.abuse.ch.rpz r,]
(A)llow / [(D)eny] / (I)gnore / (G)lob / Glob with (E)xtension / (N)ew / Audi(t) / Abo(r)t / (F)inish

I want to allow access to all RPZ files in /var/lib/rpz-downloader, so I press N and I change the path to /var/lib/rpz-downloader/*.rpz and press A to accept this.

Knot Resolver tries to open an IPv4 TCP socket:

Profile:        /usr/sbin/kresd
Network Family: inet
Socket Type:    stream

 [1 - include <abstractions/apache2-common>]
  2 - include <abstractions/nameservice>
  3 - network inet stream,
(A)llow / [(D)eny] / (I)gnore / Audi(t) / Abo(r)t / (F)inish

The proposed abstractions are too broad, so I select 3. You will also need to allow network inet dgram (IPv4 UDP), network inet6 stream (IPv6 TCP) and network inet6 dgram (IPv6 UDP).

When you have processed all events, you will get this:

= Changed Local Profiles =

The following local profiles were changed. Would you like to save them?

 [1 - /usr/sbin/kresd]
(S)ave Changes / Save Selec(t)ed Profile / [(V)iew Changes] / View Changes b/w (C)lean profiles / Abo(r)t

Only one profile is modified and you can view the changes by pressing V. You can save the changes by pressing S. After doing so, aa-genprof will continue running and you can press S to scan the logs again for any newly generated events. This way you can gradually improve your profile. If you are finished, you can quit aa-genprof by pressing F. You can run aa-genprof for your process again at any time, it will then modify the existing profile if it finds any new events.

When you have finished; check with aa-status whether your profile is still in complain mode, and if not change it back to complain mode with aa-complain.

This is how my final profile /etc/apparmor.d/usr.sbin.kresd looks:

abi <abi/3.0>,

include <tunables/global>

/usr/sbin/kresd flags=(complain) {
  include <abstractions/base>

  capability net_bind_service,

  network inet dgram,
  network inet stream,
  network inet6 dgram,
  network inet6 stream,

  /etc/group r,
  /etc/knot-resolver/kresd.conf r,
  /etc/nsswitch.conf r,
  /etc/passwd r,
  /etc/ssl/certs/ca-certificates.crt r,
  /etc/ssl/openssl.cnf r,
  /sys/kernel/mm/transparent_hugepage/enabled r,
  /usr/sbin/kresd mr,
  /usr/share/dns/root.hints r,
  /usr/share/dns/root.key r,
  /usr/share/lua/**.lua r,
  /var/lib/rpz-downloader/*.rpz r,
  owner /run/knot-resolver/control/ w,
  owner /run/knot-resolver/control/* w,
  owner /var/cache/knot-resolver/data.mdb rw,
  owner /var/cache/knot-resolver/lock.mdb rwk,

}

The flags=(complain) indicate that this profile will be loaded in complain mode. Keep it like that for some time until you are sure no new events are generated any more. You can check this by running aa-logprof. aa-logprof is similar to aa-genprof, except that it also processes past events logged in /var/log/audit.log and processes events for all processes for which you have defined an AppArmmor profile, and not a single one like aa-genprof. If you are sure the profile is complete, enforce it by running aa-enforce /usr/sbin/kresd.

Example 2: creating an AppArmor profile for Postfix

Another process marked by aa-unconfined on my system is /usr/lib/postfix/sbin/master, Postfix’ master process. I create a profile with aa-genprof:

# aa-genprof /usr/lib/postfix/sbin/master

In another shell, I stop, start, restart and reload Postfix:

# systemctl stop postfix@- && systemctl start postfix@- && systemctl restart postfix@- && systemctl reload postfix@-

To further exercise Postfix, send some mails, both outgoing mails to other mail servers as incoming mails originating from an external mail server. Check the mail queue with

# mailq

I’m not going through all events generated by Postfix, but I want to point out one particular type of event which we did not see with Knot Resolver:

Profile:  /usr/lib/postfix/sbin/master
Execute:  /usr/lib/postfix/sbin/showq
Severity: unknown

(I)nherit / (C)hild / (P)rofile / (N)amed / (U)nconfined / (X) ix On / (D)eny / Abo(r)t / (F)inish

The master process tries to start another executable, in this case showq. You have several options:

  • (I)nherit: the new process inherits the profile from the parent process. This means that it will run with the same permissions as the parent process.
  • (C)hild: the new process will use a subprofile defined within this profile. This is useful if you want to run the process with a different profile, depending on how it was started.
  • (P)rofile: the new process will run in its own generic profile, in this case /etc/apparmor.d/usr.lib.postfix.sbin.showq . Use this if you want to run it in the same profile as when you would have started /usr/lib/postfix/sbin/showq directly.
  • (N)amed: the new process will run with the profile with a name of your choice.
  • (U)confined: the new process will run unconfined, without any AppArmor restrictions.

Here I choose P because I want showq to run always with the same profile, no matter how it is started. Not only will aa-genprof create a rule which allows master to launch showq, it will also create a profile for showq, and start logging future events.

Then you will get the question whether you want to sanitize the environment:

Should AppArmor sanitise the environment when
switching profiles?

Sanitising environment is more secure,
but some applications depend on the presence
of LD_PRELOAD or LD_LIBRARY_PATH.

[(Y)es] / (N)o

My Postfix installation does not need any enviroment variables like LD_LIBRARY_PATH to be set, so it can safely sanitize the environment. I press Y.

When you have finished, make sure that all profiles are running in complain mode, otherwise you risk problems with mail delivery:

# cd /etc/apparmor.d
# for i in usr.lib.postfix.sbin.*; do aa-complain $i; done

My /etc/apparmor.d/usr.lib.postfix.sbin.master profile looks like this at the moment after some manual tweaking:

abi <abi/3.0>,

include <tunables/global>

/usr/lib/postfix/sbin/master flags=(complain) {
  include <abstractions/base>
  include <abstractions/postfix-common>

  capability dac_read_search,
  capability kill,
  capability net_bind_service,

  network inet stream,
  network inet6 dgram,
  network inet6 stream,
  network netlink raw,

  signal send peer=/usr/lib/postfix/sbin/*,

  /usr/lib/postfix/sbin/anvil Px,
  /usr/lib/postfix/sbin/cleanup Px,
  /usr/lib/postfix/sbin/lmtp Px,
  /usr/lib/postfix/sbin/local Px,
  /usr/lib/postfix/sbin/master mr,
  /usr/lib/postfix/sbin/pickup Px,
  /usr/lib/postfix/sbin/postscreen Px,
  /usr/lib/postfix/sbin/proxymap Px,
  /usr/lib/postfix/sbin/qmgr Px,
  /usr/lib/postfix/sbin/scache Px,
  /usr/lib/postfix/sbin/showq Px,
  /usr/lib/postfix/sbin/smtp Px,
  /usr/lib/postfix/sbin/smtpd Px,
  /usr/lib/postfix/sbin/tlsmgr Px,
  /usr/lib/postfix/sbin/trivial-rewrite Px,
  owner /etc/gai.conf r,
  owner /etc/group r,
  owner /etc/nsswitch.conf r,
  owner /etc/passwd r,
  owner /var/lib/postfix/master.lock rwk,
  owner /var/spool/postfix/pid/master.pid rwk,
  owner /var/spool/postfix/private/anvil w,
  owner /var/spool/postfix/private/bounce w,
  owner /var/spool/postfix/private/bsmtp w,
  owner /var/spool/postfix/private/defer w,
  owner /var/spool/postfix/private/discard w,
  owner /var/spool/postfix/private/dnsblog w,
  owner /var/spool/postfix/private/error w,
  owner /var/spool/postfix/private/ifmail w,
  owner /var/spool/postfix/private/lmtp w,
  owner /var/spool/postfix/private/local w,
  owner /var/spool/postfix/private/maildrop w,
  owner /var/spool/postfix/private/mailman w,
  owner /var/spool/postfix/private/proxymap w,
  owner /var/spool/postfix/private/proxywrite w,
  owner /var/spool/postfix/private/relay w,
  owner /var/spool/postfix/private/retry w,
  owner /var/spool/postfix/private/rewrite w,
  owner /var/spool/postfix/private/scache w,
  owner /var/spool/postfix/private/scalemail-backend w,
  owner /var/spool/postfix/private/smtp w,
  owner /var/spool/postfix/private/smtp-amavis w,
  owner /var/spool/postfix/private/smtpd w,
  owner /var/spool/postfix/private/tlsmgr w,
  owner /var/spool/postfix/private/tlsproxy w,
  owner /var/spool/postfix/private/trace w,
  owner /var/spool/postfix/private/uucp w,
  owner /var/spool/postfix/private/verify w,
  owner /var/spool/postfix/private/virtual w,
  owner /var/spool/postfix/public/cleanup w,
  owner /var/spool/postfix/public/flush w,
  owner /var/spool/postfix/public/pickup rw,
  owner /var/spool/postfix/public/qmgr rw,
  owner /var/spool/postfix/public/showq w,

}

I’m not going to add all profiles for other Postfix processes here, but you get the idea. More rules might still be needed, so that’s why I keep them running in complain mode and I regularly run aa-logprof.

Debugging problems caused by AppArmor

AppAmor logs events in /var/log/audit/audit.log if you have auditd running. You can grep for apparmor to see all events.

To easily process all new events logged in /var/log/audit/audit.log you can use aa-logprof:

# aa-logprof

Sources and more information