Protecting your Linux server against security exploits with AppArmor

AppArmor is a security feature available in the Linux kernel which adds Mandatory Access Control (MAC) to your system. Mandatory Access Control defines a security policy which the administrator of the system defines and cannot be overridden by the user. You use AppArmor to harden your system against all kinds of attacks, protecting your system against known or unknown zero-day exploits. It does so by restricting the applications on your system. You define an AppArmor profile for security sensitive applications which confines them to access only specific files, network access and other capabilities. This way it will become much harder or even impossible for an attacker to abuse a security hole in a service for which you have defined an AppArmor profile. SuSE sometimes calls this immunization.

AppArmor is similar to SELinux (Security-Enhanced Linux), the other well-known Linux security module which is used by default on Fedora and Red Hat Enterprise Linux. Debian and Ubuntu have opted for AppArmor by default. AppArmor profiles are a bit easier to manage than SELinux.

Setting up Apparmor on Debian

First we install all the AppArmor utilities and all default profiles:

# apt install apparmor apparmor-profiles apparmor-utils apparmor-profiles-extra libpam-apparmor auditd

With the aa-status command you can check the AppArmor status:

# aa-status
43 profiles are loaded.
20 profiles are in enforce mode.
23 profiles are in complain mode.
0 profiles are in kill mode.
0 profiles are in unconfined mode.
17 processes have profiles defined.
0 processes are in enforce mode.
14 processes are in complain mode.
   /usr/sbin/php-fpm8.2 (834763) php-fpm
   /usr/sbin/php-fpm8.2 (834764) php-fpm
   /usr/sbin/php-fpm8.2 (834765) php-fpm
   /usr/sbin/php-fpm8.2 (834766) php-fpm
   /usr/sbin/php-fpm8.2 (834767) php-fpm
   /usr/sbin/php-fpm8.2 (834768) php-fpm
   /usr/sbin/php-fpm8.2 (834769) php-fpm
   /usr/sbin/php-fpm8.2 (834770) php-fpm
   /usr/sbin/php-fpm8.2 (834771) php-fpm
   /usr/sbin/php-fpm8.2 (834772) php-fpm
   /usr/sbin/php-fpm8.2 (834773) php-fpm
   /usr/sbin/php-fpm8.2 (834774) php-fpm
   /usr/sbin/php-fpm8.2 (834775) php-fpm
   /usr/sbin/php-fpm8.2 (834826) php-fpm
3 processes are unconfined but have a profile defined.
   /usr/bin/freshclam (797288)
   /usr/sbin/clamd (662585)
   /usr/sbin/sssd (647439)
0 processes are in mixed mode.
0 processes are in kill mode.

We see that AppArmor is enabled and that it has 32 profiles of which 16 are being enforced and another 16 are in complain mode, which means that a warning will be logged when they violate the policy, but they will not be blocked. Currently 1 running process is confined in enforce mode.

All the profiles are installed in /etc/apparmor.d. The text files, usually named after the full path to the binary with the slashes replaced by dots.

To completely disable an Apparmor profile you use the aa-disable command. For example:

# aa-disable /usr/sbin/haveged

To enable a profile in complain mode, use the aa-complain command:

# aa-complain /usr/sbin/haveged

Use the aa-enforce command to set enforce an Apparmor profile:

# aa-enforce /usr/sbin/haveged

If you make a modification to one of the profiles in /etc/apparmor.d, you will need to make the kernel reload the profile with the apparmor_parser command:

# apparmor_parser -r /etc/apparmor.d/usr.sbin.haveged

If you want to remove a profile currently loaded in the kernel, you can use this command:

# apparmor_parser -R /etc/apparmor.d/usr.sbin.haveged

When you have done this, you can remove the file if you don’t need it any more.

Creating Apparmor profiles with aa-genprof

I recommend creating an AppArmor profile for every service which is accessible via the network end/or deals with data from the outside world.

You can run the aa-unconfined command to find applications which listen on a TCP or UDP port and do not have an AppArmor profile loaded. These are excellent candidates to create a profile for.

Example 1: Creating an AppArmor profile for Knot Resolver

One of the unconfined processes indicated by aa-unconfined, is /usr/sbin/kresd which is Knot Resolver, the caching DNS resolver. I will generate a profile with aa-genprof. I need two shells for that: one where I run aa-genprof, and another one where I exercise the functionality of Knot Resolver. aa-genprof creates an empty profile and puts it in complain mode, so that all events generated by the process will be logged. aa-genprof will then propose rules for any of them.

So in the first shell I run:

# aa-genprof /usr/sbin/kresd
Updating AppArmor profiles in /etc/apparmor.d.

Before you begin, you may wish to check if a
profile already exists for the application you
wish to confine. See the following wiki page for
more information:

Profiling: /usr/sbin/kresd

Please start the application to be profiled in
another window and exercise its functionality now.

Once completed, select the "Scan" option below in
order to scan the system logs for AppArmor events.

For each AppArmor event, you will be given the
opportunity to choose whether the access should be
allowed or denied.

[(S)can system log for AppArmor events] / (F)inish

Now in the second shell we need to exercise the functionality of kresd. Let’s start by stopping, starting and restarting kresd:

# systemctl stop system-kresd.slice && systemctl start && systemctl restart system-kresd.slice

I also recommend reloading the service, but the kresd service does not support this, so this is of no use here. Now we exercise kresd by doing some DNS lookups, for example with the host and the kdig commands:

$ host
$ kdig -t mx

Wait for some time, and then you go back to the shell where aa-genprof is running and press S to scan the audit log for any generated events.

Profile:    /usr/sbin/kresd
Capability: net_bind_service
Severity:   8

 [1 - include <abstractions/nis>]
  2 - capability net_bind_service,
(A)llow / [(D)eny] / (I)gnore / Audi(t) / Abo(r)t / (F)inish

Here kresd requests the net_bind_service capability. This is needed to listen to a privileged port (port number < 1024). aa-genprof proposes 2 possible rules. The first option is to include abstractions/nis which contains capability net_bind_service. Files in /etc/apparmor.d/abstractions contain common rules which may be included in multiple different profiles. Often these abstractions are too broad and in case of doubt I recommend to choose the more specific option, which is option 2 here. I press 2 select this option, then press A to allow this.

Then Knot Resolver checks whether we have transparent hugepages enabled by reading /sys/kernel/mm/transparent_hugepage/enabled. This is because Knot Resolve uses jemalloc which reads this file.

Profile:  /usr/sbin/kresd
Path:     /sys/kernel/mm/transparent_hugepage/enabled
New Mode: r
Severity: 4

 [1 - /sys/kernel/mm/transparent_hugepage/enabled r,]
(A)llow / [(D)eny] / (I)gnore / (G)lob / Glob with (E)xtension / (N)ew / Audi(t) / Abo(r)t / (F)inish

I press A to allow this.

Knot Resolver uses lua for plug-ins, and so wants to read lua code:

Profile:  /usr/sbin/kresd
Path:     /usr/share/lua/5.1/cqueues/socket.lua
New Mode: r
Severity: unknown

 [1 - include <abstractions/totem>]
  2 - /usr/share/lua/5.1/cqueues/socket.lua r,
(A)llow / [(D)eny] / (I)gnore / (G)lob / Glob with (E)xtension / (N)ew / Audi(t) / Abo(r)t / (F)inish

Now I want to allow reading of all lua files in /usr/share/lua, independent of the lua version, so that when Knot Resolver starts using a newer lua version in the future, the rule is still valid. Press 2 select that rule and then press E for glob with extension multiple times, until you get option 5 - /usr/share/lua/**.lua r,. This matches all files which name end with .lua anywhere within /usr/share/lua. Make sure this options is selected and then press A to allow this.

kresd creates a control file:

Profile:  /usr/sbin/kresd
Path:     /run/knot-resolver/control/1
New Mode: owner w
Severity: unknown

 [1 - owner /run/knot-resolver/control/1 w,]
(A)llow / [(D)eny] / (I)gnore / (G)lob / Glob with (E)xtension / (N)ew / Audi(t) / (O)wner permissions off / Abo(r)t / (F)inish

The mode owner w means the permission to write the file only if the user as which the writing process is running is also the owner of the file.

If you are running multiple kresd processes, each process will create its own control file, so we want to use a wildcard here. Press G for glob so that you get the option 2 – owner /run/knot-resolver/control/* w, and press A to accept this.

kresd tries to take an exclusive lock on /var/cache/knot-resolver/lock.mdb:

Profile:  /usr/sbin/kresd
Path:     /var/cache/knot-resolver/lock.mdb
New Mode: owner rwk
Severity: unknown

 [1 - owner /var/cache/knot-resolver/lock.mdb rwk,]
(A)llow / [(D)eny] / (I)gnore / (G)lob / Glob with (E)xtension / (N)ew / Audi(t) / (O)wner permissions off / Abo(r)t / (F)inish

Notice the mode rwk: k is the permission, and in combination with w this is an exclusive lock. You can find the explanation of all file permissions in the AppArmor Core Policy Reference. Press A to allow this.

I’m using rpz-downloader to download RPZ files containing malicious domains I want to block. Knot Resolver wants to read these RPZ files:

Profile:  /usr/sbin/kresd
Path:     /var/lib/rpz-downloader/
New Mode: r
Severity: unknown

 [1 - /var/lib/rpz-downloader/ r,]
(A)llow / [(D)eny] / (I)gnore / (G)lob / Glob with (E)xtension / (N)ew / Audi(t) / Abo(r)t / (F)inish

I want to allow access to all RPZ files in /var/lib/rpz-downloader, so I press N and I change the path to /var/lib/rpz-downloader/*.rpz and press A to accept this.

Knot Resolver tries to open an IPv4 TCP socket:

Profile:        /usr/sbin/kresd
Network Family: inet
Socket Type:    stream

 [1 - include <abstractions/apache2-common>]
  2 - include <abstractions/nameservice>
  3 - network inet stream,
(A)llow / [(D)eny] / (I)gnore / Audi(t) / Abo(r)t / (F)inish

The proposed abstractions are too broad, so I select 3. You will also need to allow network inet dgram (IPv4 UDP), network inet6 stream (IPv6 TCP) and network inet6 dgram (IPv6 UDP).

When you have processed all events, you will get this:

= Changed Local Profiles =

The following local profiles were changed. Would you like to save them?

 [1 - /usr/sbin/kresd]
(S)ave Changes / Save Selec(t)ed Profile / [(V)iew Changes] / View Changes b/w (C)lean profiles / Abo(r)t

Only one profile is modified and you can view the changes by pressing V. You can save the changes by pressing S. After doing so, aa-genprof will continue running and you can press S to scan the logs again for any newly generated events. This way you can gradually improve your profile. If you are finished, you can quit aa-genprof by pressing F. You can run aa-genprof for your process again at any time, it will then modify the existing profile if it finds any new events.

When you have finished; check with aa-status whether your profile is still in complain mode, and if not change it back to complain mode with aa-complain.

This is how my final profile /etc/apparmor.d/usr.sbin.kresd looks:

abi <abi/3.0>,

include <tunables/global>

/usr/sbin/kresd flags=(complain) {
  include <abstractions/base>

  capability net_bind_service,

  network inet dgram,
  network inet stream,
  network inet6 dgram,
  network inet6 stream,

  /etc/group r,
  /etc/knot-resolver/kresd.conf r,
  /etc/nsswitch.conf r,
  /etc/passwd r,
  /etc/ssl/certs/ca-certificates.crt r,
  /etc/ssl/openssl.cnf r,
  /sys/kernel/mm/transparent_hugepage/enabled r,
  /usr/sbin/kresd mr,
  /usr/share/dns/root.hints r,
  /usr/share/dns/root.key r,
  /usr/share/lua/**.lua r,
  /var/lib/rpz-downloader/*.rpz r,
  owner /run/knot-resolver/control/ w,
  owner /run/knot-resolver/control/* w,
  owner /var/cache/knot-resolver/data.mdb rw,
  owner /var/cache/knot-resolver/lock.mdb rwk,


The flags=(complain) indicate that this profile will be loaded in complain mode. Keep it like that for some time until you are sure no new events are generated any more. You can check this by running aa-logprof. aa-logprof is similar to aa-genprof, except that it also processes past events logged in /var/log/audit.log and processes events for all processes for which you have defined an AppArmmor profile, and not a single one like aa-genprof. If you are sure the profile is complete, enforce it by running aa-enforce /usr/sbin/kresd.

Example 2: creating an AppArmor profile for Postfix

Another process marked by aa-unconfined on my system is /usr/lib/postfix/sbin/master, Postfix’ master process. I create a profile with aa-genprof:

# aa-genprof /usr/lib/postfix/sbin/master

In another shell, I stop, start, restart and reload Postfix:

# systemctl stop postfix@- && systemctl start postfix@- && systemctl restart postfix@- && systemctl reload postfix@-

To further exercise Postfix, send some mails, both outgoing mails to other mail servers as incoming mails originating from an external mail server. Check the mail queue with

# mailq

I’m not going through all events generated by Postfix, but I want to point out one particular type of event which we did not see with Knot Resolver:

Profile:  /usr/lib/postfix/sbin/master
Execute:  /usr/lib/postfix/sbin/showq
Severity: unknown

(I)nherit / (C)hild / (P)rofile / (N)amed / (U)nconfined / (X) ix On / (D)eny / Abo(r)t / (F)inish

The master process tries to start another executable, in this case showq. You have several options:

  • (I)nherit: the new process inherits the profile from the parent process. This means that it will run with the same permissions as the parent process.
  • (C)hild: the new process will use a subprofile defined within this profile. This is useful if you want to run the process with a different profile, depending on how it was started.
  • (P)rofile: the new process will run in its own generic profile, in this case /etc/apparmor.d/usr.lib.postfix.sbin.showq . Use this if you want to run it in the same profile as when you would have started /usr/lib/postfix/sbin/showq directly.
  • (N)amed: the new process will run with the profile with a name of your choice.
  • (U)confined: the new process will run unconfined, without any AppArmor restrictions.

Here I choose P because I want showq to run always with the same profile, no matter how it is started. Not only will aa-genprof create a rule which allows master to launch showq, it will also create a profile for showq, and start logging future events.

Then you will get the question whether you want to sanitize the environment:

Should AppArmor sanitise the environment when
switching profiles?

Sanitising environment is more secure,
but some applications depend on the presence

[(Y)es] / (N)o

My Postfix installation does not need any enviroment variables like LD_LIBRARY_PATH to be set, so it can safely sanitize the environment. I press Y.

When you have finished, make sure that all profiles are running in complain mode, otherwise you risk problems with mail delivery:

# cd /etc/apparmor.d
# for i in usr.lib.postfix.sbin.*; do aa-complain $i; done

My /etc/apparmor.d/usr.lib.postfix.sbin.master profile looks like this at the moment after some manual tweaking:

abi <abi/3.0>,

include <tunables/global>

/usr/lib/postfix/sbin/master flags=(complain) {
  include <abstractions/base>
  include <abstractions/postfix-common>

  capability dac_read_search,
  capability kill,
  capability net_bind_service,

  network inet stream,
  network inet6 dgram,
  network inet6 stream,
  network netlink raw,

  signal send peer=/usr/lib/postfix/sbin/*,

  /usr/lib/postfix/sbin/anvil Px,
  /usr/lib/postfix/sbin/cleanup Px,
  /usr/lib/postfix/sbin/lmtp Px,
  /usr/lib/postfix/sbin/local Px,
  /usr/lib/postfix/sbin/master mr,
  /usr/lib/postfix/sbin/pickup Px,
  /usr/lib/postfix/sbin/postscreen Px,
  /usr/lib/postfix/sbin/proxymap Px,
  /usr/lib/postfix/sbin/qmgr Px,
  /usr/lib/postfix/sbin/scache Px,
  /usr/lib/postfix/sbin/showq Px,
  /usr/lib/postfix/sbin/smtp Px,
  /usr/lib/postfix/sbin/smtpd Px,
  /usr/lib/postfix/sbin/tlsmgr Px,
  /usr/lib/postfix/sbin/trivial-rewrite Px,
  owner /etc/gai.conf r,
  owner /etc/group r,
  owner /etc/nsswitch.conf r,
  owner /etc/passwd r,
  owner /var/lib/postfix/master.lock rwk,
  owner /var/spool/postfix/pid/ rwk,
  owner /var/spool/postfix/private/anvil w,
  owner /var/spool/postfix/private/bounce w,
  owner /var/spool/postfix/private/bsmtp w,
  owner /var/spool/postfix/private/defer w,
  owner /var/spool/postfix/private/discard w,
  owner /var/spool/postfix/private/dnsblog w,
  owner /var/spool/postfix/private/error w,
  owner /var/spool/postfix/private/ifmail w,
  owner /var/spool/postfix/private/lmtp w,
  owner /var/spool/postfix/private/local w,
  owner /var/spool/postfix/private/maildrop w,
  owner /var/spool/postfix/private/mailman w,
  owner /var/spool/postfix/private/proxymap w,
  owner /var/spool/postfix/private/proxywrite w,
  owner /var/spool/postfix/private/relay w,
  owner /var/spool/postfix/private/retry w,
  owner /var/spool/postfix/private/rewrite w,
  owner /var/spool/postfix/private/scache w,
  owner /var/spool/postfix/private/scalemail-backend w,
  owner /var/spool/postfix/private/smtp w,
  owner /var/spool/postfix/private/smtp-amavis w,
  owner /var/spool/postfix/private/smtpd w,
  owner /var/spool/postfix/private/tlsmgr w,
  owner /var/spool/postfix/private/tlsproxy w,
  owner /var/spool/postfix/private/trace w,
  owner /var/spool/postfix/private/uucp w,
  owner /var/spool/postfix/private/verify w,
  owner /var/spool/postfix/private/virtual w,
  owner /var/spool/postfix/public/cleanup w,
  owner /var/spool/postfix/public/flush w,
  owner /var/spool/postfix/public/pickup rw,
  owner /var/spool/postfix/public/qmgr rw,
  owner /var/spool/postfix/public/showq w,


I’m not going to add all profiles for other Postfix processes here, but you get the idea. More rules might still be needed, so that’s why I keep them running in complain mode and I regularly run aa-logprof.

Debugging problems caused by AppArmor

AppAmor logs events in /var/log/audit/audit.log if you have auditd running. You can grep for apparmor to see all events.

To easily process all new events logged in /var/log/audit/audit.log you can use aa-logprof:

# aa-logprof

Sources and more information

Secure and private DNS with Knot Resolver

Update 5 March 2018: this post was updated to work around a problem with the RPZ file from being ignored because it contains CRLF instead of LF where Knot Resolver does not expect them (bug 453) and to fix an error in the configuration of the predict module.

Knot Resolver is a modern, feature-rich recursive DNS server. It is used by Cloudflare for its public DNS service.

To install it on Debian, run:

# apt-get install knot-resolver knot-dnsutils lua-cqueues

The knot-dnsutils contains the kdig command which is useful for testing your DNS server. lua-cqueues is needed for automatic detection of changes in the RPZ file.

By default the kresd daemon will listen on localhost only (see /lib/systemd/system/kresd.socket). If you want it to be available on other addresses, you will need to override the kresd.socket file. Execute

# systemctl edit kresd.socket

This will create the file /etc/systemd/system/kresd.socket.d/override.conf. Add a ListenStream and ListenDatagram line for all addresses you want it to listen on. For example:


If you want to listen on all interfaces, it is enough to put this in the file:


You can do the same with kresd-tls.socket to define the addresses on which to listen over DNS-over-TLS requests (port 853).

Knot Resolver’s configuration file is /etc/knot-resolver/kresd.conf. I give an example configuration file with comments:

-- Default empty Knot DNS Resolver configuration in -*- lua -*-
-- Switch to unprivileged user --

-- Set the size of the cache to 1 GB
cache.size = 1*GB

-- Uncomment this only if you need to debug problems.
-- verbose(true)

-- Enable optional modules
modules = {
  'serve_stale < cache',
  'workarounds < iterate',

-- Accept all requests from these subnets
view:addr('', function (req, qry) return policy.PASS end)
view:addr('[::1]/128', function (req, qry) return policy.PASS end)
view:addr('', function (req, qry) return policy.PASS end)

-- Drop everything that hasn't matched
view:addr('', function (req, qry) return policy.DROP end)

-- Use the RPZ list
policy.add(policy.rpz(policy.DENY, '/etc/knot-resolver/',true))

-- Forward all requests for to and
policy.add(policy.suffix(policy.FORWARD({'', ''}), {todname('')}))

-- Uncomment one of the following stanzas in case you want to forward all requests to or via DNS-over-TLS.

-- policy.add(policy.all(policy.TLS_FORWARD({
--          { '', hostname='', ca_file='/etc/ssl/certs/ca-certificates.crt' },
--          { '2606:4700:4700::1111', hostname='', ca_file='/etc/ssl/certs/ca-certificates.crt' },
-- })))

-- policy.add(policy.all(policy.TLS_FORWARD({
--           { '', hostname='', ca_file='/etc/ssl/certs/ca-certificates.crt' },
--           { '2620:fe::fe', hostname='', ca_file='/etc/ssl/certs/ca-certificates.crt' },
-- })))

-- Prefetch learning (20-minute blocks over 24 hours)
predict.config({ window = 20, period = 72 })

I use the RPZ file, which contains a blacklist of malicious domains. You will have to download it first:

# cd /etc/knot-resolver
# curl | sed -e 's/\r$//' -e '/'> /etc/knot-resolver/

I use sed to convert CRLF in LF (otherwise Knot Resolver fails to parse the file), and I filter out According to it hosts some malware, but there is too much useful stuff there too to block the domain completely.

In order to update it automatically, create /etc/systemd/system/update-urlhaus-abuse-ch.service:

Description=Update RPZ file from for Knot Resolver

ExecStart=/bin/sh -c 'curl | sed -e 's/\r$//' -e '/'> /etc/knot-resolver/'

and then create a timer which will run the service approximately every 10-15 minutes./etc/systemd/system/update-urlhaus-abuse-ch.timer:

Description=Update RPZ file from for Knot Resolver



Use the first two commands to enable and start the timer. You can check the status using the last command:

# systemctl enable update-urlhaus-abuse-ch.timer
# systemctl start update-urlhaus-abuse-ch.timer
# systemctl list-timers

Now you need to enable and start one or more instances of kresd. kresd is single-threaded, so if you want to make use of all of your CPU cores, you can start as many instances as the numbers of cores you have. For example in order to enable and start 4 instances run this command:

# systemctl enable --now kresd@{1..4}.service

More information