Frederik Himpe | Page 2 | Random thoughts of a Linux sysadmin

Web application firewall: Modsecurity and Core Rule Set

Posted on April 10, 2022October 6, 2022 by Frederik

A web application firewall (WAF) filters HTTP traffic. By integrating this in your web server, you can make sure potentially dangerous requests are blocked before they arrive to your web application or sensitive data leaks out of your web server. This way you add an extra defensive layer potentially offering extra protection against zero-day vulnerabilities in your web server or web applications. In this blog post, I give a tutorial how to install and configure ModSecurity web application firewall and the Core Rule Set on Debian. With some minor adaptions you can also use this guide for setting up ModSecurity on Ubuntu or other distributions.

ModSecurity is the most well-known open source web application firewall. The future of ModSecurity does not look too bright but fortunately with Coraza WAF an alternative which is completely compatible with ModSecurity is in development. At this moment Coraza only integrates with the Caddy web server, and does not have a connector for Apache or NGinx so for that reason it is currently not yet usable as a replacement for ModSecurity.

While ModSecurity provides the framework for filtering HTTP traffic, you also need rules which define what to bloc and that’s where the Core Rule Set (CRS) comes in. CRS is a set of generic rules winch offer protection to a various range of common attacks via HTTP, such as SQL injection, code injection and cross-site scripting (XSS) attacks.

Install ModSecurity and the Core Rule Set on Debian

I install the Apache module for ModSecurity, the geoip-database, which can be used for blocking all requests from certain countries, and modsecurity-crs, which contains the Core Rule Set. I take this package from testing, because it has a newer version (version 3.3.2 at the time of writing). There is no risk in taking this package from testing, because it only contains the rules and does not depend on any other packages from testing/unstable. If you prefer faster updates, you can also use unstable.

# apt install libapache2-mod-security2 geoip-database
# apt install -t testing modsecurity-crs

Configuring ModSecurity

In order to load the ModSecurity module in Apache, run this command:

# a2enmod security2

Then copy the example ModSecurity configuration file to /etc/modsecurity/modsecurity.conf:

cp /etc/modsecurity/modsecurity.conf-recommended /etc/modsecurity/modsecurity.conf

Now edit /etc/modsecurity/modsecurity.conf. I highlight some of the options:

SecRuleEngine on
SecRequestBodyLimit 536870912
SecRequestBodyNoFilesLimit 131072
SecAuditLog /var/log/apache2/modsec_audit.log
#SecRule MULTIPART_UNMATCHED_BOUNDARY "!@eq 0" \
#"id:'200004',phase:2,t:none,log,deny,msg:'Multipart parser detected a possible unmatched boundary.'"
SecPcreMatchLimit 500000
SecPcreMatchLimitRecursion 500000
SecStatusEngine Off

The SecRuleEngine option controls whether rules should be processed. If set to Off, you completely disable all rules, with On you enable them and it will block malicious actions. If set to DetectionOnly, ModSecurity will only log potential malicious activity flagged by your rules, but will not block them. DetectionOnly can be useful for temporary trying out the rules in order to find false positives before you really start blocking potential malicious activity.

The SecAuditLog option defines a file which contains audit logs. This file will contain detailed logs about every request triggering a ModSecurity rule.

The SecPcreMatchLimit and SecPcreMatchLimitRecursion set the match limit and match limit recursion for the regular expression library PCRE. Setting this high enough will prevent errors that the PCRE limits were exceeded while analyzing data, but setting it too high can make ModSecurity vulnerable to a Denial of Service (DoS) attack. A Core Rule Set developer recommends a value of 50000 so that’s what I use here.

I change SecRequestBodyLimit to a higher value to allow large file uploads.

I disable the rule 200004 because it is known to cause false positives.

Set SecStatusEngine to Off to prevent ModSecurity sending version information back its developers.

After changing any configuration related to ModSecurity or the Core Rule Set, reload your Apache web server:

# systemctl reload apache2

Configuring the Core Rule Set

The Core Rule Set can be configured via the file /etc/modsecurity/crs/crs-setup.conf.

Anomaly Scoring

By default the Core Rule Set is using anomaly scoring mode. This means that individual rules add to a so called anomaly score, which at the end is evaluated. If the anomaly score exceeds a certain threshold, then the traffic is blocked. You can read more about this configuration in crs-setup.conf but the default configuration should be fine for most people.

Setting the paranoia level

The paranoia level is a number from 1 to 4 which determines which rules are active and contribute to the anomaly scoring. The higher the paranoia level, the more rules are activated and hence the more aggressive the Core Rule Set is, offering more protection but potentially also causing more false positives. By default the paranoia level is set to 1. If you work with sensitive data, it is recommended to increase the paranoia level.

The executing paranoia level defines the rules which will be executed but their score will not be added to the anomaly scoring. When HTTP traffic hits rules of the executing paranoia level, this traffic will only be logged but not be blocked. It is a especially useful to prepare for increasing the paranoia level and finding false positives on this higher level, without causing any disruption for your users.

To set the paranoia level to 1 and the executing paranoia level to 2, make sure you have these rules set in crs-setup.conf:

SecAction \
  "id:900000,\
   phase:1,\
   nolog,\
   pass,\
   t:none,\
   setvar:tx.paranoia_level=1"
SecAction \
  "id:900001,\
   phase:1,\
   nolog,\
   pass,\
   t:none,\
   setvar:tx.executing_paranoia_level=2"

Once you have fixed all false positives, you can raise the paranoia level to 2 to increase security.

Defining the allowed HTTP methods

By default the Core Rule Set only allows the GET, HEAD, POST and OPTIONS HTTP methods. For many standard sites this will be enough but if your web applications also use restful APIs or WebDAV, then you will need to add the required methods. Change rule 900200, and add the HTTP methods mentioned in the comments in crs-setup.conf.

SecAction \
 "id:900200,\
  phase:1,\
  nolog,\
  pass,\
  t:none,\
  setvar:'tx.allowed_methods=GET HEAD POST OPTIONS'"

Disallowing old HTTP versions

There is a rule which determines which HTTP versions you allow in HTTP requests. I uncomment it and modify it to only allow HTTP versions 1.1 and 2.0. Legitimate browsers and bots always use one of these modern HTTP versions and older versions usually are a sign of malicious activity.

SecAction \
 "id:900230,\
  phase:1,\
  nolog,\
  pass,\
  t:none,\
  setvar:'tx.allowed_http_versions=HTTP/1.1 HTTP/2 HTTP/2.0'"

Blocking specific countries

Personally I’m not a fan of completely blocking all traffic from a whole country, because you will also block legitimate visitors to your site, but in case you want to this, you can configure this in crs-setup.conf:

SecGeoLookupDB /usr/share/GeoIP/GeoIP.dat
SecAction \
 "id:900600,\
  phase:1,\
  nolog,\
  pass,\
  t:none,\
  setvar:'tx.high_risk_country_codes='"

Add the two-letter country codes you want to block to the last line (before the two quotes), multiple country codes separated by a space.

Make sure you have the package geoip-database installed.

Core Rule Set Exclusion rules for well-known web applications

The Core Rule Set contains some rule exclusions for some well-known web applications like WordPress, Drupal and NextCloud which reduces the number of false positives. I add the following section to crs-setup.conf which will allow me to enable the exclusions in the Apache configuration by setting the WEBAPPID variable in the Apache configuration whenever I need them.

SecRule WEBAPPID '@beginsWith wordpress' 'id:20000,phase:1,nolog,pass,t:none setvar:tx.crs_exclusions_wordpress=1'
SecRule WEBAPPID '@beginsWith drupal' 'id:20001,phase:1,nolog,pass,t:none setvar:tx.crs_exclusions_drupal=1'
SecRule WEBAPPID '@beginsWith dokuwiki' 'id:20002,phase:1,nolog,pass,t:none setvar:tx.crs_exclusions_dokuwiki=1'
SecRule WEBAPPID '@beginsWith nextcloud' 'id:20003,phase:1,nolog,pass,t:none setvar:tx.crs_exclusions_nextcloud=1'
SecRule WEBAPPID '@beginsWith cpanel' 'id:20004,phase:1,nolog,pass,t:none setvar:tx.crs_exclusions_cpanel=1'
SecRule WEBAPPID '@beginsWith xenforo' 'id:20005,phase:1,nolog,pass,t:none setvar:tx.crs_exclusions_xenforo=1'

Adding rules for Log4Shell and Spring4Shell detection

At the end of 2021 a critical vulnerability CVE-2021-44228, named Log4Shell, was detected in Log4j, which allows remote attackers to run code on a server with the vulnerable Log4j version. While the Core Rule Set offered some mitigation of this vulnerability out of the box, this protection was not complete. New improved detection rules against Log4Shell were developed. Because of the severity of this bug and the fact that it’s being exploited in the wild, I strongly recommend adding this protection manually when using ModSecurity version 3.3.2 (or older). Newer, not yet released versions, should have complete protection out of the box.

First modify /etc/apache2/mods-enabled/security2.conf so that it looks like this:

<IfModule security2_module>
        # Default Debian dir for modsecurity's persistent data
        SecDataDir /var/cache/modsecurity

        # Include all the *.conf files in /etc/modsecurity.
        # Keeping your local configuration in that directory
        # will allow for an easy upgrade of THIS file and
        # make your life easier
        IncludeOptional /etc/modsecurity/*.conf

        # Include OWASP ModSecurity CRS rules if installed
        IncludeOptional /usr/share/modsecurity-crs/*.load
        SecRuleUpdateTargetById 932130 "REQUEST_HEADERS"
</IfModule>

Then create the file /etc/modsecurity/99-CVE-2021-44228.conf with this content:

# Generic rule against CVE-2021-44228 (Log4j / Log4Shell)
# See https://coreruleset.org/20211213/crs-and-log4j-log4shell-cve-2021-44228/
SecRule REQUEST_LINE|ARGS|ARGS_NAMES|REQUEST_COOKIES|REQUEST_COOKIES_NAMES|REQUEST_HEADERS|XML://*|XML://@* "@rx (?:\${[^}]{0,4}\${|\${(?:jndi|ctx))" \
    "id:1005,\
    phase:2,\
    block,\
    t:none,t:urlDecodeUni,t:cmdline,\
    log,\
    msg:'Potential Remote Command Execution: Log4j CVE-2021-44228', \
    tag:'application-multi',\
    tag:'language-java',\
    tag:'platform-multi',\
    tag:'attack-rce',\
    tag:'OWASP_CRS',\
    tag:'capec/1000/152/137/6',\
    tag:'PCI/6.5.2',\
    tag:'paranoia-level/1',\
    ver:'OWASP_CRS/3.4.0-dev',\
    severity:'CRITICAL',\
    setvar:'tx.rce_score=+%{tx.critical_anomaly_score}',\
    setvar:'tx.anomaly_score_pl1=+%{tx.critical_anomaly_score}'"

In March 2022 CVE-2022-22963, another remote code execution (RCE) vulnerability was published in the Spring framework was published. The Core Rule Set developed a new rule to protect against this vulnerability which will be included in the next version, but the rule can be added manually if you are running the Core Rule Set version 3.3.2 or older.

To do so, create the file /etc/modsecurity/99-CVE-2022-22963.conf with this content:

# This rule is also triggered by the following exploit(s):
# - https://www.rapid7.com/blog/post/2022/03/30/spring4shell-zero-day-vulnerability-in-spring-framework/
# - https://www.ironcastle.net/possible-new-java-spring-framework-vulnerability-wed-mar-30th/
#
SecRule ARGS|ARGS_NAMES|REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|REQUEST_BODY|REQUEST_HEADERS|XML:/*|XML://@* \
    "@rx (?:class\.module\.classLoader\.resources\.context\.parent\.pipeline|springframework\.context\.support\.FileSystemXmlApplicationContext)" \
    "id:1006,\
    phase:2,\
    block,\
    t:urlDecodeUni,\
    msg:'Remote Command Execution: Malicious class-loading payload',\
    logdata:'Matched Data: %{MATCHED_VAR} found within %{MATCHED_VAR_NAME}',\
    tag:'application-multi',\
    tag:'language-java',\
    tag:'platform-multi',\
    tag:'attack-rce',\
    tag:'OWASP_CRS',\
    tag:'capec/1000/152/248',\
    tag:'PCI/6.5.2',\
    tag:'paranoia-level/2',\
    ver:'OWASP_CRS/3.4.0-dev',\
    severity:'CRITICAL',\
    setvar:'tx.rce_score=+%{tx.critical_anomaly_score}',\
    setvar:'tx.anomaly_score_pl2=+%{tx.critical_anomaly_score}'"

Don’t forget to reload your Apache configuration after adding these rules.

Testing ModSecurity and checking the logs

We can now easily test ModSecurity by doing a request which tries to abuse a cross-site scripting (XSS) vulnerability:

$ curl -I "https://example.org/?search=<script>alert('CRS+Sandbox+Release')</script>"

This should return HTTP response 403 (Forbidden).

Whenever something hits your ModSecurity rules, this will be logged in your Apache error log. The above request has created these messages in the error log:

[Sat Apr 09 22:22:02.716558 2022] [:error] [pid 847584:tid 140613499016960] [client client-ip:49688] [client client-ip] ModSecurity: Warning. detected XSS using libinjection. [file "/usr/share/modsecurity-crs/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "55"] [id "941100"] [msg "XSS Attack Detected via libinjection"] [data "Matched Data: XSS data found within ARGS:search: <script>alert('CRS Sandbox Release')</script>"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.2"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname "example.org"] [uri "/"] [unique_id "YlHq6gKxO9SgyEd0xH9N5gADLgA"]
[Sat Apr 09 22:22:02.716969 2022] [:error] [pid 847584:tid 140613499016960] [client client-ip:49688] [client client-ip] ModSecurity: Warning. Pattern match "(?i)<script[^>]*>[\\\\s\\\\S]*?" at ARGS:search. [file "/usr/share/modsecurity-crs/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "82"] [id "941110"] [msg "XSS Filter - Category 1: Script Tag Vector"] [data "Matched Data: <script> found within ARGS:search: <script>alert('CRS Sandbox Release')</script>"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.2"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname "example.org"] [uri "/"] [unique_id "YlHq6gKxO9SgyEd0xH9N5gADLgA"]
[Sat Apr 09 22:22:02.717249 2022] [:error] [pid 847584:tid 140613499016960] [client client-ip:49688] [client client-ip] ModSecurity: Warning. Pattern match "(?i:(?:<\\\\w[\\\\s\\\\S]*[\\\\s\\\\/]|['\\"](?:[\\\\s\\\\S]*[\\\\s\\\\/])?)(?:on(?:d(?:e(?:vice(?:(?:orienta|mo)tion|proximity|found|light)|livery(?:success|error)|activate)|r(?:ag(?:e(?:n(?:ter|d)|xit)|(?:gestur|leav)e|start|drop|over)|op)|i(?:s(?:c(?:hargingtimechange ..." at ARGS:search. [file "/usr/share/modsecurity-crs/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "199"] [id "941160"] [msg "NoScript XSS InjectionChecker: HTML Injection"] [data "Matched Data: <script found within ARGS:search: <script>alert('CRS Sandbox Release')</script>"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.2"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname "example.org"] [uri "/"] [unique_id "YlHq6gKxO9SgyEd0xH9N5gADLgA"]
[Sat Apr 09 22:22:02.718018 2022] [:error] [pid 847584:tid 140613499016960] [client client-ip:49688] [client client-ip] ModSecurity: Access denied with code 403 (phase 2). Operator GE matched 5 at TX:anomaly_score. [file "/usr/share/modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "93"] [id "949110"] [msg "Inbound Anomaly Score Exceeded (Total Score: 15)"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.2"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "example.org"] [uri "/"] [unique_id "YlHq6gKxO9SgyEd0xH9N5gADLgA"]
[Sat Apr 09 22:22:02.718596 2022] [:error] [pid 847584:tid 140613499016960] [client client-ip:49688] [client client-ip] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/usr/share/modsecurity-crs/rules/RESPONSE-980-CORRELATION.conf"] [line "91"] [id "980130"] [msg "Inbound Anomaly Score Exceeded (Total Inbound Score: 15 - SQLI=0,XSS=15,RFI=0,LFI=0,RCE=0,PHPI=0,HTTP=0,SESS=0): individual paranoia level scores: 15, 0, 0, 0"] [ver "OWASP_CRS/3.3.2"] [tag "event-correlation"] [hostname "example.org"] [uri "/"] [unique_id "YlHq6gKxO9SgyEd0xH9N5gADLgA"]

In the first 3 lines we see that we hit different filters which check for XSS vulnerabilities, more specifically rules 941100, 941110 and 941160 all of them having the tag paranoia-level/1.

Then the fourth line shows that we hit rule 949110 which caused the web server to return the HTTP 403 Forbidden response because the inbound anomaly score, 15, is higher than 5. Then rule 980130 gives us some more information about the scoring: we hit a score of 15 at the paranoia level 1, while rules at the other paranoia levels rules contributed 0 to the total score. We also see the scores for individual types of attack: in this case all 15 points where scored by rules detecting XSS attacks. This is the meaning of the different abbreviations used:

SQLI	SQL injection
XSS	cross-site scripting
RFI	remote file inclusion
LFI	local file inclusion
RCE	remote code execution
PHPI	PHP injection
HTTP	HTTP violation
SESS	session fixation

More detailed logs about the traffic hitting the rules can be found in the file /var/log/apache2/modsec_audit.log.

Fixing false positives

First of all, in order to minimize the amount of false positives, you should set the WEBAPPID variable if you are using one of the known web applications for which the Core Rule Set has a default exclusion set. These web applications are currently WordPress, Drupal, Dokuwiki, Nextcloud, Xenforo and cPanel. You can do so by using the <a href="https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#SecWebAppId">SecWebAppId</a> option in a VirtualHost of Location definition in the Apache configuration. For example if you have a VirtualHost which is used by Nextcloud, set this within the VirtualHost definition:

<Virtualhost nextcloud.example.org>
    ...OTHER OPTIONS HERE...
    <IfModule security2_module>
        SecWebAppId "nextcloud"
    </IfModule>
</VirtualHost>

If you have a WordPress installation in a subdirectory, then add SecWebAppId within Location tags.

<Location /wordpress>
    <IfModule security2_module>
        SecWebAppId "wordpress-mysite"
    </IfModule>
</Location>

If you have multiple WordPress sites, give each of them a unique WEBAPPID which name starts with wordpress. Add a different suffix for every instance so that each one run its in own application namespace in ModSecurity.

If you still encounter false positives, you can completely disable rules by using the configuration directive SecRuleRemoveById. I strongly recommend not disabling rules globally, but limiting its removal to the specific location from which you want them to be removed, for example by putting them with <Location> or <LocationMatch> tags in the Apache configuration. For example:

<LocationMatch ^/wp-admin/(admin-ajax|post)\.php>
    <IfModule security2_module>
        SecRuleRemoveById 941160 941100 941130 932105 932130 932100
    </IfModule>
</LocationMatch>

Pay attention not to disable any of the 949*, 959*, and 980* rules: disabling the 949* and 959* rules would disable all the blocking rules, while disabling the 980* rules would give you less information about what is happening in the logs.

Conclusion

ModSecurity and the Core Rule Set offer an additional security layer for web servers in your defence in depth strategy. I strongly recommend implementing this on your servers because it makes it harder to abuse security vulnerabilities.

Keep an eye on the Core Rule Set blog and Twitter account: sometimes they post new rules for specific new critical vulnerabilities, which can be worthwhile to add to your configuration.

Fixing crackling/popping while playing music in Debian GNU/Linux

Posted on September 10, 2021September 10, 2021 by Frederik

I was experiencing crackling/popping sounds while playing music with Rhythmbox in my Debian GNU/Linux Testing (Bookworm) system. The noises start when starting music playback and stop as soon as I stop the playback.

I came around this bug report for Pipewire but I’m currently still using PulseAudio 15 instead. However it contained a comment which led to the solution: speech-dispatcher is known for causing problems of crackling sounds.

Speech-dispatcher is used for speech synthesis often used by blind or visually impaired people. If you don’t use this functionality, you can either disable speech-dispatcher (speechd) or completely uninstall it:

Disable speech-dispatcher by editing /etc/speech-dispatcher/speechd.conf, removing the comment sign # before this line near the end of the file:

DisableAutoSpawn

Then you will have to log out and log in your desktop or you can kill all speech-dispatcher processes manually.

If you want, you can remove speech-dispatcher completely by running this command:

# apt remove speech-dispatcher

Note that this will remove the gnome meta-package, which in itself is harmless, but might lead to new GNOME components not being installed automatically in the future.

This fixed the constant popping crackling noises while playing music completely for me.

Missing video thumbnails in Nautilus in Debian Bullseye

Posted on June 19, 2021June 19, 2021 by Frederik

I am using Debian Bullseye and already for a long time I noticed that Nautilus failed to generate thumbnails for certain video files, leading to only a generic video icon instead of a thumbnail.

In the kernel log (dmesg), you will see this error:

qtdemux0:sink[70839]: segfault at 0 ip 0000000000000000 sp 00007f724fe61d18 error 14 in totem-video-thumbnailer[5638abe9b000+3000]
Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6.

This is Debian bug #967941: gnome-video-thumbnailer crashes with a segmentation fault when thumbnailing H.264 encoded video files when you have the package libopenblas0-pthread installed.

Available Work-arounds

Remove libopenblas0-pthread

# apt-get remove libopenblas0-pthread

This might not be possible if you need this package for other reasons.

For performance reasons you might also want to install the BLIS BLAS implementation and set it as default. Install the packages libblis3-pthread and libblis64-3-pthread and set them default using the update-alternatives commands from the next work-around.

Switch the default BLAS implementation from OpenBlas to Atlas

If you cannot uninstall libopenblas0-pthread, you can change the default BLAS and LAPACK implementation on your system to a different implementation than OpenBLAS

# update-alternatives --config libblas64.so.3-x86_64-linux-gnu
# update-alternatives --config libblas.so.3-x86_64-linux-gnu
# update-alternatives --config liblapack.so.3-x86_64-linux-gnu

Choose a different implementation than openblas in all 3 cases. Usually Atlas will be installed and available by default. You can also install the packages libblis3-pthread and libblis64-3-pthread and choose the BLIS implementation for the former two and Atlas for the latter.

Disable the sandbox in totem-video-thumbnailer

Totem-video-thumbnailer uses some kind of sandbox limiting how much CPU and memory resources the process can use. This enhances security and prevents the process from using all resources in case something would go wrong. OpenBLAS causes problems in combination with these restrictions. You can disable them in totem-video-thumbnailer by calling it with the -l option. To automatically let GNOME call it with that option, edit the file /usr/share/thumbnailers/totem.thumbnailer and edit the Exec= line so that it looks like this:

Exec=/usr/bin/totem-video-thumbnailer -l -s %s %u %o

If you can, rather use one of two previous work-arounds, because disabling this restrictions could have security implications.

Regenerating the failed thumbnails

Finally after implementing one of the 3 work-arounds, you will need to regenerate the failed thumbnails. GNOME Thumbnail Factory keeps a list of all failed thumbnails, so that it does not retry to generate them over and over again. They are stored in ~/.cache/thumbnails/fail/gnome-thumbnail-factory so you can just delete this directory:

rm -rf ~/.cache/thumbnails/fail/gnome-thumbnail-factory

Now when you browse a directory which contains video files with missing thumbnails with Nautilus, it should generate them automatically.

Using HTTP headers to protect your visitor’s security and privacy

Posted on April 17, 2021April 17, 2021 by Frederik

Recently there has been a lot of controversy over Google starting to use Federated Learning of Cohorts (FLoC) in its Chrome browser. This new technique is used to track users without using third party cookies, but has severe privacy implications because it actually makes fingerprinting users easier and can reveal your interests to websites.

To prevent tracking by FLoC and other tracking techniques, there is only one good solution: stop using Google Chrome. The best privacy friendly browser is Firefox, especially if you set it to strict enhanced tracking protection. If you really need to use Chrome, then at least install one of the open source extensions which disable FLoC and Privacy Badger for other tracking protection.

As a website owner, you can also do something to protect your users. You can opt out your website to be included in cohort computation by sending the header Permissions-Policy: interest-cohort=()

This can be easily done for all your websites by modifying your Apache configuration. While at it, you should set some other security and privacy related headers, notably:

X-Frame-Options "SAMEORIGIN": this makes sure that browsers will not allow your website to be included in an iframe on another domain.
X-Content-Type-Options "nosniff": This will prevent the browser from trying to automatically detect the file type of a downloaded file instead of using the MIME type sent by the server. This can mitigate attacks where a hacker manages to upload a malicious file by giving it a filename which makes it look like a harmless file type which is then served to your visitors.
Referrer-Policy "no-referrer-when-downgrade": when a visitor clicks on a link, the browser will only send the referrer when it’s not going from a HTTPS to a HTTP connection. This is fine if your URLs don’t contain any private information. If they do, then consider using strict-origin-when-cross-origin, so that only your domain name instead of the complete URL is sent as referrer if people click on a link leading to an external website, or even same-origin, which prevents any referrer being sent to external sites. You should probably do this for an internal website, web application or wiki, webmail, etc. More information about Referrer-Policy

To set these in Apache in Debian, create a file /etc/apache2/conf-available/security-headers.conf with these contents:

<IfModule mod_headers.c>
   Header always set X-Frame-Options "SAMEORIGIN"
   Header always set X-Content-Type-Options "nosniff"
   Header always set Referrer-Policy "no-referrer-when-downgrade"
   Header always set Permissions-Policy: interest-cohort=()
</IfModule>

Then make sure the mod_headers module is loaded and this file is enabled by running these commands:

# a2enmod headers
# a2enconf security-headers
# systemctl reload apache2

Another important header to set in your SSL virtualhosts is the HSTS header: it ensures that the browser will automatically use HTTPS every time when connecting to the website in the future. Place this in your SSL enabled virtualhost:

<IfModule mod_headers.c>
   Header always set Strict-Transport-Security "max-age=63072000"
</IfModule>

Then you should also add this to your non-SSL virtualhost to redirect all visitors using HTTP to HTTPS:

<IfModule mod_rewrite.c>
   RewriteEngine on
   RewriteCond %{HTTPS} !=on
   RewriteRule .* https://%{SERVER_NAME}%{REQUEST_URI} [R=301,L]
</IfModule>

Of course make sure mod_rewrite is enabled if that’s not yet the case:

# a2enmod rewrite
# systemctl reload apache2

You can check your server configuration on securityheaders.com. There you can also find more information about the headers Cross-Origin-Embedder-Policy, Cross-Origin-Opener-Policy and Cross-Origin-Resource-Policy, some other security related headers. Because they require more changes to your website to implement correctly, I’m not discussing them here.

More information

How to set the header to opt out from FLoC with different web servers and in different web applications: https://paramdeo.com/blog/opting-your-website-out-of-googles-floc-network
How to fight back against FLoC: https://plausible.io/blog/google-floc
Am I FLoCed?: https://amifloced.org/

LineageOS 18.1 (Android 11) on OnePlus 3/3T status

Posted on April 3, 2021October 2, 2021 by Frederik

Table of contents
Useful links
Questions, problems and solutions
Is it possible to upgrade from LineageOS 17 to 18.1?
Which gapps should I use?
I cannot find Netflix in the Google Play Store
My bank app refuses to start because the phone is rooted
Known bugs in LineageOS 18.1 on OnePlus 3/3T
The phone crashes from time to time
No way to disable a SIM card when using multiple SIMs

Now that LineageOS 18.1 is officially out, it’s time to take a look at how it runs on Oneplus 3/3T phones, just like I did for LineageOS 17.1 on Oneplus 3/3T. At this time I am not yet running LineageOS 18.1 myself so the information here is based on what I read in the LineageOS 18.1 on Oneplus 3/3T thread on XDA-developers forum. Because of this, information here can be incomplete and inaccurate. If you encounter any problems, search in the thread on XDA Developers and post a message there if necessary. This page will be updated as more additional information becomes known.

Useful links

Download LineageOS builds for Oneplus 3/3T
LineageOS changelog
LineageOS 18.1 on Oneplus 3/3T thread on XDA-developers forum
I strongly recommend skimming through the last 5 pages of this thread and using the search function before asking a question there, because people will get annoyed and give you a blunt answer if you ask about a topic discussed before.

Questions, problems and solutions

Is it possible to upgrade from LineageOS 17 to 18.1?

Even though this is usually discouraged on XDA and people will often refuse to give you any support if you did not do a clean installation, it is actually possible. The procedure can lead to problems, so make sure you have proper back-ups and know how to use TWRP.

First make a complete back-up of your device using TWRP. I strongly recommend to back up everything, both the system partition files as the system image. While this is both the same, sometimes a restored system only boots successfully with one of them. Definitely also back up data and boot.

After a successful complete backup, connect your phone to your computer, and via the android notification that it’s charging via USB, select that you want to transfer files. Now open your device’s internal storage disk on your computer, and copy the complete backup in TWRP/BACKUPS/ to your computer. I recommend also making a complete back-up of all apps, with an app like OAndBackupX and then copying this back-up to your PC. While connected to your PC, also make a back-up of your photos in the DCIM directory, because these are not included in backups made with TWRP and OAndBackupX.

Download the following files, either using a browser in Android, either using your computer and in that case copy them to your phone’s internal storage.

the latest TWRP
the latest LineageOS nightly zip
the latest MindTheGapps (ARM64 version) for Android 11
the latest Magisk zip
the latest Magisk uninstaller zip

Now go to TWRP and install Magisk Unnstaller zip. Then choose Wipe – Advanced wipe, and wipe system, boot, cache and dalvik (don’t wipe data or storage: you will loose all data on your phone if you do so). Then go to Install – Install image, and choose the twrp image. Choose recovery and let it flash TWRP. Then reboot your phone back into the new recovery. Then flash LineageOS 18.1 via Install, then wipe cache and dalvik cache, and reboot. If all goes fine, you can boot back in TWRP and flash Magisk.

If all this fails, restore the backups you made.

Which gapps should I use?

You can use MindTheGapps or OpenGapps for LineageOS 18.1. If you choose OpenGapps, use OpenGapps nano. Never use any of the more complete packages, they will cause problems. Install the missing Google applications afterwards from the Play Store.

I cannot find Netflix in the Google Play Store

When you go the Settings in the Play Store app, under Play Protect certification you will see that your device is not certified.

To solve this, flash the latest Magisk zip in TWRP. Then back in Android go to Settings – Apps & Notifications and there go to Play Store App-Info. Go to Storage and there clear the cache and the storage. Now restart your phone. If you go to the Play Store settings, you will see that the message that it is not certified is gone, and you will be able to install Netflix. After clearing the storage, verify that the Play Store settings are OK for you, because these will have been reset.

Android Gadget Hacks: Fix Play Store Uncertified Errors When You Forget to Flash Magisk

My bank app refuses to start because the phone is rooted

Flash Magisk in TWRP. Then Start Magisk Manager and tap on SafetyNet Check. It should return Success and a green check mark for both CtsProfile and basicIntegrity. Then in Magisk’s menu go to Settings and make sure Magisk Hide is enabled. Then in the menu go to Magisk Hide and enable it for your bank app and for Google Play Services. If it does not work yet, try to remove data and cache of your bank app and reboot. Then try to reconfigure your account.

Known bugs in LineageOS 18.1 on OnePlus 3/3T

The phone crashes from time to time

Some users are reporting stability problems: they experience random reboots in LineageOS because of a kernel panic, at a frequency of about once a week. Logs posted on XDA point to a kernel oops in ffs_data_clear, which is a function in the USB gadget code.

https://forum.xda-developers.com/t/rom-official-nightly-lineageos-18-1-for-oneplus-3-3t.4230665/post-85294671

No way to disable a SIM card when using multiple SIMs

https://forum.xda-developers.com/oneplus-3/oneplus-3–3t-cross-device-development/rom-lineageos-17-0-oneplus-3-3t-t3990061/post82770687#post82770687
https://forum.xda-developers.com/oneplus-3/oneplus-3–3t-cross-device-development/rom-lineageos-17-0-oneplus-3-3t-t3990061/post82771459#post82771459

In LineageOS 16.0 it’s possible to disable one of the SIM cards if you have two SIMs installed. In LineageOS 17.1 and 18.1, the only way to disable a SIM is to physically remove the card from the slot.

This problem is not OnePlus 3/3T specific: it is like this in all LineageOS builds for all devices, and is a deliberate choice by the developers, because it was too hard to implement this for Qualcomm devices (“it’s not a bug, it’s a feature”).

Using Fangfrisch to improve ClamAV malware detection in e-mail

Posted on March 28, 2021April 26, 2023 by Frederik

If you have an e-mail server, maybe you already are using the open source tools Amavis and ClamAV to detect malicious e-mails. ClamAV’s default virus signatures however, while being useful, still only detect a limited set of malware. Fortunately there exist third-party unofficial signatures which you can use to drastically improve the detection rate so that it becomes similar to the detection rate of commercial anti-virus software. These third-party signatures are a must-have on any mail server using ClamAV.

ClamAV can also be very useful to scan the webroot directories on web servers in order to detect malicious PHP scripts which may be installed on your server after an intrusion in a web application. Several of the third-part ClamAV definitions proposed here, specialize in this kind of malware.

Fangfrisch is a utility which automates downloading third-party ClamAV virus definitions. It has several free repositories containing ClamAV signatures configured by default, but you can also easily add other ones.

Third-party ClamAV virus definitions

SaneSecurity

SaneSecurity is a set of signatures focusing on so-called 0-day and 0-hour malware, which means that it includes hashes of new malicious files being sent by e-mail. Furthermore it contains signatures for malicious URLs, common spam and phishing messages and generic signatures which detect some commonly types of techniques used in malware, such as exe files with a double extenstion (for example pdf.exe), exe files hidden in ISO files and often abused functions in MS Office macros. Sanesecurity also distributes signatures from other sources, such as phishing URLs from phishtank.com. A complete list of available signature databases can be found on their website.

SaneSecurity signatures are free to use, however a donation is appreciated.

SecuriteInfo

The French company SecuriteInfo claims that their ClamAV definitions add 4.000.000 signatures for malware and spam not detected by the official ClamAV signatures. There is a free feed of the signatures available, however it only contains signatures older than 30 days. For up-to-date 0-day malware detection, you will need one of the paid plans.

In the MalwareBazaar database you can see that SecuriteInfo is often the only ClamAV definitions database which detects malicious Windows binaries.

Malwarepatrol

MalwarePatrol is a commercial threat intelligence company which offers free and paid feeds, including ClamAV virus signatures which contain URL’s pointing to malware files on the world wide web. The free feed is only updated every 72 hours, while the paid feed is updated every 4 hours. If you need an invoice or want to use the feeds for protecting customers of your company, you will have to use the commercial feeds.

Quickly after integrating the free feed in my Amavis installation, I noticed it was wrongly blocking legitimate mails with links to arxiv.org/pdf, which is used a lot in academic environments. So be careful when you integrate this feed. For this reason you should not automatically block all messages marked by Malwarepatrol. With Amavis you can give them them a positive spam score. In combination with the score from the Bayes spam filter, legitimate mails will not be blocked.

MalwareExpert

MalwareExpert offers a commercial, paid feed of signatures which detect malicious PHP files, meant for scanning your web servers.

URLHaus

URLHaus is a project from abuse.ch collecting URLs of sites distributing malware. They offer a ClamAV signature database of all these malicious URLs so that you can block all e-mail containing links to sites distributing malware. This database can be used for free, both for commercial and non-commercial purposes.

Clam-punch

Clam-punch is described as a “highly curated set of signatures for ClamAV”. It seems to focus mainly on malicious macros in MS Office documents. They do not seem to be updated regularly any more, however as the signatures appear to be rather generic, they can probably still be useful.

TwinWave Security Twinclams

Twinclams is a Github repository by TwinWave Security and contains signatures for malicious MS Office documents. The author of twinclams appears to be a contributor of Clam-punch. This ClamAV virus database is getting updated daily and appears to be highly effective in detecting newly found Office documents with malicious macros. Recent malicious Office document in the MalwareBazaar database are almost always detected by the Twinclams definitions, sometimes even detecting files which are not detected yet by commercial anti-virus software.

R-FX Networks Linux Malware Detect

Part of its Linux Malware Detect (LMD) tool, R-FX Networks offers a set of ClamAV signatures specializing in detecting Linux specific malware, including malicious PHP scripts, trojan horses such as malicious IRC bots, worms, etc.

InterServer

Another set of ClamAV signatures focused on malicious PHP scripts is maintained by the hosting provider InterServer. I highly recommend this feed if you want to scan your web servers.

ditekshen

The ClamAV signatures by security reseacher ditekshen add detection for various Windows, MacOS and Linux trojan horses en ransomware.

Installing Fangfrisch

Unfortunately Debian does not have a package for Fangfrisch, so you need to install it manually. In contrast to the documentation, I prefer to install fangfrisch in /opt though, using /var/lib/fangfrisch only for the database, and saving the signatures in /var/lib/fangfrisch/signatures, so that we can check them later on before letting ClamAV use them.

# mkdir -m 0770 -p /var/lib/fangfrisch/signatures
# chgrp -R clamav /var/lib/fangfrisch
# mkdir /opt/fangfrisch 
# cd /opt/fangfrisch
# apt-get install python3-venv python3-pip
# python3 -m venv venv
# source venv/bin/activate
# pip3 install fangfrisch

Configuring Fangfrisch

Create the file /etc/fangfrisch/fangfrisch.conf with these contents:

[DEFAULT]
db_url = sqlite:////var/lib/fangfrisch/db.sqlite

# The following settings are optional. Other sections inherit
# values from DEFAULT and may also overwrite values.

local_directory = /var/lib/fangfrisch/signatures
max_size = 5MB
on_update_exec = /usr/local/bin/setup-clamav-sigs
on_update_timeout = 42
log_level = info

[malwarepatrol]
enabled = no
# Replace with your personal Malwarepatrol receipt
# receipt = XXXXXXXXX
# change product id if necessary.
# 32 = free guard, 33 = Basic Defense yearly, 34 = Basic Defense monthly, 37 = Basic Defense EDU/Contributor
# product = 32

# This is untested
[malwareexpert]
enabled = no
max_size = 20M
prefix = https://signatures.malware.expert
interval = 1d
# serial_key = xxxxxxx
url_malware.expert_fp = ${prefix}/${serial_key}/malware.expert.fp
url_malware.expert_hdb = ${prefix}/${serial_key}/malware.expert.hdb
url_malware.expert_ldb = ${prefix}/${serial_key}/malware.expert.ldb
url_malware.expert.ndb = ${prefix}/${serial_key}/malware.expert.ndb

[sanesecurity]
prefix = https://ftp.swin.edu.au/sanesecurity/
max_size = 10M
enabled = yes
interval = 1h
url_malwareexpert_fp = disabled
url_malwareexpert_hdb = disabled
url_malwareexpert_ldb = disabled
url_malwareexpert_ndb = disabled

[securiteinfo]
enabled = no
# uncomment the next line if you want to use the securiteinfoold.hdb database with old signatures
# max_size = 500MB
# Replace with your personal SecuriteInfo customer ID
# customer_id = abcdef123456
# Remove the exclamation mark before the next databases if you have the Professional subscription
!url_0hour = ${prefix}securiteinfo0hour.hdb
!url_securiteinfo_mdb = ${prefix}securiteinfo.mdb
# The next databases are disabled by default in fangfrisch because they are prone to false positives, but we reduce most of them to a spam score in Amavis
url_old = ${prefix}securiteinfoold.hdb
url_spam_marketing = ${prefix}spam_marketing.ndb

[urlhaus]
enabled = yes
max_size = 2MB

[twinwave]
enabled = yes
max_size = 2M
integrity_check = disabled
interval = 1h
prefix = https://raw.githubusercontent.com/twinwave-security/twinclams/master/
url_twinclams = ${prefix}twinclams.ldb
url_twinwave_ign2 = ${prefix}twinwave.ign2

[clampunch]
enabled = yes
max_size = 2M
integrity_check = disabled
interval = 24h
prefix = https://raw.githubusercontent.com/wmetcalf/clam-punch/master/
url_miscreantpunch099low = ${prefix}MiscreantPunch099-Low.ldb
url_exexor99 = ${prefix}exexor99.ldb
url_miscreantpuchhdb = ${prefix}miscreantpunch.hdb

[rfxn]
enabled = yes
interval= 4h
integrity_check = disabled
prefix = https://www.rfxn.com/downloads/
url_rfxn_ndb = ${prefix}rfxn.ndb
url_rfxn_hdb = ${prefix}rfxn.hdb
url_rfxn_yara = ${prefix}rfxn.yara

[interserver]
enabled = yes
interval = 1d
integrity_check = disabled
prefix = https://sigs.interserver.net/
url_interserver_sha256 = ${prefix}interserver256.hdb
url_interserver_topline = ${prefix}interservertopline.db
url_interserver_shell = ${prefix}shell.ldb
url_interserver_whitelist = ${prefix}whitelist.fp

[ditekshen]
enabled = yes
interval = 1d
integrity_check = disabled
prefix = https://raw.githubusercontent.com/ditekshen/detection/master/clamav/
url_ditekshen_ldb = ${prefix}clamav.ldb
filename_ditekshen_ldb = ditekshen.ldb

Don’t forget to customize the entries if you have any paid subscription. Set some safe file permissions on this file, especially if it contains tokens of commercial subscriptions:

# chown root:clamav /etc/fangfrisch/fangfrisch.conf
# chmod 640 /etc/fangfrisch/fangfrisch.conf

Checking the new signatures before letting ClamAV use them

In the configuration file above, I call the script setup-clamav-sigs whenever signatures were updated. This script will check whether the downloaded files are really different than the signatures in /var/lib/clamav, and if they are, check them to ensure they don’t contain errors preventing ClamAV to load them. If ClamAV can successfully load them, it will copy them to /var/lib/clamav and restart clamdscan if it’s loaded. Download the script from the GitLab repository, place it in /usr/local/bin and make sure it’s executable.

Initalizing Fangfrisch

When you have set up Fangfrisch, you have to initialize it by executing

sudo -u clamav -- /opt/fangfrisch/venv/bin/fangfrisch --conf /etc/fangfrisch.conf initdb

Configuring Clamav

SecuriteInfo recommends setting these settings in /etc/clamav/clamd.conf in order to get the best detection while still avoiding too many false positives:

DetectPUA yes
ExcludePUA PUA.Win.Packer
ExcludePUA PUA.Win.Trojan.Packed
ExcludePUA PUA.Win.Trojan.Molebox
ExcludePUA PUA.Win.Packer.Upx
ExcludePUA PUA.Doc.Packed

Configuring Amavis

I assume you already have a working Amavis instance. Sanesecurity gives some recommendations for Amavis for the best results. Add this to the configuration, for example In Debian you can create the file /etc/amavis/conf.d/50-clamav:

@keep_decoded_original_maps = (new_RE(
  qr'^MAIL$',   # retain full original message for virus checking (can be slow)
  qr'^MAIL-UNDECIPHERABLE$', # recheck full mail if it contains undecipherables
  qr'^(ASCII(?! cpio)|text|uuencoded|xxencoded|binhex)'i,
# qr'^Zip archive data',     # don't trust Archive::Zip
));

@virus_name_to_spam_score_maps =
   (new_RE(  # the order matters!
     [ qr'^TwinWave\.'                                      => undef ],# keep as infected
     [ qr'^MiscreantPunch\.'                                => undef ],# keep as infected
     [ qr'^Structured\.(SSN|CreditCardNumber)\b'            => 2.0 ],
     [ qr'^(Heuristics\.)?Phishing\.'                       => 2.0 ],
     [ qr'^(Email|HTML)\.Phishing\.(?!.*Sanesecurity)'      => 10.0 ],
     [ qr'^Sanesecurity\.(Malware|Badmacro|Foxhole|Rogue|Trojan)\.' => undef ],# keep as infected
     [ qr'^Sanesecurity\.Phishing\.'                        => 6.0 ],
     [ qr'^Sanesecurity\.Blurl\.'                           => 4.0 ],
     [ qr'^Sanesecurity\.Jurlbl\.'                          => 2.0 ],
     [ qr'^Sanesecurity\.Spam\.'                            => 2.0 ],
     [ qr'^Sanesecurity\.SpamL\.'                           => 2.0 ],
     [ qr'^Sanesecurity\.Junk\.'                            => 4.0 ],
     [ qr'^Sanesecurity\.Scam4\.'                           => 2.0 ],
     [ qr'^Sanesecurity\.'                                  => 0.1 ],
     [ qr'^Sanesecurity.TestSig_'                           => 0   ],
     [ qr'^Email\.Spam\.Bounce(\.[^., ]*)*\.Sanesecurity\.' => 0   ],
     [ qr'^BofhlandMW\.'                                    => undef ],# keep as infected
     [ qr'^Bofhland\.Malware\.'                             => undef ],# keep as infected
     [ qr'^Bofhland\.'                                      => 2.0 ],
     [ qr'^winnow.malware\.'                                => undef ],# keep as infected
     [ qr'^winnow\_'                                        => 2.0 ],
     [ qr'^PhishTank\.Phishing\.'                           => 6.0 ],
     [ qr'^Porcupine\.Malware\.'                            => undef ],# keep as infected
     [ qr'^Porcupine\.'                                     => 2.0 ],
     [ qr'^Email\.Spammail\b'                               => 2.0 ],
     [ qr'^Safebrowsing\.'                                  => 2.0 ],
     [ qr'^winnow\.(phish|spam)\.'                          => 2.0 ],
     [ qr'^SecuriteInfo.com\.Phish'                         => 6.0 ],
     [ qr'^SecuriteInfo.com\.Spam'                          => 2.0 ],
     [ qr'^MBL_'                                            => 4.0 ],
   ));

These settings ensure that not only the different parts and attachments will be separately scanned by ClamAV, but also the mail as a whole. Then we will reduce some ClamAV virus detections from infected to a spam score in Spamassassin. I do this on rules which could be more likely causing false positives. You can adapt the score to your own situation.

Automatically running Fangfrisch using a systemd timer

Create the file /etc/systemd/system/fangfrisch.service:

[Unit]
Description=Download unofficial clamav virus definition files
ConditionPathExists=/var/lib/fangfrisch/db.sqlite

[Service]
Type=oneshot
User=clamav
WorkingDirectory=/var/lib/fangfrisch
ExecStart=/opt/fangfrisch/venv/bin/fangfrisch --conf /etc/fangfrisch/fangfrisch.conf refresh

[Install]
WantedBy=multi-user.target

Then create the file /etc/systemd/system/fangfrisch.timer:

[Unit]
Description=Download unofficial clamav virus definition files
Requires=fangfrisch.service

[Timer]
Unit=fangfrisch.service
Persistent=true
OnUnitActiveSec=10min
RandomizedDelaySec=30

[Install]
WantedBy=timers.target

Run this command to activate the timer:

# systemctl enable --now fangfrisch.timer

Ignoring virus definitions

It is possible that you hit false positives with certain definitions. If you want to completely disable a specific virus definition, you can add its name to a text file with the ign2 extension in /var/lib/clamav/. For example for me the Sanesecurity.Badmacro.Doc.hypers caused a false positive for me, so I created the file /var/lib/clamav/local_whitelist.ign2 with content:

Sanesecurity.Badmacro.Doc.hypers

To reload the database after changing the database, run

# sudo -u clamav clamdscan --reload

Conclusion

With free third-party databases (Sanesecurity, URLHaus, Clam-punch, Twinclams, R-FX MLD, Interserver and ditekShen) it is possible to drastically improve the detection rate of ClamAV so that it becomes an excellent virus scanner for e-mail and web servers at least if you use it In combination with Amavis’ configuration to block malicious file types (such as exe, com, vbs, dll, pif, etc.) and a well configured and trained Spamassassin. If you want the best protection, add a subscription to the SecuriteInfo feeds.

Setting up Linux on a desktop or laptop system, part 1: choosing a Linux distribution and a desktop environment

Posted on August 31, 2020October 13, 2020 by Frederik

Which Linux distribution?

The most widely used distribution is Ubuntu. Other popular desktop distributions are Linux Mint, Fedora, OpenSUSE Tumbleweed and Manjaro. Personally I prefer to use Debian, not only on servers but also on desktop systems, but this distribution does require more manual work to set up on a desktop system, so it is not the easiest choice if you are new to Linux. In that case I would recommend some of the Ubuntu variants or Linux Mint.

Choose a recent version of your Linux distribution of choice. If you use Ubuntu, install the latest LTS version (20.04 Focal Fossa at the time of writing) and if you encounter problems related to hardware support, consider using the latest non-LTS version (20.10 Groovy Gorilla will be available end of October 2020. You can download daily builds if you want to test this version in development). I recommend Debian users to install the testing version on recent hardware.

Which desktop environment?

You will have to choose which desktop environment you want to use on your Linux system. Depending on your distribution, you make this choice before downloading the ISO, or during the installation.

The most widely used desktop environments are GNOME, KDE Plasma, Cinnamon and XFCE. The first three desktop environments get updates and improvements more often than XFCE and are more complete: for this reason I would recommend one of these. XFCE on the other hand is interesting as a lightweight desktop for older hardware.

KDE is very customizable, but the many configuration options can be overwhelming at times. GNOME is less customizable and tries to deliver a user-friendly modern desktop out of the box. If you want a simple clean desktop with a more traditional, Windows- or macOS-like desktop, then you can consider Cinnamon.

In the next table you will find links to the different editions of popular distributions. Some distributions have a default or preferred desktop, one in which they invest most work. I have set these in bold.

	GNOME	KDE Plasma	Cinnamon	XFCE
Ubuntu	Ubuntu Desktop	Kubuntu	Ubuntu Cinnamon Remix	Xubuntu
Linux Mint	no installer available	no installer available	Linux Mint Cinnamon	Linux Mint XFCE
Fedora	Fedora Workstation	Fedora KDE Plasma	Fedora Cinnamon	Fedora XFCE
OpenSUSE Tumbleweed	You choose your desktop during the installation: OpenSUSE Tumbleweed installation ISOs. Cinnamon can only be added after the installation.
Manjaro	Manjaro GNOME	Manjaro KDE Plasma	Manjaro Cinnamon	Manjaro XFCE
Debian	You choose your desktop during the installation: Debian Testing installer

If you like KDE Plasma, you can also consider installing the KDE Neon distribution. It is an Ubuntu LTS with the latest KDE packages installed. This way it provides a more up to date and more standard KDE experience than Kubuntu. However it misses a lot of standard non-KDE application, which you will have to install yourself afterwards.

If you want to get a feeling of these distributions and desktops, you can test them out in a web browser on the website distrotest.net. All in all, I would recommend not spending too much time on choosing your distribution and desktop. Pick one of the beginner-friendly distributions and try it. If you really don’t like it, it is easy enough to install another distribution.

The truth about decentralized contact tracing apps like Immuni

Posted on July 21, 2020October 19, 2020 by Frederik

Lees dit artikel in het Nederlands: De waarheid over gedecentraliseerde contact tracing apps zoals Coronalert

Too long to read? Skip to the conclusion.

Table of contents
DP-3T
Apple/Google Exposure Notifications API
Frequently Asked Questions – Debunking some myths
Will these apps violate my privacy? Will the authorities know who I meet and where I am, what I do?
How can I be sure that the app really works as promised and really does not collect and send private information?
Why should I trust Google and Apple, who have a bad track record in privacy?
Why does this app require Location setting to be enabled on my Android phone if no location information is collected?
Will this app give me a warning every time an infected person passes nearby, resulting in many false positives?
Are these apps useful if not everyone or at least a large part of the population downloads them?
Will this drain the battery of my phone?
Do I need to install another app when I go abroad?
What do experts say about these apps?
Conclusion
Further information
History of this article

Those that know me a bit, know that I think that privacy is very important. For this reason when they started talking about contact tracing apps for COVID-19 a few months ago, I thought this was an extremely bad idea. Having an app constantly tracking where you are and who you meet, is only something which you think is possible in undemocratic nations and dictatorial regimes. Something you expect in North Korea, China but not in European countries, where our privacy is supposed to be protected by the GDPR. And then what about the reliability of these apps? Bluetooth was never made for this. It would result in many false positives and negatives. No way I would ever install such an app.

That was at least my opinion a couple of months ago. Now my opinion about this matter has completely changed. Reading about decentralized solutions based on the Google and Apple Exposure Notification (GAEN) API and DP-3T, has completely changed my mind. I use the Italian contact tracing app Immuni and I am willing to use a similar decentralized app from any country where I am staying.

DP-3T

Decentralized Privacy-Preserving Proximity Tracing or DP-3T is an open protocol developed by different universities amongst others ETH and EPFL from Switzerland, KU Leuven from Belgium, Tu Delft from the Netherlands.

This comic shows a simple explanation of how it works.

Technically speaking, every day a new random seed (based on the seed of the day before) is created on every user’s phone, and this daily seed is saved on the phone for 14 days. From this seed are derived ephemeral identifiers (EphIDs). These EphIDs change several times an hour.

EphIDs are exchanged with other users of the tracking app by Bluetooth Low Energy (BLE) and every phone saves the EphIDs he received locally on the phone, together with the date and the attenuation of the signal, which can be used to estimate the distance.

When a user is tested positive for COVID-19, this user can, with the help of authorized health personal, upload the seed of the first day he was contagious, to a central server. All previous daily seeds are deleted from the infected user’s phone, and a completely new random daily seed is created, so that he does not become trackable in the future.

All other app users regularly download a list of all daily seeds of all contagious users from the central server and they can derive all EphIDs from them. The app compares all these EphIDs with the list of stored EphIDs which they met recently. Based on the amount of matching EphIDs the app can calculate how long the two have seen each other and based on the stored attenuation it can estimate the distance. If they were longer than a certain time within a certain distance, a warning will be given to the user that they were exposed to a contagious user, with instructions on what to do.

More details can be found in the DP-3T white paper.

Apple/Google Exposure Notifications API

The Google/Apple Exposure Notifications API (sometimes abbreviated as GAEN) is an API created from a joint effort by Google and Apple and enables the creation of decentralized contact tracing apps. This API, which is based on the principles of the DP-3T protocol described above, can only be used by apps approved by Google and Apple (only one per country, and created by official healthcare instances). Only decentralized contact tracing apps which do not collect any location information can get approved.

Apple by default does not allow background apps to use Bluetooth, except for approved contact tracing apps using this API. This means that on Apple iPhone, this API is the only way to create a reliable contact tracing app. Apps which don’t use this API, such as the StopCovid France app, have to apply work-arounds to keep the app waking up in the background, making them potentially less reliable and draining the battery more.

The DP-3T framework has since been modified to make use of the Exposure Notifications API.

Some of the apps currently available using the Google/Apple Exposure Notifications API are SwissCovid (Switzerland), Immuni (Italy), Corona-Warn-App (Germany).

The source code of the implementation of the framework for Android and iOS of the API were published the second half of July 2020.

Frequently Asked Questions – Debunking some myths

These apps appear to be the subject of deliberate fake news campaigns or at least emotional reactions resulting from a lack of understanding of how they work. Here I will try to address some questions.

Will these apps violate my privacy? Will the authorities know who I meet and where I am, what I do?

The applications based on the Google Apple Exposure Notification API do not know any personal information about the users: they don’t know your name, your phone number, where you live, or any other personal data. They also do not collect location data, so they don’t know where you are.

The only thing these apps do, is exchanging anonymous codes with other people in your neighbourhood. These codes change multiple times a day, making it impossible to keep tracking you.

The exchanged codes are only stored on your own phone and not in a central database. So there is no way for the authorities to know how many people and who you met.

Contact tracing apps usually apply even more extensive kinds of measures to protect security and privacy: for example dummy uploads are done in order to prevent network traffic analysis revealing a positive test, CA or certificate pinning to prevent MITM attacks, etc…

This is not a mass surveillance tool or Big Brother, as some try to let you believe.

How can I be sure that the app really works as promised and really does not collect and send private information?

These apps are usually open source, which means you can check the code to see how they work and what they do exactly. Even if you personally don’t have the knowledge to check the code, rest assured that there are enough experts taking a look at this, and will shout loudly when something is wrong. It has to be said: the only ones shouting loudly about these apps are politicians and activists who clearly have never looked at the source code, nor at the documentation. For example code reviews of Coronalert and Immuni have found these apps do live up to their privacy claims.

Here are some links to the source code of the different apps and their documentation:

Immuni (Italy) – Documentation
SwissCovid (Switzerland) – DP-3T Documentation
Corona-Warn-App (Germany) – Documentation
CoronaMelder (The Netherlands) – Documentation
Coronalert (Belgium) – Documentation

On the issue tracker of these apps you can report problems and ask questions.

Also the source code of the Exposure Notifications framework, used by these apps, is available:

Why should I trust Google and Apple, who have a bad track record in privacy?

Actually Google and Apple don’t even need this API to track you. If you are running a phone running iOS or Android with Google Play Services, you actually already have much larger privacy problems than these decentralized, open source contact tracing apps. The same if you are using any of Facebook, Twitter, Instagram, TikTok, NetFlix, Spotify, FaceApp, Tinder. These do know your name, your location, your interests, your friends, and this without this API. Decentralized contact tracing apps, do know much less than any of these apps. This picture compares the different permissions SwissCovid, Facebook and Whatsapp can request.

That being said, there is now a way to run these contact tracing apps without using any Google services on your phone. The microg project now includes its own completely open source implementation of the Exposure Notification API. It can be installed on an Android distribution like LineageOS. It is confirmed that SwissCovid and Immuni work with microg’s implementatoin of the API, probably other apps do too. This way you can run these applications without having to rely on any of Google’s or Apple’s proprietary binaries.

Why does this app require Location setting to be enabled on my Android phone if no location information is collected?

To scan for nearby Bluetooth devices on Android the Location setting needs to be switched on because Bluetooth scanning can in theory be used to determine your location. For example this is used by navigation apps to determine your location in underground tunnels. In reality, apps making use of the GAEN API are not allowed to request your location. It can be verified in the source code of the app that at no time it determines your location. In Settings > Location > App permission you can still disable location access to apps. In Android 11, which came out in September 2020, it is noe required any more to have Location enabled on your device in order to use contact tracing apps based on the Exposure Notification API.

Will this app give me a warning every time an infected person passes nearby, resulting in many false positives?

The apps will only give warnings when certain conditions, usually defined by the government based on epidemiological data, are satisfied. For example, the Italian Immuni app will give a warning when someone is standing within a distance for 2 metres for at least 15 minutes. The SwissCovid app requires a 15 minute contact within 1,5 metre distance. Coronalert will show you also low-risk exposures in the app, however only in case of a high-risk exposure (at least 15 minutes within 1,5 – 2 m of distance) will result in a red screen and an explicit exposure notification. Only in this case a test and quarantine is recommended.

The distance is estimated from the attenuation of the signal. Unfortunately, attenuation will depend on lots of parameters, such as the the phone model being used and the direction in which it is hold, etc. Google adds a per device correction value to the attenuation so that values should be similar between different devices. The thresholds being used are based on experiments in different environments and can be modified in the future in order to lower false positives and negatives. Immuni for example uses an attenuation threshold of 73 dBm.

So no, contact tracing apps are not going to give you a warning when anyone who shortly passed nearby, is tested positive. Only when reasonable thresholds are exceeded, you will get a warning. Nevertheless, false positive and false negatives are possible. Authorities also realize this, and view the tracing app as a support tool for tracing, but not as a complete replacement for manual contact tracing. A contact tracing app also does not replace a diagnostic test.

Are these apps useful if not everyone or at least a large part of the population downloads them?

Contact tracing apps are certainly useful, even if only a part of the population uses them.

A highly quoted article from Oxford University states that if 60% of the population installs the contact tracing app, this can completely stop the epidemic. However what is often not quoted is the next part of the phrase: “even with lower numbers of app users, we still estimate a reduction in the number of coronavirus cases and deaths.” They estimate that that one infection will be averted for every one to two users.

So even much lower numbers than 60% are useful to help flattening or crushing the curve, saving lives.

Will this drain the battery of my phone?

By using Bluetooth Low Energy, battery consumption should be limited. Bluetooth Low Energy was created specifically for low energy consumption and is also being used to connect to smartwatches and wireless headphones. Battery consumption should be less than 5% in the worst case, if you otherwise did have Bluetooth completely disabled.

Do I need to install another app when I go abroad?

With support of the European Union, a gateway service has been built that allows the exchange of keys of infected persons between European countries. The 19th of October 2020 the Italian (Immuni), German (Corona-Warn-App) and the Irish app (StopCOVID Tracker) started using this. It is planned that other decentralized contact tracing apps of the European Union will connect to this this gateway too, for example the Belgian Coronalert will add support in November.

However note that keys can never be exchanged with the StopCovid France app because this one uses a centralized system instead of the decentralized DP-3T.

What do experts say about these apps?

First it’s important to repeat that DP-3T, and hence the Google/Apple Exposure Notifications framework based on it, were designed by academics from universities in different countries. Prof. Bart Preneel (KU Leuven), cryptographer, who contributed to the DP-3T framework, says that “for once, Google and Apple are on the right side of privacy“.

The British Information Commissioner’s Office (the national data protection authority) “believes the CTF (Google/Apple’s contact tracing framework) is aligned with the principles of data protection by design and by default, including design principles around data minimisation and security.”

In a report from prof. Douglas Leith (Trinity College Dublin), analysis of network traffic of contact tracing apps was done. He concludes: “We find that the health authority client apps are generally well behaved from a privacy point of view, although the privacy of the Irish, the Polish and Latvian apps could be improved.” They criticize the closed source nature of the Google/Apple Exposure Notifications framework though and the fact that Google Play Services sends private data to Google (something which happens on any Android phone having Google Play Services installed, irrespective of the presence and usage of this framework). Update 23 July 2020: the source code of the framework itself is now available.

The biggest criticism of the framework appears to come from prof. Serge Vaudenay, cryptographer of EPFL. He complains about the closed source nature of the GAEN, and the fact that some attacks are possible. There is an answer by the DP3T team to one of his papers. Update 23 July 2020: the source code of the GAEN framework itself is now available.

Conclusion

Forget all conspiracy theories and emotional objections by privacy activists who never looked at how these apps work: open source decentralized contact tracing apps making use of the Google/Apple Exposure Notification API are not Big Brother, no mass surveillance instrument. The protocol has been developed by academics specialized in security and privacy in IT and the source code of the apps can be verified by anyone. Extensive documentation describes the working of the apps and what is being done to protect the privacy of the users. By using anonymous ephemeral IDs and not collecting any location information, these contact tracing apps know less from you than the average social network app or your phone’s OS itself, so if you are worried about privacy, you have more important things to look at.

Contact tracing apps can be very useful in combating this epidemic, also if only a small part of the population is using them. For me it’s simply a matter of responsibility installing these apps: to protect others, to protect our society and economy and in the end to get protected myself by others using the app.

Further information

History of this article

Update 23 July 2020: Added links to the source of Google and Apple Exposure Notifications framework – Added info about Belgian contact tracing app in development – Added link to picture comparing permissions requested by SwissCovid, Facebook and Whatsapp

Update 6 September 2020: Added question about Location requirement on Android phones

Update 19 September 2020: Added link to source code of Belgian Coronalert app – Added info about an Expsore Notifications API implementation in microg, enabling you to run these apps without Google services

Update 4 October: add information about EU gateway service

Update 19 October: EU federation gateway service in use by 3 apps

LineageOS 17.1 (Android 10) on OnePlus 3/3T status

Posted on July 19, 2020October 7, 2020 by Frederik

Table of contents
Useful links
Questions, problems and solutions
Is it possible to upgrade from LineageOS 16 to 17.1?
Which gapps should I use?
I cannot find Netflix in the Google Play Store
My bank app refuses to start because the phone is rooted
Nothing happens when you press the Recent Apps button
Notification and alarm sounds are always muted
Known bugs in LineageOS 17.1 on OnePlus 3/3T
No way to disable a SIM card when using multiple SIMs
Solved problems in earlier LineageOS 17 builds
GPS not working reliably
The microphone stops working especially after making a call in apps like Telegram, Whatsapp, Skype

I own a OnePlus 3 phone and I am currently running Android 10 based LineageOS 17.1 on it. Unfortunately there is no handy (for end-users) list of known bugs and frequently asked questions for LineageOS on the OnePlus 3 and the only way to know them is to read through the LineageOS 17.1 on Oneplus 3 thread on the XDA-developers forum. The lack of such a handy list results in people asking about the same problems over and over again and this is a source of frustration for the more experienced visitors and developers in the thread, who often answer these questions in a blunt way.

Because I think such a list is really needed and lacking, I am creating one here, which I will try to keep up to date.

As the OnePlus 3T runs the same LineageOS builds the information here is also valid for that phone.

Useful links

Download LineageOS builds for Oneplus 3/3T
LineageOS changelog
LineageOS 17 on Oneplus 3/3T thread on XDA-developers forum
I strongly recommand skimming through the last 5 pages of this thread and using the search function before asking a question there, because people will get annoyed and give you a blunt answer if you ask about a topic discussed before.

Questions, problems and solutions

Is it possible to upgrade from LineageOS 16 to 17.1?

Even though they discourage to do this on XDA and will refuse to give you any support if you did not do a clean installation, it is actually possible. The procedure can lead to problems, so make sure you have proper back-ups and know how to use TWRP.

After a successful complete backup, connect your phone to your computer, and via the android notification that it’s charging via USB, select that you want to transfer files. Now open your device’s internal storage disk on your computer, and copy the complete backup in TWRP/BACKUPS/ to your computer. If you have TitaniumBackup, I recommend also making a complete back-up of all apps, and then copying this back-up which is stored in the TitaniumBackup directory on your internal storage to your PC. While connected to your PC, also make a back-up of your photos in the DCIM directory, because these are not included in backups made with TWRP and TitaniumBackup.

Download the following files, either using a browser in Android, either using your computer and in that case copy them to your phone’s internal storage.

the latest TWRP
the latest LineageOS nightly zip
the latest OpenGapps nano
the latest Magisk zip
the latest Magisk uninstaller zip
optional: the latest MadKernel

Before proceeding, reset all app permissions in LineageOS 16, because they can cause a boot loop in LineageOS 17.1 otherwise. To do so, connect your phone to your computer, enable USB debuggin in the Developer settings, and run

$ adb shell pm reset-permissions

You can also run pm reset-permissions from a terminal on your phone.

Also completely disable AFWall+ if you have installed this. Because Android 10 adds NetworkStack (com.android.networkstack, com.android.networkstack.permissionconfig) which needs to be allowed in the firewall to get network access, you can have network problems in case AFWall+ is enabled after the upgrade. I even experienced spontaneous reboots.

Now go to TWRP and choose Install – Install image, and choose the twrp image. Choose recovery and let it flash TWRP. Then reboot your phone back into the new recovery.

Now in TWRP install the Magisk uninstaller zip. I recommend uninstalling Magisk completely because sometimes it results in boot loops. Then flash LineageOS 17.1 and OpenGapps, then wipe cache and dalvik cache, and reboot. If all goes fine, you can boot back in TWRP and flash Magisk and optionally MadKernel.

If you end up in a boot loop (the boot appears to hang on the boot logo and after a long time returns to TWRP with message RescueParty), go back to TWRP, choose Wipe – Advanced wipe, and wipe system, boot, cache and dalvik. Then Install LineageOS and OpenGapps zip and reboot.

If all this fails, restore the backups you made.

In LineageOS 17.1, I recommend starting up all programs and giving them back the permissions they need. This is particularly important for the Clock app, in order to make alarms work fine. I also recommend resetting all notification, alarm and ringtone sounds, because on my phone they were muted after the update until I reset them. Before enabling AFWall+ again, make sure you allow NetworkStack.

Which gapps should I use?

Use OpenGapps nano. Never use any of the more complete packages, they will cause problems. Install the missing Google applications afterwards from the Play Store.

I cannot find Netflix in the Google Play Store

When you go the Settings in the Play Store app, under Play Protect certification you will see that your device is not certified.

Android Gadget Hacks: Fix Play Store Uncertified Errors When You Forget to Flash Magisk

My bank app refuses to start because the phone is rooted

Nothing happens when you press the Recent Apps button

This happens when you use a different launcher than the default Trebuchet launcher (for example Nova Launcher or Lawnchair) and have frozen Trebuchet. Unfreeze Trebuchet. In case you have completely uninstalled Trebuchet, reflashing LineageOS should bring it back.

Reddit: LOS17.1: OP3: Recent app switcher doesn’t work

Another possible cause is that you used a different OpenGapps version than the nano package. If this is the case, wipe the system partition, reinstall LineageOS and install OpenGapps nano.

Notification and alarm sounds are always muted

Even when your phone is not in silent mode, it’s possible that notification, alarm and ringtone sounds are not played back, and your phone only vibrates. The solution is to reset all sounds via Settings – Sounds – Advanced. Set them to a different sound, tap on Save at the right top, and then set it back to your preferred soundand again save it. It should now work. You will have to do the same in the Clocks application: for every alarm, tap on the Bell icon, and set your preferred alarm sound. If you have any apps which have their own notification sound setting (for example K9-Mail), you probably will also have to set it again in that app.

Known bugs in LineageOS 17.1 on OnePlus 3/3T

No way to disable a SIM card when using multiple SIMs

In LineageOS 16.0 it’s possible to disable one of the SIM cards if you have two SIMs installed. In LineageOS 17.1, the only way to disable a SIM is to physically remove the card from the slot.

Solved problems in earlier LineageOS 17 builds

GPS not working reliably

https://forum.xda-developers.com/oneplus-3/oneplus-3–3t-cross-device-development/rom-lineageos-17-0-oneplus-3-3t-t3990061/post83069417#post83069417
https://forum.xda-developers.com/oneplus-3/oneplus-3–3t-cross-device-development/rom-lineageos-17-0-oneplus-3-3t-t3990061/post83075619#post83075619

Getting a GPS fix took a long time or was even impossible in build before 13 July 2020. Update to a recent build, and try to clear A-GPS data with an app like GPS Status if you experience this problem.

The microphone stops working especially after making a call in apps like Telegram, Whatsapp, Skype

https://gitlab.com/LineageOS/issues/android/-/issues/1805

After making a call in a VoIP application like Telegram, Whatsapp and Skype, the microphone only works in calls but not in other applications. This means the microphone does not work when recording video, when recording voice message in different apps, Google Assistant stops listening to commands, etc. The phone needs to be rebooted to make the microphone work in all apps again.

This bug was fixed by this merge request which is included in nightly build 20200831 and later (31 August 2020) and the solution was confirmed on the XDA developers forum.

Teufel Connector review

Posted on October 27, 2019August 23, 2020 by Frederik

Already for some time, I was thinking of buying a music streamer to connect to my hifi set in order to listen to my audio cd collection I ripped to my computer (using whipper, and tagged with Musicbrainz Picard). Because I have many mix albums, gapless playback support is important. I would prefer a very broad codec support, at least including FLAC and Ogg Vorbis but by preference also Opus. Because the proprietary applications by the music streamer vendors don’t always get very good reviews, I would like to be able to control them with a third party app like BubbleUPNP, still with gapless playback. And of course I don’t want to break the bank too much.

I recently thought I found the perfect device for that: the Teufel Connector. Because of a sale I could buy this streamer for less than 140 €.

The Teufel Connector is a little box, unfortunately not being of the same size as a standard hifi component. It does not have a remote control and it also does not have a display, so the only way to control it is via the Raumfeld app on your smartphone or tablet. You can connect it via Ethernet or wifi. It does have analog and digital optical outputs, and features also analog inputs, which allows it to stream any other audio device to other multi-room Raumfeld devices in your house.

Setup

The Raumfeld app guides you through the setup process. In contrast to some information I read on the Internet, I did not have to connect my device with an Ethernet cable to set it up: you can immediately set it up to connect to your wifi network via your smartphone. I had some trouble that when I entered the wrong wifi password, not only did I have to restart the device in order to restart the startup procedure, I also had to delete all data of the Raumfeld app in order to get back to the setup screen. Maybe there is another way, but apparently this was not very intuitive. In the end I managed to set it up and the device became available.

The Raumfeld app

Playback from UPNP/DLNA server

Adding a remote UPNP/DLNA server to the Raumfeld app can be done through the app’s settings. It should list all UPNP servers, so you just have to pick it from a list. All music will then appear in the My Music part in the application.

It properly shows the album art, even if they were not saved in the music files or directory. However a serious problem shows up: it does not properly show albums containing tracks by different artists, such as compilations. Even though the Album Artist tag is correctly set (e.g. to Various Artists for compilations) Raumfeld will not create one album which contains all tracks. Instead it will create a different album with the same name for every single different artist on the disc. Obviously this is a huge problem, because it makes it impossible to play back entire compilation albums in one go. There is a tab Directory Structure in the app, but when trying to browse the directory structure of the server, it states that browsing is not available for remote music servers. So this is not an alternative either to browse albums correctly.

Playback from SMB server

So if Raumfeld does not show albums correctly when accessing them via UPNP, does it do any better when accessing them through SMB?

Adding an SMB share happens again through the app’s settings. The app never mentions SMB but only uses the term “network sources”, so this was a bit confusing. You have to enter hostname or IP address, username and password, and then you will be able to select the share you want to use. Adding your library via SMB takesa much longer because now all files have to be indexed by the Connector itself, while with UPNP this is done server side.

The good news is that this time, the problem with the compilation albums does not occur. What I don’t like however, is that albums containing multiple disks are still shown as multiple albums, one for each disk. The first disk just has the album name, the second one the album name followed by [disc 2], and so on. This way, playing a complete album requires a manual action again to switch discs, which is a pity. I would have preferred them to be shown as one album, with a division between the different discs in the track listing. Gapless playback works fine.

Several times while playing it would all of a sudden randomly switch to another track. This seems to happen especially while it’s scanning the music on the SMB source. This is very annoying and a serious bug which needs to be fixed. When it has finished scanning, this problem does not occur any more. In the app, you can set it to scan automatically every day, or to scan only when you manually ask it do so. Anyway, scanning your library is needed when you added new music files, so this is a huge problem if your library is not static. I contacted support for this problem, and they even do not consider this to be a bug. They blame the wifi network, and tell me that it probably would not happen if I would play MP3 files instead of FLAC. This seems complete nonsense to me: while scanning the music files, there is about 120 KB/s traffic between server and Connector, and when simultaneously playing a FLAC file it would jump occasionally to less than 2 MB/s. With the server connected by UTP to the router, and the Connector being less than 5 meters away from the router in the same room, bandwidth problems cannot explain this at all.

Playback from a local USB device

The Connector has a USB port to which you can connect an external disk. The Connector automatically indexes all music files and adds them to the library. I have not extensively tested this feature.

Playback of Internet streams

Raumfeld supports playback of a number of different Internet sources, such as Spotify, TuneIn, Soundcloud, Tidal and Napster. Maybe this is not the most complete offering of streaming services (for example Google Play Music, Amazon Music and Deezer are not there), but these other services can usually by accessed by means of Chromecast. More on that later.

TuneIn

For radio streams, TuneIn is used. You can check if your favourite station is included on the TuneIn website. If it’s not available, you can always add a custom stream in the Raumfeld app, but currently you need the beta version for that.

There is no way to link Raumfeld with your TuneIn account, so it is not possible to import your TuneIn Favourites in case you used TuneIn before.

Unfortunately many radio streams use a bitrate which is a bit too low for perfect quality. Of course this is not Teufel’s nor TuneIn’s fault, but keep this in mind in case you consider replacing your FM or DAB+ tuner by a network streamer.

Spotify

The Connector supports Spotify Connect, which means you can play Spotify music on your Connector directly from the Spotify app if you have a Spotify Premium subscription. I don’ t have such a subscription at the moment so I did not test it.

Soundcloud

To use Soundcloud, you need to log in with a Soundcloud account. Then all the artists you follow, tracks you liked, etc. will appear in the Raumfeld app. For an unknown reason to me, I could not find some tracks available on the website, such as the Purified radio show.

Unfortunately it is not possible to like tracks and follow artists on Soundcloud via the Raumfeld app, so you still need to do this via the website.

Use of the Raumfeld app

The Raumfeld app contains the basic features to get your music playing, but they should definitely take a look at the Spotify app to see how things can be made much more user friendly.

The Raumfeld app has a permanent notification in your notification list, where you can see what is playing, pause playback, switch to the previous and next song, and change the output volume (at least if you are using the analog output to your amplifier). There is also a widget available if you prefer that.

Playing just random tracks from your library, but it’s not as easy as it could be. Unlike Spotify, there is no big green Shuffle button in the All tracks tab page of the app. Instead you have to start playback by choosing a random song yourself and start playback, and enable the Shuffle option in the now playing window. Actually, after a while I discovered that there is actually an easy option hidden in the Playlist section, where there is a pre-defined playlist “My Music Shuffle”. This could be easier to find.

In the Now Playing window, there there is no option to go immediately to the album of the playing track, or to the list of all tracks of that artist. This is a handy feature that can be found in Spotify and other music players.

There is no way to properly close the Raumfeld app, except for going to Android’s settings and killing the app there. I have noticed several times when going away from home, and then connecting to another wifi network, the permanent notification would still be there as if I can start playback immediately. After some time eventually, the notification will say that the player is not available, giving you the options to close the app by pressing the X in the notification. I think there really should be a way to manually quit the application at any time.

While it’s possible to add different network sources to your music library, they all get mixed up in one big library, while I would prefer them to be in two different libraries. For example, I have two different SMB shares, one for classical music, and another one for pop music. When I want to play music, I want to have the possibility of seeing only the albums from the classical music library or only albums of the pop music library. This could be done by creating subitems for every single network sources under My Music in the menu. Choosing My Music would bring you to a combined library, while tapping one of the subitems would only show the contents of that network source. Unfortunately, the app does not make that possible, and you end up with all network sources mixed up.

The Dutch translation of the Raumfeld app needs some work. For example there is a button “Verwijder bron” (= Remove source) in the sources settings. When choosing this option it asks for confirmation: “Verplaats bron?” which actually means “Move source”. When clicking on the Delete button in a playlist, the possible answers to the question whether you are sure are “Geen” (= None) and “Ja” (= Yes). And in the Playlist section, the wrong word Schuffles is used. Even if not really a great translation, at least Shuffles would more acceptable. I noticed other errors too, so they really need to do some work proofreading the Dutch translation.

Playback via BubbleUPNP

I had a disappointing experience using the Raumfeld app, when playing either from my UPNP server or my SMB server. What if I use BubbleUPNP instead?

BubbleUPNP sees the Connector in 3 different ways: one as a single Connector UPNP render, one as meta-device named after the room (this would contain all Teufel UPNP devices you have added to that room), and once as a Chromecast device.

When playing music to the Connector, the playback time is not updated in BubbleUPNP. You cannot go forward or backwards within the track and get the error: “Seek mode not support (code: 710)”. When the song is finished and it continues with the next song, BubbleUPNP does not even notice that it went to the next song. So this is not usable.

When using the virtual room device as a renderer, plaback time is updated and seeking does work. However, gapless playback is not supported at all. This is also the case when letting BubbleUPNP stream to the Chromecast device.

All in all, the Connector does not have flawless UPNP support making it BubbleUPNP not a usable alternative to the Raumfeld app.

Chromecast support

Every application which supports Google Chromecast, and many music applications in Android do, can cast their stream to the Connector. It’s as easy as pressing the cast button in the application, and then selecting your Connector. It works simple and good and allows you to listen to streaming services which are not natively supported by the Raumfeld app.

Audio quality

Hifi magazines usually spend the most time writing about this subject when reviewing audio hardware, but I have the feeling that this is purely psychological and that there is rarely any audible difference between different audio source hardware. I can only say that there is nothing wrong with the audio quality of this device, and I don’t believe a device which costs 5 or 10 times as much can sound any better.

Of course all depends of the quality of your source material. I am using FLAC files directly ripped from CD, so there is no quality loss there. When listening to lower bitrate Internet streams (and unfortunately many of them still use older codecs such as MP3 in combination with a not high enough bitrate), you will of course clearly hear that it’s far from CD quality, but that is not something your hardware can fix.

The Connector should support playback of hi-res audio files up to 24 bit 192 Khz, but I have not tested this.

Support

On the Dutch website, there is contact page mentioning e-mail and a contact form, however no e-mail address nor contact form for technical support questions can be found there. On the repair and returns page there is the address service@teufelaudio.nl. This should be easier to find.

Support has been useless to me. I contacted them regarding the problem with the compilation albums on UPNP but seemed clueless, and they blamed the problems on the automatic switching of tracks on my wifi network, denying any bug there.

Conclusion

The Teufel Connector is a versatile machine supporting many codecs (including Opus and high-res music files) and gapless playback. Thanks to the Chromecast support, you can stream many online services to the device, even though the Raumfeld app itself only supports a limited selection of sources. The price is low, as is power consumption. So in theory this should be a great device, at least if you can live without a dedicated remote control, display and control buttons on the device itself.

Unfortunately the Connector is completely let down by the buggy firmware and Raumfeld app. UPNP was totally useless to me because it does not correctly show albums containing tracks of different artists. SMB on the other hand, requires a lengthy scanning process, during which it is impossible to listen to music from your library because it randomly switches tracks the whole time. BubbleUPNP is not a usable alternative for the Connector, because the Connector’s implementation as a UPNP media renderer also appears to be incomplete and buggy.

Several times during usage I encountered hangs in the Raumfeld app, or of the Connector device itself, requiring to disconnect the power to force a hard reset. Whether I used the stable version of the app or the beta version, did not make any difference. I did not try the beta version of the firmware.

All these problems could still be fixable by firmware and app updates. But is it realistic that they still will get fixed, knowing that this device is already on the market for years, and is being superseded by the Teufel Streamer now? The Raumfeld app only gets a score of 3.0 in the Android Play Store and there are many complaints, also in combination with other Teufel Raumfeld devices.

In conclusion I cannot recommend this device if you want to use it mainly to play your local music library. If you want to use it only to listen to online streams, than you could consider it, but otherwise, look further.

It’s disappointing that in 2019 finding a good streamer is still not easy. Much more important than the hardware, is the firmware and software. If you have to choose which music streamer to buy, I strongly recommend reading the app reviews on the Play Store instead of reviews of so-called hifi and multimedia magazines. Looking at the app reviews, then Sonos with a score of 4.0 seems to be the best one, followed by Heos from Denon and Marantz (3.6). Cambridge Audio’s StreamMagic (2.9, however only 7 reviews now as it is brand new), Onkyo’s Controller (2.9), Pioneer’s Remote Control (3.0), which appears to be the same as Onkyo’s, Yamaha’s Musiccast (3.0) don’t seem to be any better than Raumfeld unfortunately.

What’s your experience with this or with other streamers, such as the Denon DNP-800NE or Marantz NA6006, the Yamaha NP-S303, or anything else? Do they support gapless playback, also when using BubbleUPNP? How stable are they and is the app user friendly? Can you create multiple, separate libraries in it?

Install ModSecurity and the Core Rule Set on Debian

Configuring ModSecurity

Configuring the Core Rule Set

Anomaly Scoring

Setting the paranoia level

Defining the allowed HTTP methods

Disallowing old HTTP versions

Blocking specific countries

Core Rule Set Exclusion rules for well-known web applications

Adding rules for Log4Shell and Spring4Shell detection

Testing ModSecurity and checking the logs

Fixing false positives

Conclusion

Available Work-arounds

Remove libopenblas0-pthread

Switch the default BLAS implementation from OpenBlas to Atlas

Disable the sandbox in totem-video-thumbnailer

Regenerating the failed thumbnails

More information

Table of contents

Useful links

Questions, problems and solutions

Is it possible to upgrade from LineageOS 17 to 18.1?

Which gapps should I use?

I cannot find Netflix in the Google Play Store

My bank app refuses to start because the phone is rooted

Known bugs in LineageOS 18.1 on OnePlus 3/3T

The phone crashes from time to time

No way to disable a SIM card when using multiple SIMs

Third-party ClamAV virus definitions

SaneSecurity

SecuriteInfo

Malwarepatrol

MalwareExpert

URLHaus

Clam-punch

TwinWave Security Twinclams

R-FX Networks Linux Malware Detect

InterServer

ditekshen

Installing Fangfrisch

Configuring Fangfrisch

Checking the new signatures before letting ClamAV use them

Initalizing Fangfrisch

Configuring Clamav

Configuring Amavis

Automatically running Fangfrisch using a systemd timer

Ignoring virus definitions

Conclusion

Which Linux distribution?

Which desktop environment?

Table of contents

DP-3T

Apple/Google Exposure Notifications API

Frequently Asked Questions – Debunking some myths

Will these apps violate my privacy? Will the authorities know who I meet and where I am, what I do?

How can I be sure that the app really works as promised and really does not collect and send private information?

Why should I trust Google and Apple, who have a bad track record in privacy?

Why does this app require Location setting to be enabled on my Android phone if no location information is collected?

Will this app give me a warning every time an infected person passes nearby, resulting in many false positives?

Are these apps useful if not everyone or at least a large part of the population downloads them?

Will this drain the battery of my phone?

Do I need to install another app when I go abroad?

What do experts say about these apps?

Conclusion

Further information

History of this article

Table of contents

Useful links

Questions, problems and solutions

Is it possible to upgrade from LineageOS 16 to 17.1?

Which gapps should I use?

I cannot find Netflix in the Google Play Store

My bank app refuses to start because the phone is rooted

Nothing happens when you press the Recent Apps button

Notification and alarm sounds are always muted

Known bugs in LineageOS 17.1 on OnePlus 3/3T

No way to disable a SIM card when using multiple SIMs

Solved problems in earlier LineageOS 17 builds

GPS not working reliably