If you have an e-mail server, maybe you already are using the open source tools Amavis and ClamAV to detect malicious e-mails. ClamAV’s default virus signatures however, while being useful, still only detect a limited set of malware. Fortunately there exist third-party unofficial signatures which you can use to drastically improve the detection rate so that it becomes similar to the detection rate of commercial anti-virus software. These third-party signatures are a must-have on any mail server using ClamAV.
ClamAV can also be very useful to scan the webroot directories on web servers in order to detect malicious PHP scripts which may be installed on your server after an intrusion in a web application. Several of the third-part ClamAV definitions proposed here, specialize in this kind of malware.
Fangfrisch is a utility which automates downloading third-party ClamAV virus definitions. It has several free repositories containing ClamAV signatures configured by default, but you can also easily add other ones.
Third-party ClamAV virus definitions
SaneSecurity
SaneSecurity is a set of signatures focusing on so-called 0-day and 0-hour malware, which means that it includes hashes of new malicious files being sent by e-mail. Furthermore it contains signatures for malicious URLs, common spam and phishing messages and generic signatures which detect some commonly types of techniques used in malware, such as exe files with a double extenstion (for example pdf.exe), exe files hidden in ISO files and often abused functions in MS Office macros. Sanesecurity also distributes signatures from other sources, such as phishing URLs from phishtank.com. A complete list of available signature databases can be found on their website.
SaneSecurity signatures are free to use, however a donation is appreciated.
SecuriteInfo
The French company SecuriteInfo claims that their ClamAV definitions add 4.000.000 signatures for malware and spam not detected by the official ClamAV signatures. There is a free feed of the signatures available, however it only contains signatures older than 30 days. For up-to-date 0-day malware detection, you will need one of the paid plans.
In the MalwareBazaar database you can see that SecuriteInfo is often the only ClamAV definitions database which detects malicious Windows binaries.
Malwarepatrol
MalwarePatrol is a commercial threat intelligence company which offers free and paid feeds, including ClamAV virus signatures which contain URL’s pointing to malware files on the world wide web. The free feed is only updated every 72 hours, while the paid feed is updated every 4 hours. If you need an invoice or want to use the feeds for protecting customers of your company, you will have to use the commercial feeds.
Quickly after integrating the free feed in my Amavis installation, I noticed it was wrongly blocking legitimate mails with links to arxiv.org/pdf, which is used a lot in academic environments. So be careful when you integrate this feed. For this reason you should not automatically block all messages marked by Malwarepatrol. With Amavis you can give them them a positive spam score. In combination with the score from the Bayes spam filter, legitimate mails will not be blocked.
MalwareExpert
MalwareExpert offers a commercial, paid feed of signatures which detect malicious PHP files, meant for scanning your web servers.
URLHaus
URLHaus is a project from abuse.ch collecting URLs of sites distributing malware. They offer a ClamAV signature database of all these malicious URLs so that you can block all e-mail containing links to sites distributing malware. This database can be used for free, both for commercial and non-commercial purposes.
Clam-punch
Clam-punch is described as a “highly curated set of signatures for ClamAV”. It seems to focus mainly on malicious macros in MS Office documents. They do not seem to be updated regularly any more, however as the signatures appear to be rather generic, they can probably still be useful.
TwinWave Security Twinclams
Twinclams is a Github repository by TwinWave Security and contains signatures for malicious MS Office documents. The author of twinclams appears to be a contributor of Clam-punch. This ClamAV virus database is getting updated daily and appears to be highly effective in detecting newly found Office documents with malicious macros. Recent malicious Office document in the MalwareBazaar database are almost always detected by the Twinclams definitions, sometimes even detecting files which are not detected yet by commercial anti-virus software.
R-FX Networks Linux Malware Detect
Part of its Linux Malware Detect (LMD) tool, R-FX Networks offers a set of ClamAV signatures specializing in detecting Linux specific malware, including malicious PHP scripts, trojan horses such as malicious IRC bots, worms, etc.
InterServer
Another set of ClamAV signatures focused on malicious PHP scripts is maintained by the hosting provider InterServer. I highly recommend this feed if you want to scan your web servers.
ditekshen
The ClamAV signatures by security reseacher ditekshen add detection for various Windows, MacOS and Linux trojan horses en ransomware.
Installing Fangfrisch
Unfortunately Debian does not have a package for Fangfrisch, so you need to install it manually. In contrast to the documentation, I prefer to install fangfrisch in /opt though, using /var/lib/fangfrisch only for the database, and saving the signatures in /var/lib/fangfrisch/signatures, so that we can check them later on before letting ClamAV use them.
# mkdir -m 0770 -p /var/lib/fangfrisch/signatures
# chgrp -R clamav /var/lib/fangfrisch
# mkdir /opt/fangfrisch
# cd /opt/fangfrisch
# apt-get install python3-venv python3-pip
# python3 -m venv venv
# source venv/bin/activate
# pip3 install fangfrisch
Configuring Fangfrisch
Create the file /etc/fangfrisch/fangfrisch.conf with these contents:
[DEFAULT]
db_url = sqlite:////var/lib/fangfrisch/db.sqlite
# The following settings are optional. Other sections inherit
# values from DEFAULT and may also overwrite values.
local_directory = /var/lib/fangfrisch/signatures
max_size = 5MB
on_update_exec = /usr/local/bin/setup-clamav-sigs
on_update_timeout = 42
log_level = info
[malwarepatrol]
enabled = no
# Replace with your personal Malwarepatrol receipt
# receipt = XXXXXXXXX
# change product id if necessary.
# 32 = free guard, 33 = Basic Defense yearly, 34 = Basic Defense monthly, 37 = Basic Defense EDU/Contributor
# product = 32
# This is untested
[malwareexpert]
enabled = no
max_size = 20M
prefix = https://signatures.malware.expert
interval = 1d
# serial_key = xxxxxxx
url_malware.expert_fp = ${prefix}/${serial_key}/malware.expert.fp
url_malware.expert_hdb = ${prefix}/${serial_key}/malware.expert.hdb
url_malware.expert_ldb = ${prefix}/${serial_key}/malware.expert.ldb
url_malware.expert.ndb = ${prefix}/${serial_key}/malware.expert.ndb
[sanesecurity]
prefix = https://ftp.swin.edu.au/sanesecurity/
max_size = 10M
enabled = yes
interval = 1h
url_malwareexpert_fp = disabled
url_malwareexpert_hdb = disabled
url_malwareexpert_ldb = disabled
url_malwareexpert_ndb = disabled
[securiteinfo]
enabled = no
# uncomment the next line if you want to use the securiteinfoold.hdb database with old signatures
# max_size = 500MB
# Replace with your personal SecuriteInfo customer ID
# customer_id = abcdef123456
# Remove the exclamation mark before the next databases if you have the Professional subscription
!url_0hour = ${prefix}securiteinfo0hour.hdb
!url_securiteinfo_mdb = ${prefix}securiteinfo.mdb
# The next databases are disabled by default in fangfrisch because they are prone to false positives, but we reduce most of them to a spam score in Amavis
url_old = ${prefix}securiteinfoold.hdb
url_spam_marketing = ${prefix}spam_marketing.ndb
[urlhaus]
enabled = yes
max_size = 2MB
[twinwave]
enabled = yes
max_size = 2M
integrity_check = disabled
interval = 1h
prefix = https://raw.githubusercontent.com/twinwave-security/twinclams/master/
url_twinclams = ${prefix}twinclams.ldb
url_twinwave_ign2 = ${prefix}twinwave.ign2
[clampunch]
enabled = yes
max_size = 2M
integrity_check = disabled
interval = 24h
prefix = https://raw.githubusercontent.com/wmetcalf/clam-punch/master/
url_miscreantpunch099low = ${prefix}MiscreantPunch099-Low.ldb
url_exexor99 = ${prefix}exexor99.ldb
url_miscreantpuchhdb = ${prefix}miscreantpunch.hdb
[rfxn]
enabled = yes
interval= 4h
integrity_check = disabled
prefix = https://www.rfxn.com/downloads/
url_rfxn_ndb = ${prefix}rfxn.ndb
url_rfxn_hdb = ${prefix}rfxn.hdb
url_rfxn_yara = ${prefix}rfxn.yara
[interserver]
enabled = yes
interval = 1d
integrity_check = disabled
prefix = https://sigs.interserver.net/
url_interserver_sha256 = ${prefix}interserver256.hdb
url_interserver_topline = ${prefix}interservertopline.db
url_interserver_shell = ${prefix}shell.ldb
url_interserver_whitelist = ${prefix}whitelist.fp
[ditekshen]
enabled = yes
interval = 1d
integrity_check = disabled
prefix = https://raw.githubusercontent.com/ditekshen/detection/master/clamav/
url_ditekshen_ldb = ${prefix}clamav.ldb
filename_ditekshen_ldb = ditekshen.ldb
Don’t forget to customize the entries if you have any paid subscription. Set some safe file permissions on this file, especially if it contains tokens of commercial subscriptions:
# chown root:clamav /etc/fangfrisch/fangfrisch.conf
# chmod 640 /etc/fangfrisch/fangfrisch.conf
Checking the new signatures before letting ClamAV use them
In the configuration file above, I call the script setup-clamav-sigs whenever signatures were updated. This script will check whether the downloaded files are really different than the signatures in /var/lib/clamav, and if they are, check them to ensure they don’t contain errors preventing ClamAV to load them. If ClamAV can successfully load them, it will copy them to /var/lib/clamav and restart clamdscan if it’s loaded. Download the script from the GitLab repository, place it in /usr/local/bin and make sure it’s executable.
Initalizing Fangfrisch
When you have set up Fangfrisch, you have to initialize it by executing
sudo -u clamav -- /opt/fangfrisch/venv/bin/fangfrisch --conf /etc/fangfrisch.conf initdb
Configuring Clamav
SecuriteInfo recommends setting these settings in /etc/clamav/clamd.conf in order to get the best detection while still avoiding too many false positives:
DetectPUA yes
ExcludePUA PUA.Win.Packer
ExcludePUA PUA.Win.Trojan.Packed
ExcludePUA PUA.Win.Trojan.Molebox
ExcludePUA PUA.Win.Packer.Upx
ExcludePUA PUA.Doc.Packed
Configuring Amavis
I assume you already have a working Amavis instance. Sanesecurity gives some recommendations for Amavis for the best results. Add this to the configuration, for example In Debian you can create the file /etc/amavis/conf.d/50-clamav:
@keep_decoded_original_maps = (new_RE(
qr'^MAIL$', # retain full original message for virus checking (can be slow)
qr'^MAIL-UNDECIPHERABLE$', # recheck full mail if it contains undecipherables
qr'^(ASCII(?! cpio)|text|uuencoded|xxencoded|binhex)'i,
# qr'^Zip archive data', # don't trust Archive::Zip
));
@virus_name_to_spam_score_maps =
(new_RE( # the order matters!
[ qr'^TwinWave\.' => undef ],# keep as infected
[ qr'^MiscreantPunch\.' => undef ],# keep as infected
[ qr'^Structured\.(SSN|CreditCardNumber)\b' => 2.0 ],
[ qr'^(Heuristics\.)?Phishing\.' => 2.0 ],
[ qr'^(Email|HTML)\.Phishing\.(?!.*Sanesecurity)' => 10.0 ],
[ qr'^Sanesecurity\.(Malware|Badmacro|Foxhole|Rogue|Trojan)\.' => undef ],# keep as infected
[ qr'^Sanesecurity\.Phishing\.' => 6.0 ],
[ qr'^Sanesecurity\.Blurl\.' => 4.0 ],
[ qr'^Sanesecurity\.Jurlbl\.' => 2.0 ],
[ qr'^Sanesecurity\.Spam\.' => 2.0 ],
[ qr'^Sanesecurity\.SpamL\.' => 2.0 ],
[ qr'^Sanesecurity\.Junk\.' => 4.0 ],
[ qr'^Sanesecurity\.Scam4\.' => 2.0 ],
[ qr'^Sanesecurity\.' => 0.1 ],
[ qr'^Sanesecurity.TestSig_' => 0 ],
[ qr'^Email\.Spam\.Bounce(\.[^., ]*)*\.Sanesecurity\.' => 0 ],
[ qr'^BofhlandMW\.' => undef ],# keep as infected
[ qr'^Bofhland\.Malware\.' => undef ],# keep as infected
[ qr'^Bofhland\.' => 2.0 ],
[ qr'^winnow.malware\.' => undef ],# keep as infected
[ qr'^winnow\_' => 2.0 ],
[ qr'^PhishTank\.Phishing\.' => 6.0 ],
[ qr'^Porcupine\.Malware\.' => undef ],# keep as infected
[ qr'^Porcupine\.' => 2.0 ],
[ qr'^Email\.Spammail\b' => 2.0 ],
[ qr'^Safebrowsing\.' => 2.0 ],
[ qr'^winnow\.(phish|spam)\.' => 2.0 ],
[ qr'^SecuriteInfo.com\.Phish' => 6.0 ],
[ qr'^SecuriteInfo.com\.Spam' => 2.0 ],
[ qr'^MBL_' => 4.0 ],
));
These settings ensure that not only the different parts and attachments will be separately scanned by ClamAV, but also the mail as a whole. Then we will reduce some ClamAV virus detections from infected to a spam score in Spamassassin. I do this on rules which could be more likely causing false positives. You can adapt the score to your own situation.
Automatically running Fangfrisch using a systemd timer
Create the file /etc/systemd/system/fangfrisch.service:
[Unit]
Description=Download unofficial clamav virus definition files
ConditionPathExists=/var/lib/fangfrisch/db.sqlite
[Service]
Type=oneshot
User=clamav
WorkingDirectory=/var/lib/fangfrisch
ExecStart=/opt/fangfrisch/venv/bin/fangfrisch --conf /etc/fangfrisch/fangfrisch.conf refresh
[Install]
WantedBy=multi-user.target
Then create the file /etc/systemd/system/fangfrisch.timer:
[Unit]
Description=Download unofficial clamav virus definition files
Requires=fangfrisch.service
[Timer]
Unit=fangfrisch.service
Persistent=true
OnUnitActiveSec=10min
RandomizedDelaySec=30
[Install]
WantedBy=timers.target
Run this command to activate the timer:
# systemctl enable --now fangfrisch.timer
Ignoring virus definitions
It is possible that you hit false positives with certain definitions. If you want to completely disable a specific virus definition, you can add its name to a text file with the ign2 extension in /var/lib/clamav/. For example for me the Sanesecurity.Badmacro.Doc.hypers caused a false positive for me, so I created the file /var/lib/clamav/local_whitelist.ign2 with content:
Sanesecurity.Badmacro.Doc.hypers
To reload the database after changing the database, run
# sudo -u clamav clamdscan --reload
Conclusion
With free third-party databases (Sanesecurity, URLHaus, Clam-punch, Twinclams, R-FX MLD, Interserver and ditekShen) it is possible to drastically improve the detection rate of ClamAV so that it becomes an excellent virus scanner for e-mail and web servers at least if you use it In combination with Amavis’ configuration to block malicious file types (such as exe, com, vbs, dll, pif, etc.) and a well configured and trained Spamassassin. If you want the best protection, add a subscription to the SecuriteInfo feeds.
