Ham and Spam: Difference between revisions

From Motorcycle Action Group Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
Line 4: Line 4:
Now we need to actually give it examples from which it can learn.
Now we need to actually give it examples from which it can learn.


== The Spam and Ham folders ===
== The Spam and Ham folders ==


If you create two new folders in your account, on the same level as the Junk and Inbox folders, called "Ham" and "Spam" (note capital letters), this will be your signup to the teaching system. They may well already be there.
If you create two new folders in your account, on the same level as the Junk and Inbox folders, called "Ham" and "Spam" (note capital letters), this will be your signup to the teaching system. They may well already be there.
Line 28: Line 28:
=== Virus Check ===
=== Virus Check ===


A virus scanner called [https://www.clamav.net/ Clam Anti Virus] is used to screen all mails
A virus scanner called [https://www.clamav.net/ Clam Anti Virus] is used to screen all mails. This scanner sits at the front door and checks all incoming mails before they are even accepted. If it comes back as spam, the mail is rejected, so it doesn't even get inside our systems.


== To Be Continued ==
=== MailScanner ===
 
This is a framework designed to bolt all the various anti-spam/anti-virus technologies together, and it also does a few checks of its own.
 
=== Spam Assassin ===
 
The single most popular email spam filter out there is "SpamAssassin", an open source project. This tool is used behind the scenes by all the best known hosting services out there. It runs the mail to be examined through a basket of checks for oddities in the mail headers, for rude words and such. It also checks the mail servers that have sent the mail and links within it against online lists.
 
=== SPF ===
 
This is a very simple mechanism designed to inform other sites which mail servers are the ones that should be sending MAG email. If another mail server on the Internet receives a mail claiming to come form a ???@mag-uk.org address, the receiving server can now double check that it came from a genuine MAG server.
 
=== DKIM ===
 
All mails sent from MAG's servers are signed using a protocol called "DKIM". This means that the recipient can check that the mail has not been altered en-route and more importantly that the sender really was a MAG server. The same utility also checks the mails coming into MAG's servers to see whether any signatures it finds from other sites are genuine or not. A lot of the major email hosters (Google, Yahoo etc) also use this system, so our servers can now tell if someone is pretending to send a mail from Google, but is actually sending it from their own server; a behaviour repeatedly found in spam mails.
 
=== DMARC ===
 
This is a quite new protocol, also used by the major mail providers, that draws DKIM and SPF together and provides instructions to other servers about what to do with mails that fail the SPF or DKIM tests. It is still quite new, but has proved to be very powerful.
 
= In Closing =
 
If you have any questions or worries about any of this, do please email the systems administrator at [http://mailto:mag-uk-sysadmin@mag-uk.org mag-uk-sysadmin@mag-uk.org].
 
--[[User:Meredith|Meredith]] ([[User talk:Meredith|talk]]) 13:40, 17 November 2016 (GMT)

Latest revision as of 14:40, 17 November 2016

The Plague

We along with all other email users on this planet are plagued by Spam. The main part of the problem is that machines cannot discriminate between the good stuff, called Ham and the bad stuff, Spam. One way around this is to teach the machines the difference. There are now packages that can do a very basic form of learning and there is one installed on our server.

Now we need to actually give it examples from which it can learn.

The Spam and Ham folders

If you create two new folders in your account, on the same level as the Junk and Inbox folders, called "Ham" and "Spam" (note capital letters), this will be your signup to the teaching system. They may well already be there.

There is a process that runs once a minute that checks to see if anyone with a Ham or Spam folder has anything in that folder. If there is anything there, it picks it up, runs it through the learning system and then throws it away. It will then send you a thank you mail to let you know what it learned and that your efforts are appreciated by the rest of the users on the system.

Please remember that all mails put in either of those folders will be deleted, so if you want to keep a copy, you will need to have one in another folder. NB There is very little chance that mails fed to the Spam or Ham folders and deleted can be restored from backup.

Commercial Newsletters are not Spam

A lot of people think that Spam just means "Any mail I don't want to read". However, there is a lot of mail that you may not want to read that isn't Spam, for example commercial news letters and circulars. The discriminating factor being that you don't get to "unsubscribe" from true spam. Annoying but perfectly legal commercial email will have a section informing the recipient how they can get off the mailing list. Mostly this involves clicking a link marked "Unsubscribe". Sadly there is no standardised way of doing this though, so you might have to take a second glance. You may also have to get a magnifying glass out as it is quite often in a small font. However, this really is the best way of getting rid of these emails for good.

If you bung circulars and news letters in the spam bin, this is highly unlikely to stop the next in the series as it will have different contents and subject, so will have an entirely different fingerprint. It may also be something that someone else on the system is looking forward to each week, and if their copy arrives after you have binned yours, then this *will very likely stop theirs from arriving.

Explicit and Obscene Spam

Every now and then a mail address is targeted with a particularly obnoxious class of Spam, rife with obscene language. Please do drop a copy in your Spam folder, but also forward it to mag-uk-sysadmin@mag-uk.org. We take particular exception to this kind of spam and extra measures are taken to prevent it from getting through. We will also act on complaints by tracking down the network it came from and submitting a formal complaint to the administrators of that network. We regularly get positive responses and appologies from network admins along with confirmation that action, sometime legal action has been taken.

How does MAG Stop Spam

The MAG mail filtration system does not rely on any one type of analysis, there is a whole basket of tests run on every mail attempting to pass into our systems.

Virus Check

A virus scanner called Clam Anti Virus is used to screen all mails. This scanner sits at the front door and checks all incoming mails before they are even accepted. If it comes back as spam, the mail is rejected, so it doesn't even get inside our systems.

MailScanner

This is a framework designed to bolt all the various anti-spam/anti-virus technologies together, and it also does a few checks of its own.

Spam Assassin

The single most popular email spam filter out there is "SpamAssassin", an open source project. This tool is used behind the scenes by all the best known hosting services out there. It runs the mail to be examined through a basket of checks for oddities in the mail headers, for rude words and such. It also checks the mail servers that have sent the mail and links within it against online lists.

SPF

This is a very simple mechanism designed to inform other sites which mail servers are the ones that should be sending MAG email. If another mail server on the Internet receives a mail claiming to come form a ???@mag-uk.org address, the receiving server can now double check that it came from a genuine MAG server.

DKIM

All mails sent from MAG's servers are signed using a protocol called "DKIM". This means that the recipient can check that the mail has not been altered en-route and more importantly that the sender really was a MAG server. The same utility also checks the mails coming into MAG's servers to see whether any signatures it finds from other sites are genuine or not. A lot of the major email hosters (Google, Yahoo etc) also use this system, so our servers can now tell if someone is pretending to send a mail from Google, but is actually sending it from their own server; a behaviour repeatedly found in spam mails.

DMARC

This is a quite new protocol, also used by the major mail providers, that draws DKIM and SPF together and provides instructions to other servers about what to do with mails that fail the SPF or DKIM tests. It is still quite new, but has proved to be very powerful.

In Closing

If you have any questions or worries about any of this, do please email the systems administrator at mag-uk-sysadmin@mag-uk.org.

--Meredith (talk) 13:40, 17 November 2016 (GMT)