Tips for avoiding SPAM

(c) Scott M. Baker, 1997

What is SPAM?

The term SPAM is generally used to refer to off-topic commercial messages which are posted to newsgroups. These messages are usually advertisements for some form of 'Pay' service, usually of an adult nature. Typically, advertisers post SPAM messages multiple times and to multiple newsgroups.

While it's hard to blindly condemn commercialism, some of this has really gotten out of hand. Some SPAM advertisers post hundreds of bulky off-topic pictures to newsgroups where they clearly do not belong. Some of them even post adult material to non-adult groups. This kind of recklessness and complete disregard for Internet users has had a dreadful impact on usenet newsgroups.

There are ways to get by in a SPAM-filled world though. Through the use of SBNews and the techniques I describe below, I have found it easy to eliminate nearly all of the SPAM out there. My experiements show that by using the techniques below, you can easilly reduce the amount of SPAM you download to less than 1%. That's less than one SPAM download for every hundred good downloads.

How can SBNews be set up to avoid SPAM?

I will present a list of techniques below, most of which that I use myself on my own configuration:

1. "Maximum XRef Limit". The most effective technique, which also requires very little manual effort is to use the Maximum XRef Limit. This setting is located under the pull-down menu "Configure:Preferences". Typically, a SPAM advertiser posts to many newsgroups at a time. When an article is posted to many newsgroups, the news server automatically generates an XRef header in the message which contains a list of which groups the article is posted to. To make a long story short, you can tell SBNews to ignore a message that is posted to too many newsgroups. I usually find a setting of 8 to be appropriate, although these days an even lower setting (5, or even 2 or 3) may be necessary. Note: You may also want to check the "preload xref hdrs" box, described in step 3.
2. "Lockout XRef". You can find this option under the pull-down menu "Configure:Lockout:Xref Group Name". As described above, the XRef line contains a listing of newsgroups to which the message is posted to. There may be some newsgroups which contains subject matter that you are absolutely certain that you don't want to receive (perhaps material that is offensive to you). If a message is cross-posted to one of these undesirable groups, then you probably don't want it. You can enter the names of groups that are undesirable into the Lockout XRef dialog box, and SBNews will ignore any message which is cross-posted to the groups that you list. Note: You may also want to check the "preload xref hdrs" box described in step 3.
3. "Preload XRef Headers". This option is located under the pull-down menu "Configure:Preferences". By default, SBNews does not pre-download the header lines containing xref information. This is done to make the header download process faster for people who do not use the XRef options described above. Thus, SBNews doesn't know that a message should be ignored until the message has begun downloading. To abort the message, a disconnect/reconnect cycle is required, which is a bit inefficient. However, if you make heavy use of the xref limit/lockouts (described above), then you will probably want to pre-download the xref headers. Checking this option will cause SBNews to download the headers ahead of time, so that SBNews can decide whether a message should be ignored without having to begin downloading the message. I highly recommend this option if you use steps 1 & 2 above.
4. "Minimum Message Lines". Located under the pull-down menu "Configure:Preferences". There are a lot of SPAM advertisers that post short text messages into the binary groups. Although these messages are short, they still do waste some of your download time parsing through them. I recommend setting the minimum lines setting to "100". If there's anything with less than 100 lines, then it probably isn't worth having anyway.
5. "Lockout Poster". Located under the pull-down menu "Configure:Lockout:Poster", "Current:Lockout", and "Previous:Lockout". The poster is the name of the person who sent the message. SPAM advertisers usually invent bogus names, but they do tend to reuse the names frequently. Sometimes they will consistently use the same domain part of a name. When you lockout a poster, SBNews will ignore any messages posted by that name. You can manually enter lockouts using the "Configure:Lockout:Poster" dialog box, or you can automatically lockout a person who posted an unsuitable image in the previous/current thumbnails by using the "Current:Lockout" and "Previous:Lockout" options. You don't have to specify a full name, you can specify just part of the name. For example, if you locked out "@bogusspam.com", that would match "tom@bogusspam.com", "joe@bogusspam.com", etc.
6. "Lockout Subject". Located under the pull-down menu "Configure:Lockout:Subject". SPAM advertisers have to mention their service somewhere, and they usually like to do it in the subject field of the message. Sometimes they'll stick an http address in there. As with the Lockout Poster option, you don't have to type a full subject line here, you can just enter the part of the subject that identifies the SPAM advertiser. For example, "phone sex" would eliminate any message subject that had that phrase in it.
7. Add "free" to Lockout Subject. Located under the pull-down menu "Configure:Lockout:Subject". Every commercial SPAM advertiser wants you to think that their pay service is free, so the majority of them put the word "free" in the subject line. Locking out free will get rid of a considerable amount of junk. There is a risk with this option -- you will lose some relevant data from people who actually are posting something that is free, but this is the exception rather than the rule. [Recommended cautiously due to possible loss of relevant material]
8. Add "http://" to Lockout Subject. Located under the pull-down menu "Configure:Lockout:Subject". Commercial SPAM advertisers like to put their http address in the subject line so that you know how to get to their service. As with the previous step, you will probably miss out on some actual relevant material from people who are posting http addresses to their website. Again, this is the exception rather than the rule. [Recommended cautiously due to possible loss of relevant material]
9. Use the <Headers> button. You can use the <headers> button to parse through the headers of a newsgroup and lockout/reject messages as you see ones you don't want. This is rather labor intensive, but it does provide good results. SBNews will need to actually load the headers for a group before you can edit them (thus, you must be connected).
10. "Lockout Any". Located under the pull-down menu "Configure:Lockout:Any". You can lockout a phrase from any header line. This works on subject, from, posting-host, xref, and about a dozen other header lines in the message. Thus, if you know a phrase you absolutely don't want ("phone sex" is a good candidate here!), you can type in in here and SBNews will abort any message with the offending phrase.
11. "Lockout Posting Host". Located under the pull-down menu "Configure:Lockout:Posting-Host". This is a real power-user option. Each message includes a "NNTP-Posting-Host" field which identifies the host from which the message was sent. Usually this host is the ISP (Internet Service Provider) of the SPAM advertiser. To find "NNTP-Posting-Host" fields, you'll have to use the <Headers> button, and the <Read> button located in the headers list to read an offending message, then scroll through header lines of the message itself. "NNTP-Posting-Host" should be one of them. Enter the host name into the lockout dialog. Like I said, this is a power user type function and should only be used by experienced persons. Most SPAM advertisers do not bother to supply a fictitious posting host, so this usually works. However, you will reject an entire host (which could be thousands of people), so this should be used with caution. With version 5.3 and above, you can lockout the host of the current (or previous) images by using Current:Lockout or Previous:Lockout and enabling the posting host field.
12. "Preload host hdrs". You can find this option under Configuration:Preferences. If you make extensive use of the "Lockout Posting Host" feature above, then you'll want to receive all of the posting host headers ahead of time so that SBNews does not have to do a disconnect/reconnect cycle on every locked out host that is found.
13. Submit complaints. The only way SPAM can really be stopped is if enough complaints are registered to the appropriate authorities. Some ISP's have very strict anti-SPAM policies and will terminate any users who post spam. This is usually not the case for the big commercial sites though. Who do you complain to? The SPAM posters usually go to great lengths to hide their true identities so they cannot be traced. However, if you user the <Headers> button and <Read> the messages, you can find some interesting data in the message headers. Look at the "Path:", "Organization:", and "NNTP-Posting-Host:" lines -- these usually have some valid addresses. You can try sending complaints in that direction. The domain part is not enough to send to alone, you'll need a full email address. You can pre-append the names "webmaster@", "support@", "complaints@", "orders@", etc to the addresses and see if you can get somewhere. (For example, if you see "Organization: bogusspam.com", try posting to "webmaster@bogusspam.com") Sometimes your own ISP will have a complaints department to which you can attach a message to and they will follow-up and contact offender's ISP themselves. :

Tips for specific SPAM programs:

1. There have been a rash of SPAM messages with random poster names, perhaps the result of some new autoposter. These poster names tend to look something like "dsfgs@aol.com (dsfgs)" or something, with random text inserted for each message, making the lockout poster system useful. The best way to deal with these is to use the "Lockout Posting Host" feature. When you see one of these messages go by, use the Current/Previous menus and select lockout. Then check the box next to posting host.

Miscellaneous Tips:

1. You can use the "lockout" option located under the current and previous menus to quickly lockout the poster, subject, file, or posting host of the current or previous image. This is a great time-saving step. You can also edit the text in any of the fields in the dialog and lockout the modified text. [Version 5.3 and above]
2. You can use the <Headers> button to view which messages have been locked out and to add additional lockouts. Thus you can see the immediate results of your lockouts.

Summary:

So how will it end?

You have to ask yourself, why are the commercial sites advertising in the newsgroups? There can only be one reason: Because it works. Because they are actually gaining customers by forcing advertisements down their throats. There's a simple solution to this -- do not use the pay sites. There's plenty of free stuff on the web -- just look around. The newsgroups are a good example -- once you filter out the SPAM, there is a large amount of on-topic material left behind.

Maybe the solution is to communicate to the SPAMMERS that there are better ways to advertise. Posting 100 off-topic messages only enagages the average user into "ignore mode". Flooding the newsgroups with 100 off-topic messages only draws people away from the newsgroups. Posting 100 off-topic messages will generate complaints. Sooner or later a capable authority will get involved. My advice to all the SPAMMERS out there, "If you absolutely have to advertise in the newsgroups, post one on-topic advertisement. If people really want to visit your site, they'll do it."

Closing Notes:

This document was originally written for SBNews version 4.8. I will try to keep it updated as new versions are released. The above mechanisms and techniques are some of the basic building blocks in SBNews and will probably be present for all time, in more or less their current format.

You can find the latest version of SBNews at:

http://www.newsrobot.com/sbnews/sbnews.html