c0d3 :: j0rg3

A collection of tips, tricks and snips. A proud Blosxom weblog. All code. No cruft.

Sun, 13 Jul 2014

Simple Protection with iptables, ipset and Blacklists

Seems I’ve always just a few more things going on than I can comfortably handle. One of those is an innocent little server holding the beginnings of a new project.

If you expose a server to the Internet, very quickly your ports are getting scanned and tested. If you’ve an SSH server, there are going to be attempts to login as ‘root’ which is why it is ubiquitously advised that you disable root login. Also why many advise against allowing passwords at all.

We could talk for days about improvements; it’s usually not difficult to introduce some form of two-factor authentication (2FA) for sensitive points of entry such as SSH. You can install monitoring software like Logwatch which can summarize important points from your logs, such as: who has logged via SSH, how many times root was used, etc.

DenyHosts and Fail2ban are very great ways to secure things, according to your needs.

DenyHosts works primarily with SSH and asks very little from you in way of configuration, especially if you’re using a package manager to install a version that is configured for the distribution on which you’re working. If you’re installing from source you may need to find where are your SSH logs (e.g., /var/log/secure, /var/log/auth.log). It’s extremely easy to set up DenyHosts to synchronize so that you’re automatically blocking widely-known offenders whether or not they’re after your server.

In contrast, Fail2ban is going to take more work to get set up. However, it is extremely configurable and works with any log file you point it toward which means that it can watch anything (e.g., FTP, web traffic, mail traffic). You define your own jails which means you can ban problematic IP addresses according to preference. Ban bad HTTP attempts from HTTP only or stick their noses in the virtual corner and don’t accept any traffic from them until they’ve served their time-out by completely disallowing their traffic. You can even use Fail2ban to scan its own logs, so repeating offenders can be locked out for longer.

Today we’re going to assume that you’ve a new server that shouldn’t be seeing any traffic except from you and any others involved in the project. In that case, you probably want to block traffic pretty aggressively. If you’ve physical access to the server (or the ability to work with staff at the datacenter) then it’s better to err in the direction of accidentally blocking good guys than trying to be overly fault-tolerant.

The server we’re working on today is a Debian Wheezy system. It has become a common misconception that Ubuntu and Debian are, intents and purposes, interchangeable. They’re similar in many respects and Ubuntu is great preparation for using Debian but they are not the same. The differences, I think, won’t matter for this exercise but I am unsure because this was written using Wheezy.

Several minutes after bringing my new server online, I started seeing noise in the logs. I was still getting set up and really didn’t want to stop and take protective measures but there’s no point in securing a server after its been compromised. The default Fail2ban configuration was too forgiving for my use. It was scanning for 10 minutes and banning for 10 minutes. Since only a few people should be accessing this server, there’s no reason for anyone to be trying a different password every 15 minutes (for hours).

I found a ‘close-enough’ script and modified it. Here, we’ll deal with a simplified version.

First, lets create a name for these ne’er-do-wells in iptables:
iptables -N bad_traffic

For this one, we’ll use Perl. We’ll look at our Apache log files to find people sniffing ‘round and we’ll block their traffic. Specifically, we’re going to check Apache’s ‘error.log’ for the phrases ‘File does not exist’ and ‘client denied by server configuration’ and block people causing those errors. This would be excessive for servers intended to serve the general populace. For a personal project, it works just fine as a ‘DO NOT DISTURB’ sign.


#!/usr/bin/env perl
use strict;
use POSIX qw(strftime);

my $log = ($ARGV[0] ? $ARGV[0] : "/var/log/apache2/error.log");
my $chain = ($ARGV[1] ? $ARGV[1] : "bad_traffic");

my @bad = `grep -iE 'File does not exist|client denied by server configuration' $log |cut -f8 -d" " | sed 's/]//' | sort -u`;
my @ablk = `/sbin/iptables -S $chain|grep DROP|awk '{print $4}'|cut -d"/" -f1`;

foreach my $ip (@bad) {
if (!grep $_ eq $ip, @ablk) {
chomp $ip;
`/sbin/iptables -A $chain -s $ip -j DROP`;
print strftime("%b %d %T",localtime(time))." badht: blocked bad HTTP traffic from: $ip\n";
}
}

That gives us some great, utterly unforgiving, blockage. Looking at the IP addresses attempting to pry, I noticed that most of them were on at least one of the popular block-lists.

So let’s make use of some of those block-lists! I found a program intended to apply those lists locally but, of course, it didn’t work for me. Here’s a similar program; this one will use ipset for managing the block-list though only minor changes would be needed to use iptables as above:

#!/bin/bash
IP_TMP=ip.tmp
IP_BLACKLIST_TMP=ip-blacklist.tmp

IP_BLACKLIST=ip-blacklist.conf

WIZ_LISTS="chinese nigerian russian lacnic exploited-servers"

BLACKLISTS=(
"http://danger.rulez.sk/projects/bruteforceblocker/blist.php" # BruteForceBlocker IP List
"http://rules.emergingthreats.net/blockrules/compromised-ips.txt" # Emerging Threats - Compromised IPs
"http://www.spamhaus.org/drop/drop.txt" # Spamhaus Don't Route Or Peer List (DROP)
"http://www.spamhaus.org/drop/edrop.txt" # Spamhaus Don't Route Or Peer List (DROP) Extended
"http://cinsscore.com/list/ci-badguys.txt" # C.I. Army Malicious IP List
"http://www.openbl.org/lists/base.txt" # OpenBL.org 90 day List
"http://www.autoshun.org/files/shunlist.csv" # Autoshun Shun List
"http://lists.blocklist.de/lists/all.txt" # blocklist.de attackers
)

for address in "${BLACKLISTS[@]}"
do
echo -e "\nFetching $address\n"
curl "$address" >> $IP_TMP
done

for list in $WIZ_LISTS
do
wget "http://www.wizcrafts.net/$list-iptables-blocklist.html" -O - >> $IP_TMP
done

wget 'http://wget-mirrors.uceprotect.net/rbldnsd-all/dnsbl-3.uceprotect.net.gz' -O - | gunzip | tee -a $IP_TMP

grep -o '^[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}[/][0-9]\{1,3\}' $IP_TMP | tee -a $IP_BLACKLIST_TMP
grep -o '^[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}[^/]' $IP_TMP | tee -a $IP_BLACKLIST_TMP

sed -i 's/\t//g' $IP_BLACKLIST_TMP
sort -u $IP_BLACKLIST_TMP | tee $IP_BLACKLIST

rm $IP_TMP
rm $IP_BLACKLIST_TMP

wc -l $IP_BLACKLIST

if hash ipset 2>/dev/null
then
ipset flush bloxlist
while IFS= read -r ip
do
ipset add bloxlist $ip
done < $IP_BLACKLIST
else
echo -e '\nipset not found\n'
echo -e "\nYour bloxlist file is: $IP_BLACKLIST\n"
fi


Download here:
    bad_traffic.pl
    bloxlist.sh


Tags: , , , , , , , , , ,
Permalink: 20140713.simple.protection.with.iptables.ipset.and.blacklilsts

Sat, 25 Jan 2014

Network-aware Synergy client

My primary machines are *nix or BSD variants, though I certainly have some Windows-based rigs also. Today we’re going to share some love with Windows 7 and PowerShell.

One of my favorite utilities is Synergy. If you’re not already familiar it allows to you seamlessly move from the desktop of one computer to another with the same keyboard and mouse. It even supports the clipboard so you might copy text from a GNU/Linux box and paste it in a Windows’ window. Possibly, they have finished adding drag and drop to the newer versions. I am not sure because I run a relatively old version that is supported by all of the machines that I use regularly.

What’s the problem, then? The problem was that I was starting my Synergy client by hand. Even more disturbing, I was manually typing the IP address at work and at home, twice or more per weekday. This behavior became automated by my brain and continued for months unnoticed. But this is no kind of life for a geek such as myself, what with all this superfluous clicking and tapping!

Today, we set things right!

In my situation, the networks that I use happen to assign IP addresses from different subnets. If you’ve not the convenience of that situation then you might need to add something to the script. Parsing an ipconfig/ifconfig command, you could possibly use something like the Default Gateway or the Connection-specific DNS Suffix. Alternatively, you could check for the presence of some network share, a file on server or anything that would allow you to uniquely identify the surroundings.

As I imagined it, I wanted the script to accomplish the following things

  • see if Synergy is running (possibly from the last location), if so ask if we need to kill it and restart so we can identify a new server
  • attempt to locate where we are and connect to the correct Synergy server
  • if the location is not identified, ask whether to start the Synergy client

This is how I accomplished that task:

# [void] simply supresses the noise made loading 'System.Reflection.Assembly'
[void] [System.Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms")

# Define Synergy server IP addresses
$synergyServerWork = "192.168.111.11"
$synergyServerHome = "192.168.222.22"

# Define partial IP addresses that will indicate which server to use
$synergyWorkSubnets = "192.168.111", "192.168.115"
$synergyHomeSubnets = "192.168.222", "192.168.225"

# Path to Synergy Client (synergyc)
$synergyClientProgram = "C:\Program Files\Synergy\synergyc.exe"

# Path to Syngery launcher, for when we cannot identify the network
$synergyLauncherProgram = "C:\Program Files\Synergy\launcher.exe"

# Remove path and file extension to give us the process name
$processName = $synergyClientProgram.Substring( ($synergyClientProgram.lastindexof("\") + 1), ($synergyClientProgram.length - ($synergyClientProgram.lastindexof("\") + 5) ))

# Grab current IP address
$currentIPaddress = ((ipconfig | findstr [0-9].\.)[0]).Split()[-1]

# Find the subnet of current IP address
$location = $currentIPaddress.Substring(0,$currentIPaddress.lastindexof("."))


function BalloonTip ($message)
{
# Pop-up message from System Tray
$objNotifyIcon = New-Object System.Windows.Forms.NotifyIcon
$objNotifyIcon.Icon = [System.Drawing.Icon]::ExtractAssociatedIcon($synergyClientProgram)
$objNotifyIcon.BalloonTipText = $message
$objNotifyIcon.Visible = $True
$objNotifyIcon.ShowBalloonTip(15000)
}


#main

# If Synergy client is already running, do we need to restart it?
$running = Get-Process $processName -ErrorAction SilentlyContinue
if ($running) {
$answer = [System.Windows.Forms.MessageBox]::Show("Synergy is running.`nClose and start again?", "OHNOES", 4)
if ($answer -eq "YES") {
Stop-Process -name $processName
}
Else {
exit
}
}

# Do we recognize the current network?
if ($synergyWorkSubnets -contains $location) {
BalloonTip "IP: $($currentIPaddress)`nServer: $($synergyServerWork)`nConnecting to Synergy server at work."
& $synergyClientProgram $synergyServerWork
exit
}
ElseIf ($synergyHomeSubnets -contains $location) {
BalloonTip "IP: $($currentIPaddress)`nServer: $($synergyServerHome)`nConnecting to Synergy server at home."
& $synergyClientProgram $synergyServerHome
exit
}
Else {
$answer = [System.Windows.Forms.MessageBox]::Show("Network not recognized by IP address: {0}`n`nLaunch Synergy?" -f $unrecognized, "OHNOES", 4)
if ($answer -eq "YES") {
& $synergyLauncherProgram
}
}

Then I saved the script in "C:\Program Files\SynergyStart\", created a shortcut and used the Change Icon button to make the same as Synergy’s and made the Target:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -WindowStyle Hidden & 'C:\Program Files\SynergyStart\synergy.ps1'

Lastly, I copied the shortcut into the directory of things that run when the system starts up:
%APPDATA%\Microsoft\Windows\Start Menu\Programs\Startup

Now, Synergy connects to the needed server at home and work. If it can’t figure out where it is, it asks if it should run it at all.

As they say, a millisecond saved is a millisecond earned.

This post was very nearly published without a Linux equivalent. Nearly.

Same trick for bash/zsh: #!/bin/zsh

# Define Synergy server IP addresses
synergyServerWork="192.168.111.11"
synergyServerHome="192.168.222.22"

# Define partial IP addresses that will indicate which server to use
synergyWorkSubnets=("192.168.111" "192.168.115")
synergyHomeSubnets=("192.168.222" "192.168.225")

# Path to Synergy Client (synergyc)
synergyClientProgram="/usr/bin/synergyc"

# Path to QuickSyngery, for when we cannot identify the network
synergyLauncherProgram="/usr/bin/quicksynergy"

# Remove path and file extension to give us the process name
processName=`basename $synergyClientProgram`

# Grab current IP address, assumes '192' is in use. (e.g., 192.168.1.1)
currentIPaddress=`ip addr show | grep 192 | awk "{print $2}" | sed 's/inet //;s/\/.*//;s/ //g'`

# Find the subnet of current IP address
location=`echo $currentIPaddress | cut -d '.' -f 1,2,3`

for i in "${synergyWorkSubnets[@]}"
do
if [ "${i}" = "${location}" ]
then
break
fi
done

#main

# If Synergy client is already running, do we need to restart it?
running=`ps ax | grep -v grep | grep $processName`
if [ $running ]
then
if `zenity --question --ok-label="Yes" --cancel-label="No" --text="Synergy is running.\nClose and start again?"`
then
pkill $processName
else
exit
fi
fi

# Do we recognize the current network?
for i in "${synergyWorkSubnets[@]}"
do
if [ "${i}" = "${location}" ]
then
notify-send "IP:$currentIPaddress Server:$synergyServerWork [WORK]"
$synergyClientProgram $synergyServerWork
exit
fi
done

for i in "${synergyHomeSubnets[@]}"
do
if [ "${i}" = "${location}" ]
then
notify-send "IP:$currentIPaddress Server:$synergyServerWork [HOME]"
$synergyClientProgram $synergyServerHome
exit
fi
done

if `zenity --question --ok-label="Yes" --cancel-label="No" --text="Network not recognized by IP address: $currentIPaddress\nLaunch Synergy?"`
then
$synergyLauncherProgram
fi

To get it to run automatically, you might choose to call the script from /etc/init.d/rc.local.

Download here:
  PowerShell:
    synergy.ps1
  GNU/Linux:
    synergy.sh


Tags: , ,
Permalink: 20140125.network_aware_synergy_client

Thu, 06 Jun 2013

Managing to use man pages through simple CLI tips

Recently, an author I admire and time-honored spinner of the Interwebs, Tony Lawrence emphasized the value of using man pagesmanual pagesDocumentation available from the command line.
> man ls
as a sanity check before getting carried away with powerful commands. I didn’t know about this one but he has written about a situation in which killall could produce some shocking, and potentially quite unpleasant, results.

Personally, I often quickly check man pages to be certain that I am using the correct flags or, as in the above case, anticipating results that bear some resemblance to what is actually likely to happen. Yet, it seems many people flock toward SERPSearch Engine Results Page A tasteful replacement for mentioning any particular search-engine by name.
Also useful as a verb:
I dunno. You’ll have to SERP it.
s for this information.

Perhaps the most compelling reason to head for the web is leaving the cursor amid the line you’re working on, without disturbing the command. SERPing the command however, could easily lead you to information about a variant that is more common than the one available to you. More importantly, the information retrieved from the search engine is almost certainly written by someone who did read the man page — and may even come with the admonishment that you RTFMRead The F#!$!*#’n Manual as a testament to the importance of developing this habit.

This can be made easier with just a few CLI shortcuts.

<CTRL+u> to cut what you have typed so far and <CTRL+y> to paste it back.

That is, you press <CTRL+u> and the line will be cleared, so you can then type man {command} and read the documentation. Don’t hesitate to jot quick notes of which flags you intend to use, if needed. Then exit the man page, press <CTRL+y> and finish typing right where you left off.

This is another good use for screen or tmux but let’s face it. There are times when you don’t want the overhead of opening another window for a quick look-up and even instances when these tools aren’t available.

A few other tips to make life easier when building complex commands:

Use the command fc to open up an editor in which you can build your complex command and, optionally, even save it as a shell script for future reuse.

Repeat the last word from the previous command (often a filename) with <ALT+.> or use an item from the last command by position, in reverse order:
> ls -lahtr *archive*
<ALT+1+.> : *archive*
<ALT+2+.> : -lahtr
<ALT+3+.> : ls

You can also use Word Designators to use items from history, such as adding sudo to the last command typed by:
sudo !!

This allows for tricks like replacing bits of a previous command:
!:s/misspelled/corrected/

Lastly, if you need a command that was typed earlier, you can search history by pressing <CTRL+r> and start typing an identifying portion of the command.

(Note: I have used these in Zsh and Bash, specifically. They can, however, be missing or overwritten — if a feature you want isn’t working, you can bind keys in a configuration file. Don’t just write it off, once you’ve solved the problem it will never again be an intimidating one.)

Happy hacking!


Tags: , , , , , , , ,
Permalink: 20130606.managing.to.use.man.pages