Thu, 13 Jul 2017
I’m not entirely certain when BackTrack/Kali began behaving more like a regular desktop distro but I seem to recall that originally, networking subsystems were down when you booted up into Run Level 3. It was up to you to turn on the interfaces and fire up a GUI if such was desired. IMO, that’s precisely how it should be. I get it. Most of us aren’t ever won’t ever find ourselves in a clandestine lot, inside of a snack and caffeine filled, non-descript, conversion van with a Yagi pointed at the bubble-window, ready to pilfer innocent datums just trying to get by in this lossy-protocoled, collision-rife, world.
Rather, very many of us just want the stinking box online so we can run through our tutorials and hack our own intentionally vulnerable VMs. A thorough taste of hacking’s un-glamorous underbelly is quite enough for many.
I’m confident that the BT fora were inundated with fledgling hackers complaining that their fresh install couldn’t find WiFi or didn’t load the desktop. However, I feel that distros dedicated to the Red Team should try to instill good habits. Having your machine boot and activate an interface announcing your presence and spewing out MAC and hostname is bad for business. Booting into a (comparatively) heavy GUI is also not where I want to begin.
Let’s imagine that we’re trying to crack into a thing. Don’t we want to apply maximal CPU resources, rather than having GUI elements bringing little beyond cost? If you notice, very many of the related tools still live on the CLI. The typical course of development (e.g.: Nmap, Metasploit) is that the CLI version is thoroughly developed before someone drops a GUI atop (respectively: Zenmap, Armitage).
So let’s take our Kali and make a few quick changes. We want to boot up in text/CLI mode and we want networking left off until we choose to make noise. Further, we want to randomize our MAC address and hostname at every boot.
We’ll use iwconfig
to enumerate our wireless interfaces.
lo no wireless extensions.
wlan1 IEEE 802.11 ESSID:"ESSID"
Mode:Managed Frequency:2.412 GHz Access Point: 17:23:53:96:BE:67
Bit Rate=72.2 Mb/s Tx-Power=20 dBm
Retry short limit:7 RTS thr:off Fragment thr:off
Encryption key:off
Power Management:off
Link Quality=70/70 Signal level=-21 dBm
Rx invalid nwid:0 Rx invalid crypt:0 Rx invalid frag:0
Tx excessive retries:253 Invalid misc:400 Missed beacon:0
eth0 no wireless extensions.
wlan0 IEEE 802.11 ESSID:off/any
Mode:Managed Access Point: Not-Associated Tx-Power=0 dBm
Retry short limit:7 RTS thr:off Fragment thr:off
Encryption key:off
Power Management:on
We have two wireless interfaces: wlan0, wlan1
Okay, first let’s configure to start up in text mode:
> systemctl set-default multi-user.target
Created symlink /etc/systemd/system/default.target → /lib/systemd/system/multi-user.target.
Traditionally from text mode, we bring up the GUI desktop with the command startx
. Since we don’t yet have that command, let’s create it:
> echo "systemctl start gdm3.service" > /usr/sbin/startx && chmod +x /usr/sbin/startx
Disable network-manager autostart:
> systemctl disable network-manager.service
> sed -i 's/5min/30sec/' /etc/systemd/system/network-online.target.wants/networking.service
Now, let’s randomize our hostname and MAC addresses at every boot by adding some cronjobs:
> crontab -e
We’ll add two jobs to randomize MAC address and one for our host name:
@reboot macchanger -r wlan0
@reboot macchanger -r wlan1
@reboot hostname `strings /dev/urandom | grep -o '[[:alnum:]]' | head -n 30 | tr -d '\n'`
We ‘re good! We’ve improved efficiency by staving off the GUI for when we genuinely want it and improved anonymity by randomizing some common ways of identifying the rig.
Tags: BackTrack, hostname , ifconfig, Kali, MAC, macchanger, Privacy, systemctl, systemd
Permalink: 2017-07-10.improved.anonymity.on.kali.linux
Sun, 19 Feb 2017
Hello friends.
While the overall telos of this blog is to, generally speaking, convey code snippets and inspire the personal projects of others, today we’re going to do something a smidgeon different.
This will be a layman’s look at varied dimensions of information security from a comfortable distance. Over the years, I’ve secured servers, operating systems, medical data, networks, communications and I’ve unsecured many of these same things. The topics are too sprawling to be covered in a quick summary — but let’s find a point of entry.
Those of us who are passionate about information security are well aware of how daunting is the situation. For newcomers, it sometimes seems rather impossible. Pick any subject and there are probably well-informed and convincing experts in diametric equidistance from any “happy medium”.
Let’s imagine that (like most of us) you don’t have anything spectacular to protect. However, you dislike the idea of our ever-dissolving privacy. Therefore you want to encrypt communications. Maybe you begin to use Signal. However, there are criticisms that there is a “backdoor” (there is not). Further, there are accusations that open source projects are coded by those who can’t get real jobs. Conversely, open source projects are widely open for peer review. If it worries one enough they are free to review code themselves.
PGP can encrypt content but concerns surround algorithmic selections. Some are worried about metadata crumbs. Of course, there’s nothing preventing the frequent switching of keys and email addresses. You could use BitMessage, any number of chat solutions or drop at paste bins.
Let’s leave those concerns aside for when you’ve figured out what you’re intending to protect. These arguments surround any subject in information security and we’re not going to investigate them on a case by case basis. Least, not in this post.
At the coarsest granularity, the question is analogous to the practicality of locking your doors or sealing your post envelopes. Should I take measures toward privacy?
My opinion is rather predictable: of course you should!
There’s a very pragmatic explanation. If there ever comes a day when you should like to communicate privately, that’s a terrible time to start learning.
Take the easy road and start using some of the myriad tools and services available.
Should you decide to take InfoSec seriously, you’ll need to define a threat model.
That is: What am I protecting? From whom am I protecting? (e.g. what are probable attack vectors?)
That’s where you need to make choices about trusting products, protocols, methods, algorithms, companies, servers, et cet. Those are all exciting subjects to explore but all too often brushing up against them can be exasperating and cause premature burn-out.
That in mind, let’s employ the philosophy that any effort toward security is better than none and take a look at a few points where one might get wetted-toes.
If you have questions or want specific advice, there are several ways below to initiate a secure conversation with me.
Secure your browser:
Privacy Badger: Block tracking
HTTPS Everywhere: Increase your encryptioning
uBlock: Advertisements are for others
Secure communications:
Mailvelope: PGP email encryption for your major webmail provider (e.g., Gmail) | contact | pubkey
Tutanota: Encrypted webmail | Kontakt
Protonmail: Well-established provider of PGP encrypted webmail, featuring 2FA | kontakta
BitMessage: P2P encrypted communications protocol | contact: BM-2D9tDkYEJSTnEkGDKf7xYA5rUj2ihETxVR | Bitmessage channel list
[
Bitmessage in a Docker container ]
BitMessage.ch: BitMessage email gateway | contact
BitMsg.me: Online BitMessage service
Keybase.io: Keybase maps your identity to your public keys, and vice versa
Signal: PGP encrypted TXT messages
Wire: Encrypted chat, video and calls
RIOT: Open-source, IRC-based, Matrix; run your own server
Wickr: Encrypted ephemeral chat
[
n.b. Wickr’s .deb package seeks a unicode library (libicu52) which is not available to a recent Kali (or anything) install; .deb file is based on Ubuntu’s 2014 LTS release. Wickr in a Docker container ]
Explore alternate nets (e.g., Deep Web, Dark Net):
MaidSafe: Promising new alt-web project
Qubes: a reasonably secure operating system
FreeNet: Alt-net based primarily on already knowing with whom you intend to collaborate
Bitmask: VPN solution to anonymize your traffic
TAILS: A live operating system based on the Tor network
TorBrowser: Stand-alone browser for Tor (less secure than TAILS)
Whonix: the most secure (and complex) way to access the Tor network
i2p: an other approach to creating a secure and private alternate web
Morph.is: fun alt-net, aimed at producing The World Brain. Although, it’s future looks a lot less promising since the lead dev was killed.
ZeroNet: one more encrypted anonymous net
Have fun and compute safely!
Tags: 2FA, Bitmask, BitMessage, chat, EFF, encryption, FreeNet, HTTPS Everywhere, i2p, infosec, Kali, Keybase, MaidSafe, Mailvelope, Morph.is, paste bin, PGP, privacy, Privacy Badger, Protonmail, pubkey, Qubes, Signal, snarf.info, TAILS, threat model, TorBrowser, Tutanota, uBlock, Ubuntu, WhisperSystems, Whonix, Wickr, ZeroNet
Permalink: 20170219.privacy.prespective.primer
Tue, 10 Jan 2017
Mucking about with a fresh copy of Kali brings to attention that it’s packaged with an Armitage that doesn’t correctly work.
I know what you’re thinking… Good. Type the commands into Msfconsole like a real man, y’uh lazy good-fer-naught! And, in practice, that was my immediate solution. But I can’t resist a good tinker when things are misbehaving.
I was anticipating that the problem would be thoroughly solved when I ixquicked it. That was partially correct. Surprised, however, when apt-get update && apt-get upgrade
didn’t fix the issue. More surprised at the age of the issue. Most surprised that I could see lots of evidence that users have been plagued by this issue — but no clear work arounds were quickly found.
Guess what we’re doing today?
Okay. The issue is quite minor but just enough to be heartbreaking to the fledgling pentester trying to get a VM off the ground.
In brief, the owner of Armitage’s Github explains:
The MSF Scans feature in Armitage parses output from Metasploit’s portscan/tcp module and uses these results to build a list of targets it should run various Metasploit auxiliary modules against. A recent-ish update to the Metasploit Framework changed the format of the portscan/tcp module output. A patch to fix this issue just needs to account for the new format of the portscan/tcp module.
That is, a colon makes it into the input for the Msfconsole command to define RHOSTS. I.e.: set RHOSTS 172.16.223.150: - 172.16.223.150
An other kind coder tweaked the regex and submitted the patch and pull request, which was successfully incorporated into the project.
Sadly, things have stalled out there. So if this problem is crippling your rig, let’s fix it!
We just want a fresh copy of the project.
root@kali:~/armitage# git clone https://github.com/rsmudge/armitage
Cloning into ‘armitage’…
remote: Counting objects: 7564, done.
remote: Total 7564 (delta 0), reused 0 (delta 0), pack-reused 7564
Receiving objects: 100% (7564/7564), 47.12 MiB | 2.91 MiB/s, done.
Resolving deltas: 100% (5608/5608), done.
Kali is Debian-based and we’re going to need Apache Ant:
root@kali:~/armitage# apt-get install ant
Then, we’ll build our new fella:
root@kali:~/armitage# cd armitage
root@kali:~/armitage# ./package.sh
Buildfile: /root/test/armitage/build.xml
clean:
BUILD SUCCESSFUL
Total time: 0 seconds
Buildfile: /root/test/armitage/build.xml
init:
[mkdir] Created dir: /root/test/armitage/bin
compile:
[javac] Compiling 111 source files to /root/test/armitage/bin
[javac] depend attribute is not supported by the modern compiler
[javac] Note: /root/test/armitage/src/ui/MultiFrame.java uses or overrides a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.
BUILD SUCCESSFUL
Total time: 2 seconds
Buildfile: /root/test/armitage/build.xml
init:
compile:
jar:
[unzip] Expanding: /root/test/armitage/lib/sleep.jar into /root/test/armitage/bin
[unzip] Expanding: /root/test/armitage/lib/jgraphx.jar into /root/test/armitage/bin
[unzip] Expanding: /root/test/armitage/lib/msgpack-0.6.12-devel.jar into /root/test/armitage/bin
[unzip] Expanding: /root/test/armitage/lib/postgresql-9.1-901.jdbc4.jar into /root/test/armitage/bin
[unzip] Expanding: /root/test/armitage/lib/javassist-3.15.0-GA.jar into /root/test/armitage/bin
[copy] Copying 4 files to /root/test/armitage/bin/scripts-cortana
[jar] Building jar: /root/test/armitage/armitage.jar
[jar] Building jar: /root/test/armitage/cortana.jar
BUILD SUCCESSFUL
Total time: 1 second
armitage/
armitage/readme.txt
armitage/teamserver
armitage/cortana.jar
armitage/armitage.jar
armitage/armitage-logo.png
armitage/armitage
armitage/whatsnew.txt
adding: readme.txt (deflated 55%)
adding: armitage.exe (deflated 49%)
adding: cortana.jar (deflated 5%)
adding: armitage.jar (deflated 5%)
adding: whatsnew.txt (deflated 65%)
armitage/
armitage/readme.txt
armitage/teamserver
armitage/cortana.jar
armitage/armitage.jar
armitage/armitage-logo.png
armitage/armitage
armitage/whatsnew.txt
Archive: ../../armitage.zip
inflating: readme.txt
inflating: armitage.exe
inflating: cortana.jar
inflating: armitage.jar
inflating: whatsnew.txt
And here, best I can guess from messages read, is where a lot of people are running into trouble. We have successfully produced our new working copy of armitage. However, it is in our own local directory and will not be run if we just enter the command: armitage
Let’s review how to figure out what we want to do about that.
First, we want to verify what happens when we run the command armitage
.
root@kali:~/armitage# which armitage
/usr/bin/armitage
Good! Let’s check and see what that does!
root@kali:~/armitage# head /usr/bin/armitage
#!/bin/sh
cd /usr/share/armitage/
exec ./armitage “$@”
Almost there! It’s running /usr/share/armitage/armitage
with whatever variables we’ve passed in. We’ll check that out.
root@kali:~/armitage# head /usr/share/armitage/armitage
#!/bin/sh
java -XX:+AggressiveHeap -XX:+UseParallelGC -jar armitage.jar $@
We have enough information to assemble a solution.
I trust that the people behind Kali and Armitage will get this corrected so I don’t want to suggest a solution that would replace the armitage
command and prevent an updated version from running later. So, let’s just make a temporary replacement?
root@kali:~/armitage# echo -e '#!/bin/sh\njava -XX:+AggressiveHeap -XX:+UseParallelGC -jar ~/armitage/armitage.jar $@' > /usr/bin/tmparmitage
Hereafter, we can use the command ‘tmparmitage’ (either CLI or ALT-F2) to run our fresh version until things catch up.
And, of course, to save you the time, weary hacker:
Download here:
armitage_quick_fix.sh
Tags: Ant, Armitage, Debian
, Java, Kali, Metasploit, Msfconsole, regex
Permalink: 20170110.armitage.not.working.in.kali
Mon, 02 Jan 2017
Happy new year! New year means new servers, right?
That provides its own set of interesting circumstances!
The server we’re investigating in this scenario was chosen for being a dedicated box in a country that has quite tight privacy laws. And it was a great deal offered on LEB.
So herein is the fascinating bit. The rig took a few days for the provider to set up and, upon completion, the password for SSHing into the root account was emailed out. (o_0)
In very security-minded considerations, that means that there was a window of opportunity for bad guys to work on guessing the password before its owner even tuned in. That window remains open until the server is better secured. Luckily, there was a nice interface for reinstalling the OS permitting its purchaser to select a password.
My preferred approach was to script the basic lock-down so that we can reinstall the base OS and immediately start closing gaps.
In order:
Set up SSH keys (scripted)
Disable password usage for root (scripted)
Install and configure IPset (scripted. details in next post)
Install and configure fail2ban
Install and configure PortSentry
In this post, we’re focused on the first two steps.
The tasks to be handled are:
Generate keys
Configure local SSH to use key
Transmit key to target server
Disable usage of password for ‘root’ account
We’ll use ssh-keygen to generate a key — and stick with RSA for ease. If you’d prefer ECC then you’re probably reading the wrong blog but feel encouraged to contact me privately.
The code:
#!/bin/bash
#configure variables
remote_host="myserver.com"
remote_user="j0rg3"
remote_pass="thisisaratheraquitecomplicatedpasswordbatterystaple" # https://xkcd.com/936/
local_user=`whoami`
local_host=`hostname`
local_date=`date -I`
local_filename=~/.ssh/id_rsa@$remote_host
#generate key without passphrase
ssh-keygen -b 4096 -P "" -C $local_user@local_host-$local_date -f $local_filename
#add reference to generated key to local configuration
printf '%s\n' "Host $remote_host" "IdentityFile $local_filename" >> ~/.ssh/config
#copy key to remote host
sshpass -p $remote_pass ssh-copy-id $remote_user@$remote_host
#disable password for root on remote
ssh $remote_user@$remote_host "cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak && sed -i '0,/RE/s/PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config"
We just run this script soon as the OS is reinstalled and we’re substantially safer. As a Deb8 install, quickly pulling down fail2ban and PortSentry makes things quite a lot tighter.
In another post, we’ll visit the 2017 version of making a DIY script to batten the hatches using a variety of publicly provided blocklists.
Download here:
ssh_quick_fix.sh
Tags: CLI, Fail2ban, IPset, security, SSH
Permalink: 20170102.securing.a.new.server
Sun, 13 Jul 2014
Seems I’ve always just a few more things going on than I can comfortably handle. One of those is an innocent little server holding the beginnings of a new project.
If you expose a server to the Internet, very quickly your ports are getting scanned and tested. If you’ve an SSH server, there are going to be attempts to login as ‘root’ which is why it is ubiquitously advised that you disable root login. Also why many advise against allowing passwords at all.
We could talk for days about improvements; it’s usually not difficult to introduce some form of two-factor authentication (2FA) for sensitive points of entry such as SSH. You can install monitoring software like Logwatch which can summarize important points from your logs, such as: who has logged via SSH, how many times root was used, etc.
DenyHosts and
Fail2ban are very great ways to secure things, according to your needs.
DenyHosts works primarily with SSH and asks very little from you in way of configuration, especially if you’re using a package manager to install a version that is configured for the distribution on which you’re working. If you’re installing from source you may need to find where are your SSH logs (e.g., /var/log/secure, /var/log/auth.log). It’s extremely easy to set up DenyHosts to synchronize so that you’re automatically blocking widely-known offenders whether or not they’re after your server.
In contrast, Fail2ban is going to take more work to get set up. However, it is extremely configurable and works with any log file you point it toward which means that it can watch anything (e.g., FTP, web traffic, mail traffic). You define your own jails which means you can ban problematic IP addresses according to preference. Ban bad HTTP attempts from HTTP only or stick their noses in the virtual corner and don’t accept any traffic from them until they’ve served their time-out by completely disallowing their traffic. You can even use Fail2ban to scan its own logs, so repeating offenders can be locked out for longer.
Today we’re going to assume that you’ve a new server that shouldn’t be seeing any traffic except from you and any others involved in the project. In that case, you probably want to block traffic pretty aggressively. If you’ve physical access to the server (or the ability to work with staff at the datacenter) then it’s better to err in the direction of accidentally blocking good guys than trying to be overly fault-tolerant.
The server we’re working on today is a Debian Wheezy system. It has become a common misconception that Ubuntu and Debian are, intents and purposes, interchangeable. They’re similar in many respects and Ubuntu is great preparation for using Debian but they are not the same. The differences, I think, won’t matter for this exercise but I am unsure because this was written using Wheezy.
Several minutes after bringing my new server online, I started seeing noise in the logs. I was still getting set up and really didn’t want to stop and take protective measures but there’s no point in securing a server after its been compromised. The default Fail2ban configuration was too forgiving for my use. It was scanning for 10 minutes and banning for 10 minutes. Since only a few people should be accessing this server, there’s no reason for anyone to be trying a different password every 15 minutes (for hours).
I found a ‘close-enough’ script and modified it. Here, we’ll deal with a simplified version.
First, lets create a name for these ne’er-do-wells in iptables:
iptables -N bad_traffic
For this one, we’ll use Perl. We’ll look at our Apache log files to find people sniffing ‘round and we’ll block their traffic. Specifically, we’re going to check Apache’s ‘error.log’ for the phrases ‘File does not exist’ and ‘client denied by server configuration’ and block people causing those errors. This would be excessive for servers intended to serve the general populace. For a personal project, it works just fine as a ‘DO NOT DISTURB’ sign.
#!/usr/bin/env perl
use strict;
use POSIX qw(strftime);
my $log = ($ARGV[0] ? $ARGV[0] : "/var/log/apache2/error.log");
my $chain = ($ARGV[1] ? $ARGV[1] : "bad_traffic");
my @bad = `grep -iE 'File does not exist|client denied by server configuration' $log |cut -f8 -d" " | sed 's/]//' | sort -u`;
my @ablk = `/sbin/iptables -S $chain|grep DROP|awk '{print $4}'|cut -d"/" -f1`;
foreach my $ip (@bad) {
if (!grep $_ eq $ip, @ablk) {
chomp $ip;
`/sbin/iptables -A $chain -s $ip -j DROP`;
print strftime("%b %d %T",localtime(time))." badht: blocked bad HTTP traffic from: $ip\n";
}
}
That gives us some great, utterly unforgiving, blockage. Looking at the IP addresses attempting to pry, I noticed that most of them were on at least one of the popular block-lists.
So let’s make use of some of those block-lists! I found a program intended to apply those lists locally but, of course, it didn’t work for me. Here’s a similar program; this one will use ipset for managing the block-list though only minor changes would be needed to use iptables as above:
#!/bin/bash
IP_TMP=ip.tmp
IP_BLACKLIST_TMP=ip-blacklist.tmp
IP_BLACKLIST=ip-blacklist.conf
WIZ_LISTS="chinese nigerian russian lacnic exploited-servers"
BLACKLISTS=(
"http://danger.rulez.sk/projects/bruteforceblocker/blist.php" # BruteForceBlocker IP List
"http://rules.emergingthreats.net/blockrules/compromised-ips.txt" # Emerging Threats - Compromised IPs
"http://www.spamhaus.org/drop/drop.txt" # Spamhaus Don't Route Or Peer List (DROP)
"http://www.spamhaus.org/drop/edrop.txt" # Spamhaus Don't Route Or Peer List (DROP) Extended
"http://cinsscore.com/list/ci-badguys.txt" # C.I. Army Malicious IP List
"http://www.openbl.org/lists/base.txt" # OpenBL.org 90 day List
"http://www.autoshun.org/files/shunlist.csv" # Autoshun Shun List
"http://lists.blocklist.de/lists/all.txt" # blocklist.de attackers
)
for address in "${BLACKLISTS[@]}"
do
echo -e "\nFetching $address\n"
curl "$address" >> $IP_TMP
done
for list in $WIZ_LISTS
do
wget "http://www.wizcrafts.net/$list-iptables-blocklist.html" -O - >> $IP_TMP
done
wget 'http://wget-mirrors.uceprotect.net/rbldnsd-all/dnsbl-3.uceprotect.net.gz' -O - | gunzip | tee -a $IP_TMP
grep -o '^[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}[/][0-9]\{1,3\}' $IP_TMP | tee -a $IP_BLACKLIST_TMP
grep -o '^[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}[^/]' $IP_TMP | tee -a $IP_BLACKLIST_TMP
sed -i 's/\t//g' $IP_BLACKLIST_TMP
sort -u $IP_BLACKLIST_TMP | tee $IP_BLACKLIST
rm $IP_TMP
rm $IP_BLACKLIST_TMP
wc -l $IP_BLACKLIST
if hash ipset 2>/dev/null
then
ipset flush bloxlist
while IFS= read -r ip
do
ipset add bloxlist $ip
done < $IP_BLACKLIST
else
echo -e '\nipset not found\n'
echo -e "\nYour bloxlist file is: $IP_BLACKLIST\n"
fi
Download here:
bad_traffic.pl
bloxlist.sh
Tags: Apache, Bash, Blacklist, Blocklist, Debian, DenyHosts, Fail2ban, ipset, iptables, Logwatch, Perl
Permalink: 20140713.simple.protection.with.iptables.ipset.and.blacklilsts
Tue, 18 Mar 2014
I was reading about the vulnerability in the ‘random’ number generator in iOS 7 (http://threatpost.com/weak-random-number-generator-threatens-ios-7-kernel-exploit-mitigations/104757) and thought I would share a method that I’ve used. Though, I certainly was not on any version of iOS but, at least, I can help in GNU/Linux and BSDs.
What if we want to get some random numbers or strings? We need a salt or something and, in the interest of best practices, we’re trying to restrain the trust given to any single participant in the chain. That is, we do not want to generate the random data from the server that we are on. Let’s make it somewhere else!
YEAH! I know, right? Where to get a random data can be a headache-inducing challenge. Alas! The noble people at Random.org are using atmospheric noise to provide randomness for us regular folk!
Let’s head over and ask for some randomness:
https://www.random.org/strings/?num=1&len=16&digits=on&loweralpha=on&unique=on&format=html&rnd=new
Great! Still feels a little plain. What if big brother saw the string coming over and knew what I was doing? I mean, aside from the fact that I could easily wrap the string with other data of my choosing.
Random.org is really generous with the random data, so why don’t we take advantage of that? Instead of asking for a single string, let’s ask for several. Nobunny will know which one I picked, except for me!
We’ll do that by increasing the 'num'
part of the request that we’re sending:
https://www.random.org/strings/?num=25&len=16&digits=on&loweralpha=on&unique=on&format=html&rnd=new
They were also thinking of CLI geeks like us with the 'format'
variable. We set that to 'plain'
and all the fancy formatting for human eyes is dropped and we get a simple list that we don’t need to use any clever tricks to parse!
curl -s 'https://www.random.org/strings/?num=25&len=16&digits=on&loweralpha=on&unique=on&format=plain&rnd=new'
Now we’ve got 25 strings at the CLI. We can pick one to copy and paste. But what if we don’t want to do the picking? Well, we can use the local system to do that. We’ll have it pick a number between 1 and 25. It’s zero-indexed, so we’re going to increment by 1 so that our number is really between 1 and 25.
echo $(( (RANDOM % 25)+1 ))
Next, we’ll send our list to head
with our random number. Head is going to give us the first X lines of what we send to it. So if our random number is 17, then it will give us the first 17 lines from Random.org
curl -s 'https://www.random.org/strings/?num=25&len=16&digits=on&loweralpha=on&unique=on&format=plain&rnd=new' | head -$(( (RANDOM % 25)+1 ))
After that, we’ll use the friend of head
: tail
. We’ll tell tail
that we want only the last record of the list, of psuedo-random length, of strings that are random.
curl -s 'https://www.random.org/strings/?num=25&len=16&digits=on&loweralpha=on&unique=on&format=plain&rnd=new' | head -$(( (RANDOM % 25)+1 )) | tail -1
And we can walk away feeling pretty good about having gotten some properly random data for our needs!
Tags: Random, Seed
Permalink: 20140318.random.data
Thu, 06 Jun 2013
Recently, an author I admire and time-honored spinner of the Interwebs, Tony Lawrence emphasized the value of using man pagesmanual pagesDocumentation available from the command line.
> man ls as a sanity check before getting carried away with powerful commands. I didn’t know about this one but he has written about a situation in which killall
could produce some shocking, and potentially quite unpleasant, results.
Personally, I often quickly check man pages to be certain that I am using the correct flags or, as in the above case, anticipating results that bear some resemblance to what is actually likely to happen. Yet, it seems many people flock toward SERPSearch Engine Results Page A tasteful replacement for mentioning any particular search-engine by name.
Also useful as a verb:
“I dunno. You’ll have to SERP it.”s for this information.
Perhaps the most compelling reason to head for the web is leaving the cursor amid the line you’re working on, without disturbing the command. SERPing the command however, could easily lead you to information about a variant that is more common than the one available to you. More importantly, the information retrieved from the search engine is almost certainly written by someone who did read the man page — and may even come with the admonishment that you RTFMRead The F#!$!*#’n Manual as a testament to the importance of developing this habit.
This can be made easier with just a few CLI shortcuts.
<CTRL+u>
to cut what you have typed so far and <CTRL+y>
to paste it back.
That is, you press <CTRL+u>
and the line will be cleared, so you can then type man {command}
and read the documentation. Don’t hesitate to jot quick notes of which flags you intend to use, if needed. Then exit the man page, press <CTRL+y>
and finish typing right where you left off.
This is another good use for screen
or tmux
but let’s face it. There are times when you don’t want the overhead of opening another window for a quick look-up and even instances when these tools aren’t available.
A few other tips to make life easier when building complex commands:
Use the command fc
to open up an editor in which you can build your complex command and, optionally, even save it as a shell script for future reuse.
Repeat the last word from the previous command (often a filename) with <ALT+.>
or use an item from the last command by position, in reverse order:
> ls -lahtr *archive*
<ALT+1+.>
: *archive*
<ALT+2+.>
: -lahtr
<ALT+3+.>
: ls
You can also use Word Designators to use items from history, such as adding sudo
to the last command typed by:
sudo !!
This allows for tricks like replacing bits of a previous command:
!:s/misspelled/corrected/
Lastly, if you need a command that was typed earlier, you can search history by pressing <CTRL+r>
and start typing an identifying portion of the command.
(Note: I have used these in Zsh and Bash, specifically. They can, however, be missing or overwritten — if a feature you want isn’t working, you can bind keys in a configuration file. Don’t just write it off, once you’ve solved the problem it will never again be an intimidating one.)
Happy hacking!
Tags: Bash, CLI, fc, GNU-Screen, man, shortcuts, tmux, word-designators, Zsh
Permalink: 20130606.managing.to.use.man.pages
Thu, 30 May 2013
In this writer’s opinion, it is vitally important that we take reasonable measures now to help insure anonymity, lest we create a situation where privacy no longer exists, and the simple want of, becomes suspicious.
Here’s how to configure your browser to automatically use a search engine that respects your privacy.
Chrome:
- Click Settings.
- Click “Set pages” in the “On startup” section.
- Enter
https://ixquick.com/eng/
in the “Add a new page” text field.
- Click OK.
- Click “Manage search engines…”
- At the bottom of the “Search Engines” dialog, click in the “Add a new search engine” field.
- Enter
ixquick
ixquick.com
https://ixquick.com/do/search?lui=english&language=english&cat=web&query=%s
- Click “Make Default”.
- Click “Done”.
Firefox:
- Click the Tools Menu.
- Click Options.
- Click the General tab.
- In “When Firefox Starts” dropdown, select “Show my home page”.
- Enter
https://ixquick.com/eng/
in the “Home Page” text field.
- Click one of the English options here.
- Check box for “Start using it right away.”
- Click “Add”.
Opera:
- Click “Manage Search Engines
- Click “Add”
- Enter
Name: ixquick
Keyword: x
Address: https://ixquick.com/do/search?lui=english&language=english&cat=web&query=%s
- Check “Use as default search engine”
- Click “OK”
Internet Explorer:
_ ___ _ __ ___ _ _____ ___
| | / _ \| |\ \ / / | | |_ _|__ \
| | | | | | | \ \ /\ / /| | | | | | / /
| |__| |_| | |__\ V V / | |_| | | | |_|
|_____\___/|_____\_/\_/ \___/ |_| (_)
(This is not a good strategy for privacy.)
Congratulations!
\o/
You are now one step closer to not having every motion on the Internet recorded.
This is a relatively small measure, though. You can improve your resistance to prying eyes (e.g., browser fingerprinting) by using the Torbrowser Bundle, or even better, Tails, and routing your web usage through Tor, i2p, or FreeNet.
If you would like more on subjects like anonymyzing, privacy and security then drop me a line via email or Bitmessage me: BM-2D9tDkYEJSTnEkGDKf7xYA5rUj2ihETxVR
Tags: \o/, anonymous, Chrome, Firefox, FreeNet, i2p, ixquick, LOLWUT, Opera, privacy, security, Tails, Tor, TorBrowser, TorBrowser-Bundle
Permalink: 20130530.hey.you.get.offa.my.data
Thu, 23 May 2013
Working on remote servers, some tools are practically ubiquitous — while others are harder to come by. Even if you’ve the authority to install your preferred tools on every server you visit, it’s not always something you want to do. If you’ve hopped on to a friend’s server just to troubleshoot a problem, there is little reason to install tools that your friend is not in the habit of using. Some servers, for security reasons, are very tightly locked down to include only a core set of tools, to complicate the job of any prying intruders. Or perhaps it is a machine that you normally use through a graphical interface but on this occasion you need to work from the CLI.
These are very compelling reasons to get comfortable, at the very least, with tools like Vim
, mail
, grep
and sed
. Eventually, you’re likely to encounter a situation where only the classic tools are available. If you aren’t competent with those tools, you’ll end up facing the obstacle of how to get files from the server to your local environment where you can work and, subsequently, how to get the files back when you’re done. In a secured environment, this may not be possible without violating protocols.
Let’s take a look at how we can build a makeshift system monitor using some common tools. This particular configuration is for a server running PHP, MySQL and has the tools Htop and mytop installed. These can easily be replaced with top
and a small script to SHOW FULL PROCESSLIST
, if needed. The point here is illustrative, to provide a template to be modified according to each specific environment.
(Note: I generally prefer tmux to Gnu Screen but screen
is the tool more likely to be already installed, so we’ll use it for this example.)
We’re going to make a set of windows, by a configuration file, to help us keep tabs on what is happening in this system. In so doing, we’ll be using the well-known tools less
and watch
. More specifically, less +F
which tells less
to “scroll forward”. Other words, less
will continue to read the file making sure any new lines are added to the display. You can exit this mode with CTRL+c
, search the file (/), quit(q) or get back into scroll-forward mode with another uppercase F.
Using watch
, we’ll include the “-d” flag which tells watch we want to highlight any changes (differences).
We will create a configuration file for screen
by typing:
> vim monitor.screenrc
In the file, paste the following:
# Screen setup for system monitoring
# screen -c monitor.screenrc
hardstatus alwayslastline
hardstatus string '%{= kG}[ %{G}%H %{g}][%= %{=kw}%?%-Lw%?%{r}(%{W}%n*%f%t%?(%u)%?%{r})%{w}%?%+Lw%?%?%= %{g}][%{B}%Y-%m-%d %{W}%c %{g}]'
screen -t htop 0 htop
screen -t mem 1 watch -d "free -t -m"
screen -t mpstat 2 watch -d "mpstat -A"
screen -t iostat 3 watch -d "iostat"
screen -t w 4 watch -d "w"
screen -t messages 5 less +F /var/log/messages
screen -t warn 6 less +F /var/log/warn
screen -t database 7 less +F /srv/www/log/db_error
screen -t mytop 8 mytop
screen -t php 9 less +F /srv/www/log/php_error
(Note: -t
sets the title, then the window number, followed by the command running in that window)
Save the file (:wq
) or, if you’d prefer, you can grab a copy by right-clicking and saving this file.
Then we will execute screen
using this configuration, as noted in the comment:
> screen -c monitor.screenrc
Then you can switch between windows using CTRL+a, n
(next) or CTRL+a, p
(previous).
I use this technique on my own computers, running in a TTY different from the one used by X. If the graphical interface should get flaky, I can simply switch to that TTY (e.g., CTRL+ALT+F5
) to see what things are going on — and take corrective actions, if needed.
Tags: GNU-Screen, Htop, iostat, mem, mpstat, MySQL, mytop, PHP, system-monitor, watch, who
Permalink: 20130523.gnu.screen.system.monitor
Mon, 20 May 2013
I have finished (more-or-less) making a demo for the Xdebug togglin’ add-on/extension that I’ve developed.
One hundred percent of the feedback about this project has been from Chrome users. Therefore, the Chrome extension has advanced with the new features (v2.0), allowing selective en/dis-ableing portions of Xdebug’s output. That is you can set Xdebug to firehose mode (spitting out everything) and then squelch anything not immediately needed at the browser layer. The other information remains present, hidden in the background, available if you decide that you need to have a look.
The Firefox version is still at v1.2 but will be brought up to speed as time permits.
If you want that firehose mode for Xdebug, here’s a sample of some settings for your configuration ‘.ini’ file.
The demo is here.
Tags: add-on, Chrome, debugging, extension, Firefox, PHP, plugin, Xdebug
Permalink: 20130520.debugging.php.with.xdebug
Tue, 07 May 2013
Thanks for visiting my little spot on the web. This is a Blosxom ‘blog which, for those who don’t know, is a CGI written in Perl using the file-system (rather than a database).
To the CLI-addicted, this is an awesome little product. Accepting, of course, that you’re going to get under the hood if you’re going to make it the product you want. After some modules and hacking, I’m pleased with the result.
My posts are just text files, meaning I start a new one like:
vim ~posts/`date +%Y%m%d`.brief.subject.txt
Note: the back-ticks (`) tell the system that you want to execute the command between ticks, and dynamically insert its output into the command.
In this case, the command date
with these parameters:
- (+) we’re going to specify a format
- (%Y) four-digit year
- (%m) two-digit month
- (%d) two-digit day
That means the command above will use Vim to edit a text file named ‘20130507.brief.subject.txt’ in the directory I have assigned to the hash
of ‘posts’.
(using hash
this way is a function of Zsh that I’ll cover in another post)
In my CLI-oriented ‘blog, I can sprinkle in my own HTML or use common notation like wrapping a word in underscores to have it
underlined, forward-slashes for
italics and asterisks for
bold.
Toss in a line that identifies tags and, since Perl is the beast of Regex, we pick up the tags and make them links, meta-tags, etc.
Things here are likely to change a lot at first, while I twiddle with CSS and hack away at making a Blosxom that perfectly fits my tastes — so don’t be too alarmed if you visit and things look a tad wonky. It just means that I’m tinkering.
Once the saw-horses have been tucked away, I’m going to take the various notes I’ve made during my years in IT and write them out, in a very simple breakdown, aimed at sharing these with people who know little about how to negotiate the command line. The assumption here is that you have an interest in *nix/BSD. If you’ve that and the CLI is not a major part of your computing experience, it probably will be at some point. If you’re working on systems remotely, graphical interfaces often just impede you.
Once you’ve started working on remote machines, the rest is inevitable. You can either remember how to do everything two ways, through a graphical interface and CLI — or just start using the CLI for everything.
So let’s take a little journey through the kinds of things that make me love the CLI.
Tags: blogitecture, Blosxom, BSD, CGI, CLI, firstpost, hash, Perl, Vim, Zsh
Permalink: 20130507.greetings