c0d3 :: j0rg3

A collection of tips, tricks and snips. A proud Blosxom weblog. All code. No cruft.

Thu, 06 Jun 2013

Managing to use man pages through simple CLI tips

Recently, an author I admire and time-honored spinner of the Interwebs, Tony Lawrence emphasized the value of using man pagesmanual pagesDocumentation available from the command line.
> man ls
as a sanity check before getting carried away with powerful commands. I didn’t know about this one but he has written about a situation in which killall could produce some shocking, and potentially quite unpleasant, results.

Personally, I often quickly check man pages to be certain that I am using the correct flags or, as in the above case, anticipating results that bear some resemblance to what is actually likely to happen. Yet, it seems many people flock toward SERPSearch Engine Results Page A tasteful replacement for mentioning any particular search-engine by name.
Also useful as a verb:
I dunno. You’ll have to SERP it.
s for this information.

Perhaps the most compelling reason to head for the web is leaving the cursor amid the line you’re working on, without disturbing the command. SERPing the command however, could easily lead you to information about a variant that is more common than the one available to you. More importantly, the information retrieved from the search engine is almost certainly written by someone who did read the man page — and may even come with the admonishment that you RTFMRead The F#!$!*#’n Manual as a testament to the importance of developing this habit.

This can be made easier with just a few CLI shortcuts.

<CTRL+u> to cut what you have typed so far and <CTRL+y> to paste it back.

That is, you press <CTRL+u> and the line will be cleared, so you can then type man {command} and read the documentation. Don’t hesitate to jot quick notes of which flags you intend to use, if needed. Then exit the man page, press <CTRL+y> and finish typing right where you left off.

This is another good use for screen or tmux but let’s face it. There are times when you don’t want the overhead of opening another window for a quick look-up and even instances when these tools aren’t available.

A few other tips to make life easier when building complex commands:

Use the command fc to open up an editor in which you can build your complex command and, optionally, even save it as a shell script for future reuse.

Repeat the last word from the previous command (often a filename) with <ALT+.> or use an item from the last command by position, in reverse order:
> ls -lahtr *archive*
<ALT+1+.> : *archive*
<ALT+2+.> : -lahtr
<ALT+3+.> : ls

You can also use Word Designators to use items from history, such as adding sudo to the last command typed by:
sudo !!

This allows for tricks like replacing bits of a previous command:
!:s/misspelled/corrected/

Lastly, if you need a command that was typed earlier, you can search history by pressing <CTRL+r> and start typing an identifying portion of the command.

(Note: I have used these in Zsh and Bash, specifically. They can, however, be missing or overwritten — if a feature you want isn’t working, you can bind keys in a configuration file. Don’t just write it off, once you’ve solved the problem it will never again be an intimidating one.)

Happy hacking!


Tags: , , , , , , , ,
Permalink: 20130606.managing.to.use.man.pages

Thu, 23 May 2013

GNU Screen: Roll your own system monitor

Working on remote servers, some tools are practically ubiquitous — while others are harder to come by. Even if you’ve the authority to install your preferred tools on every server you visit, it’s not always something you want to do. If you’ve hopped on to a friend’s server just to troubleshoot a problem, there is little reason to install tools that your friend is not in the habit of using. Some servers, for security reasons, are very tightly locked down to include only a core set of tools, to complicate the job of any prying intruders. Or perhaps it is a machine that you normally use through a graphical interface but on this occasion you need to work from the CLI.

These are very compelling reasons to get comfortable, at the very least, with tools like Vim, mail, grep and sed. Eventually, you’re likely to encounter a situation where only the classic tools are available. If you aren’t competent with those tools, you’ll end up facing the obstacle of how to get files from the server to your local environment where you can work and, subsequently, how to get the files back when you’re done. In a secured environment, this may not be possible without violating protocols.

Let’s take a look at how we can build a makeshift system monitor using some common tools. This particular configuration is for a server running PHP, MySQL and has the tools Htop and mytop installed. These can easily be replaced with top and a small script to SHOW FULL PROCESSLIST, if needed. The point here is illustrative, to provide a template to be modified according to each specific environment.

(Note: I generally prefer tmux to Gnu Screen but screen is the tool more likely to be already installed, so we’ll use it for this example.)

We’re going to make a set of windows, by a configuration file, to help us keep tabs on what is happening in this system. In so doing, we’ll be using the well-known tools less and watch. More specifically, less +F which tells less to “scroll forward”. Other words, less will continue to read the file making sure any new lines are added to the display. You can exit this mode with CTRL+c, search the file (/), quit(q) or get back into scroll-forward mode with another uppercase F.

Using watch, we’ll include the “-d” flag which tells watch we want to highlight any changes (differences).

We will create a configuration file for screen by typing:

> vim monitor.screenrc

In the file, paste the following:

# Screen setup for system monitoring
# screen -c monitor.screenrc
hardstatus alwayslastline
hardstatus string '%{= kG}[ %{G}%H %{g}][%= %{=kw}%?%-Lw%?%{r}(%{W}%n*%f%t%?(%u)%?%{r})%{w}%?%+Lw%?%?%= %{g}][%{B}%Y-%m-%d %{W}%c %{g}]'

screen -t htop 0 htop
screen -t mem 1 watch -d "free -t -m"
screen -t mpstat 2 watch -d "mpstat -A"
screen -t iostat 3 watch -d "iostat"
screen -t w 4 watch -d "w"
screen -t messages 5 less +F /var/log/messages
screen -t warn 6 less +F /var/log/warn
screen -t database 7 less +F /srv/www/log/db_error
screen -t mytop 8 mytop
screen -t php 9 less +F /srv/www/log/php_error

(Note: -t sets the title, then the window number, followed by the command running in that window)

Save the file (:wq) or, if you’d prefer, you can grab a copy by right-clicking and saving this file.

Then we will execute screen using this configuration, as noted in the comment:

> screen -c monitor.screenrc

Then you can switch between windows using CTRL+a, n (next) or CTRL+a, p (previous).

I use this technique on my own computers, running in a TTY different from the one used by X. If the graphical interface should get flaky, I can simply switch to that TTY (e.g., CTRL+ALT+F5) to see what things are going on — and take corrective actions, if needed.


Tags: , , , , , , , , , ,
Permalink: 20130523.gnu.screen.system.monitor

Wed, 15 May 2013

Git: an untracked mess?

There may be times when you find your Git repository burdened with scads of untracked files left aside while twiddling, testing bug patches, or what-have-youse.

For the especially scatter-brained among us, these things can go unchecked until a day when the useful bits of a git status scroll off the screen due to utterly unimportant stuff. Well, hopefully unimportant.

But we’d better not just cleave away everything that we haven’t checked in. You wonder:
What if there’s something important in one of those files?

You are so right!

Let’s fix this!

Firstly, we want a solution that’s reproducible. Only want to invent this wheel once, right?

Let’s begin with the play-by-play:

Git, we want a list of what isn’t tracked: git ls-files -o --exclude-standard -z

We’ll back these files up in our home directory (~), using CPIO but we don’t want a poorly-named directory or finding anything will become its own obstacle. So we’ll take use the current date (date +%Y-%m-%d), directory (pwd) and branch we’re using (git branch) and we’ll twist all of it into a meaningful, but appropriate, directory name using sed. git ls-files -o --exclude-standard -z | cpio -pmdu ~/untracked-git-backup-`date +%Y-%m-%d`.`pwd | sed 's,^\(.*/\)\?\([^/]*\),\2,'`.`git branch | grep "*" | sed "s/* //"`/

Then Tell Git to remove the untracked files and directories: git clean -d -f

Ahhhh… Much better. Is there anything left out? Perhaps. What if we decide that moving these files away was a mistake? The kind of mistake that breaks something. If we realize right away, it’s easily-enough undone. But what if we break something and don’t notice for a week or two? It’d probably be best if we had an automated script to put things back the way they were. Let’s do that.

Simple enough. We’ll just take the opposite commands and echo them into a script to be used in case of emergency.

Create the restore script (restore.sh), to excuse faulty memory: echo "(cd ~/untracked-git-backup-`date +%Y-%m-%d`.`pwd | sed 's,^\(.*/\)\?\([^/]*\),\2,'`.`git branch | grep "*" | sed "s/* //"`/; find . -type f \( ! -iname 'restore.sh' \) | cpio -pdm `pwd`)" > ~/untracked-git-backup-`date +%Y-%m-%d`.`pwd | sed 's,^\(.*/\)\?\([^/]*\),\2,'`.`git branch | grep "*" | sed "s/* //"`/restore.sh

Make the restore script executable: chmod u+x ~/untracked-git-backup-`date +%Y-%m-%d`.`pwd | sed 's,^\(.*/\)\?\([^/]*\),\2,'`.`git branch | grep "*" | sed "s/* //"`/restore.sh

Lastly, the magic, compressed into one line that will stop if any command does not report success: a='untracked-git-backup-'`date +%Y-%m-%d`.`pwd | sed 's,^\(.*/\)\?\([^/]*\),\2,'`.`git branch | grep "*" | sed "s/* //"`; git ls-files -o --exclude-standard -z | cpio -pmdu ~/$a/ && git clean -d -f && echo "(cd ~/$a/; find . -type f \( ! -iname 'restore.sh' \) | cpio -pdm `pwd`)" > ~/$a/restore.sh && chmod +x ~/$a/restore.sh; unset a


Tags: , , , ,
Permalink: 20130515.git.untracked.mess

Mon, 13 May 2013

Zsh and hash

Documentation for this one seems a bit hard to come by but it is one of the things I love about Zsh.

I’ve seen many .bashrc files that have things like:
alias www='cd /var/www'
alias music='cd /home/j0rg3/music'

And that’s a perfectly sensible way to make life a little easier, especially if the paths are very long.

In Zsh, however, we can use the hash command and the shortcut we get from it works fully as the path. Other words, using the version above, if we want to edit ‘index.html’ in the ‘www’ directory, we would have to issue the shortcut to get there and then edit the file, in two steps:
> www
> vim index.html

The improved version in .zshrc would look like:
hash www=/var/www
hash -d www=/var/www

Then, at any time, you can use tilde (~) and your shortcut in place of path.
> vim ~www/index.html

Even better, it integrates with Zsh’s robust completions so you can, for example, type cd ~www/ and then use the tab key to cycle through subdirectories and files.

On this system, I’m using something like this:
(.zshrc)
hash posts=/home/j0rg3/weblog/posts
hash -d posts=/home/j0rg3/weblog/posts

Then we can make a function to create a new post, to paste into .zshrc. Since we want to be able to edit and save, without partial posts becoming visible, while we are working, we’ll use an extra .tmp extension at the end:
post() { vim ~posts/`date +%Y-%m`/`date +%Y%m%d`.$1.txt.tmp }

[ In-line date command unfamiliar? See earlier explanation ]

But, surely there is going to be a point when we need to save a post and finish it later. For now, let’s assume that only a single post will be in limbo at any time. We definitely don’t want to have to remember the exact name of the post — and we don’t want to have hunt it down every time.

We can make those things easier like this:
alias resume="vim `find ~posts/ -name '*.txt.tmp'`"

Now, we can just enter resume and the system will go find the post we were working on and open it up for us to finish. The file will need the extension renamed from .txt.tmp to only .txt to publish the post but, for the sake of brevity, we’ll think about that (and having multiple posts in editing) on another day.


Tags: , , , ,
Permalink: 20130513.zsh.and.hash

Wed, 08 May 2013

Deleting backup files left behind by Vim

It’s generally a great idea to have Vim keep backups. Once in awhile, they can really save your bacon.

The other side of that coin, though, is that they can get left behind here and there, eventually causing aggravation.

Here’s a snippet to find and eliminate those files from the current directory down:

find ./ -name '*~' -exec rm '{}' \; -print -or -name ".*~" -exec rm {} \; -print
This uses find from the current directory down (./) to execute an rm statement on all files with an extension ending in tilde (~)
Alternatively, you could just store your backups elsewhere. In Vim, use :help backupdir for more information.


Tags: , , , ,
Permalink: 20130508.delete.vim.backups