c0d3 :: j0rg3

A collection of tips, tricks and snips. A proud Blosxom weblog. All code. No cruft.

Fri, 17 Feb 2017

The making of a Docker: Part I - Bitmessage GUI with SSH X forwarding

Lately, I’ve been doing a lot of work from a laptop running Kali. Engaged in pursuit of a new job, I’m brushing up on some old tools and skills, exploring some bits that have changed.

My primary desktop rig is currently running Arch because I love the fine grain control and the aggressive releases. Over the years, I’ve Gentoo’d and Slacked, Crunchbanged, BSD’d, Solarised, et cet. And I’ve a fondness for all of them, especially the security-minded focus of OpenBSD. But, these days we’re usually on Arch or Kali. Initially, I went with Black Arch on the laptop but I felt the things and ways I was fixing things were too specific to my situation to be good material for posts.

Anyway, I wanted to get Bitmessage running, corresponding to another post I have in drafts. On Kali, it wasn’t going well so I put it on the Arch box and just ran it over the network. A reasonable solution if you’re in my house but also the sort of solution that will keep a hacker up at night.

If you’re lucky, there’s someone maintaining a package for the piece of software that you want to run. However, that’s often not the case.

If I correctly recall, to “fix” the problem with Bitmessage on Kali would’ve required the manual installation an older version of libraries that were already present. Those libraries should, in fact, be all ebony and ivory, living together in harmony. However, I just didn’t love the idea of that solution. I wanted to find an approach that would be useful on a broader scale.

Enter containerization/virtualization!

Wanting the lightest solution, I quickly went to Docker and realized something. I have not before built a Docker container for a GUI application. And Bitmessage’s CLI/daemon mode doesn’t provide the fluid UX that I wanted. Well, the easy way to get a GUI out of a Docker container is to forward DISPLAY as an evironment variable (i.e., docker run -e DISPLAY=$DISPLAY). Splendid!

Except that it doesn’t work on current Kali which is using QT4. There’s a when graphical apps are run as root and though it is fixed in QT5, we are using current Kali. And that means we are, by default, uid 0 and QT4.

I saw a bunch of workarounds that seemed to have spotty (at best) rates of success including seting QT’s graphics system to Native and giving Xorg over to root. They, mostly, seemed to be cargo cult solutions.

What made the most sense to my (generally questionable) mind was to use X forwarding. Since I had already been running Bitmessage over X forwarding from my Arch box, I knew it should work just the same.

To be completely truthful, the first pass I took at this was with Vagrant mostly because it’s SO easy. Bring up your Vagrant Box and then:
vagrant ssh -- -X
Viola!

Having proof of concept, I wanted a Docker container. The reason for this is practical. Vagrant, while completely awesome, has substantially more overhead than Docker by virtualizing the kernel. We don’t want a separate kernel running for each application. Therefore Docker is the better choice for this project.

Also, we want this whole thing to be seemless. We want to run the command bitmessage and it should fire up with minimal awkwardness and hopefully no extra steps. That is we do not want to run the Docker container then SSH into it and execute Bitmessage as individual steps. Even though that’s going to be how we begin.

The Bitmessage wiki accurately describes how to install the software so we’ll focus on the SSH setup. Though when we build the Dockerfile we will need to add SSH to the list from the wiki.

We’re going to want the container to start so that the SSH daemon is ready. Until then we can’t SSH (with X forwarding) into the container. Then we’ll want to use SSH to kick off the Bitmessage application, drawing the graphical interface using our host system’s X11.

We’re going to take advantage of Docker’s -v --volume option which allows us to specify a directory on our host system to be mounted inside our container. Using this feature, we’ll generate our SSH keys on the host and make them automatically available inside the container. We’ll tuck the keys inside the directory that Bitmessage uses for storing its configuration and data. That way Bitmessage’s configuration and stored messages can be persistent between runs — and all of your pieces are kept in a single place.

When we generate the container /etc/ssh/sshd_config is configured to allow root login without password only (i.e., using keys). So here’s how we’ll get this done:
mkdir -p ~/.config/PyBitmessage/keys #Ensure that our data directories exist
cd ~/.config/PyBitmessage/keys
ssh-keygen -b 4096 -P "" -C $"$(whoami)@$(hostname)-$(date -I)" -f docker-bitmessage-keys #Generate our SSH keys
ln -fs docker-bitmessage-keys.pub authorized_keys #for container to see pubkey

Build our container (sources available at Github and Docker) and we’ll make the script to handle Bitmessage to our preferences. #!/bin/bash
# filename: bitmessage
set -euxo pipefail

# open Docker container:
# port 8444 available, sharing local directories for SSH and Bitmessage data
# detatched, interactive, pseudo-tty (-dit)
# record container ID in $DID (Docker ID)
DID=$(docker run -p 8444:8444 -v ~/.config/PyBitmessage/:/root/.config/PyBitmessage -v ~/.config/PyBitmessage/keys/:/root/.ssh/ -dit j0rg3/bitmessage-gui bash)

# find IP address of new container, record in $DIP (Docker IP)
DIP=$(docker inspect $DID | grep IPAddress | cut -d '"' -f 4)

# pause for one second to allow container's SSHD to come online
sleep 1

# SSH into container and execute Bitmessage
ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -oIdentityFile=~/.config/PyBitmessage/keys/docker-bitmessage-keys -X $DIP ./PyBitmessage/src/bitmessagemain.py

# close container if Bitmessage is closed
docker kill $DID

Okay, let’s make it executable: chmod +x bitmessage

Put a link to it where it can be picked up system-wide: ln -fs ~/docker-bitmessage/bitmessage /usr/local/bin/bitmessage

There we have it! We now have a functional Bitmessage inside a Docker container. \o/

In a future post we’ll look at using eCryptfs to further protect our Bitmessage data stores.

  Project files: Github and Docker


Tags: , , , , , , , , , , ,
Permalink: 20170217.making.a.docker.bitmessage

Tue, 10 Jan 2017

[-] Auxiliary failed: Msf::OptionValidateError The following options failed to validate: RHOSTS.

Mucking about with a fresh copy of Kali brings to attention that it’s packaged with an Armitage that doesn’t correctly work.

I know what you’re thinking… Good. Type the commands into Msfconsole like a real man, y’uh lazy good-fer-naught! And, in practice, that was my immediate solution. But I can’t resist a good tinker when things are misbehaving.

I was anticipating that the problem would be thoroughly solved when I ixquicked it. That was partially correct. Surprised, however, when apt-get update && apt-get upgrade didn’t fix the issue. More surprised at the age of the issue. Most surprised that I could see lots of evidence that users have been plagued by this issue — but no clear work arounds were quickly found.

Guess what we’re doing today?

Okay. The issue is quite minor but just enough to be heartbreaking to the fledgling pentester trying to get a VM off the ground.

In brief, the owner of Armitage’s Github explains:

The MSF Scans feature in Armitage parses output from Metasploit’s portscan/tcp module and uses these results to build a list of targets it should run various Metasploit auxiliary modules against. A recent-ish update to the Metasploit Framework changed the format of the portscan/tcp module output. A patch to fix this issue just needs to account for the new format of the portscan/tcp module.

That is, a colon makes it into the input for the Msfconsole command to define RHOSTS. I.e.: set RHOSTS 172.16.223.150: - 172.16.223.150

An other kind coder tweaked the regex and submitted the patch and pull request, which was successfully incorporated into the project.

Sadly, things have stalled out there. So if this problem is crippling your rig, let’s fix it!

We just want a fresh copy of the project.
root@kali:~/armitage# git clone https://github.com/rsmudge/armitage

Cloning into ‘armitage’…
remote: Counting objects: 7564, done.
remote: Total 7564 (delta 0), reused 0 (delta 0), pack-reused 7564
Receiving objects: 100% (7564/7564), 47.12 MiB | 2.91 MiB/s, done.
Resolving deltas: 100% (5608/5608), done.

Kali is Debian-based and we’re going to need Apache Ant:
root@kali:~/armitage# apt-get install ant

Then, we’ll build our new fella:
root@kali:~/armitage# cd armitage
root@kali:~/armitage# ./package.sh

Buildfile: /root/test/armitage/build.xml

clean:

BUILD SUCCESSFUL
Total time: 0 seconds
Buildfile: /root/test/armitage/build.xml

init:
[mkdir] Created dir: /root/test/armitage/bin

compile:
[javac] Compiling 111 source files to /root/test/armitage/bin
[javac] depend attribute is not supported by the modern compiler
[javac] Note: /root/test/armitage/src/ui/MultiFrame.java uses or overrides a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.

BUILD SUCCESSFUL
Total time: 2 seconds
Buildfile: /root/test/armitage/build.xml

init:

compile:

jar:
[unzip] Expanding: /root/test/armitage/lib/sleep.jar into /root/test/armitage/bin
[unzip] Expanding: /root/test/armitage/lib/jgraphx.jar into /root/test/armitage/bin
[unzip] Expanding: /root/test/armitage/lib/msgpack-0.6.12-devel.jar into /root/test/armitage/bin
[unzip] Expanding: /root/test/armitage/lib/postgresql-9.1-901.jdbc4.jar into /root/test/armitage/bin
[unzip] Expanding: /root/test/armitage/lib/javassist-3.15.0-GA.jar into /root/test/armitage/bin
[copy] Copying 4 files to /root/test/armitage/bin/scripts-cortana
[jar] Building jar: /root/test/armitage/armitage.jar
[jar] Building jar: /root/test/armitage/cortana.jar

BUILD SUCCESSFUL
Total time: 1 second
armitage/
armitage/readme.txt
armitage/teamserver
armitage/cortana.jar
armitage/armitage.jar
armitage/armitage-logo.png
armitage/armitage
armitage/whatsnew.txt
adding: readme.txt (deflated 55%)
adding: armitage.exe (deflated 49%)
adding: cortana.jar (deflated 5%)
adding: armitage.jar (deflated 5%)
adding: whatsnew.txt (deflated 65%)
armitage/
armitage/readme.txt
armitage/teamserver
armitage/cortana.jar
armitage/armitage.jar
armitage/armitage-logo.png
armitage/armitage
armitage/whatsnew.txt
Archive: ../../armitage.zip
inflating: readme.txt
inflating: armitage.exe
inflating: cortana.jar
inflating: armitage.jar
inflating: whatsnew.txt

And here, best I can guess from messages read, is where a lot of people are running into trouble. We have successfully produced our new working copy of armitage. However, it is in our own local directory and will not be run if we just enter the command: armitage

Let’s review how to figure out what we want to do about that.

First, we want to verify what happens when we run the command armitage.
root@kali:~/armitage# which armitage

/usr/bin/armitage

Good! Let’s check and see what that does!
root@kali:~/armitage# head /usr/bin/armitage

#!/bin/sh

cd /usr/share/armitage/
exec ./armitage “$@”

Almost there! It’s running /usr/share/armitage/armitage with whatever variables we’ve passed in. We’ll check that out.
root@kali:~/armitage# head /usr/share/armitage/armitage

#!/bin/sh
java -XX:+AggressiveHeap -XX:+UseParallelGC -jar armitage.jar $@

We have enough information to assemble a solution.

I trust that the people behind Kali and Armitage will get this corrected so I don’t want to suggest a solution that would replace the armitage command and prevent an updated version from running later. So, let’s just make a temporary replacement?

root@kali:~/armitage# echo -e '#!/bin/sh\njava -XX:+AggressiveHeap -XX:+UseParallelGC -jar ~/armitage/armitage.jar $@' > /usr/bin/tmparmitage

Hereafter, we can use the command ‘tmparmitage’ (either CLI or ALT-F2) to run our fresh version until things catch up.

And, of course, to save you the time, weary hacker:

Download here:
    armitage_quick_fix.sh


Tags: , , , , , , ,
Permalink: 20170110.armitage.not.working.in.kali

Tue, 20 Dec 2016

Kicking the Crypto-tires

Some time ago I had begun work on my own Pastebin-type project with a few goals. Basically, I wanted to eat all the cakes — and have them too.

  • Both an online user interface and efficient CLI usage
  • Messages encrypted immediately such that database access does not provide one with the contents of the messages
  • Messages capable of self-destructing
  • Database schema that would allow rebuilding the user/message relationship, provided the same password but would not store those relationships
  • Also, JavaScript encryption to appeal to users who don’t know much about cryptography but would like to try
  • The project, honestly, was going swimmingly when derailed by the goings-on of life.

    One of the interesting components of the project was, of course, choosing crypto implementations. There are know shortcomings to handling it in JS but that’s still the most convenient for some users. Outside of the browser, server-side, you had all the same questions about which solution was best. Which protocol(s) should be available?

    Well, I’ve just learned about a project which I would have loved to have available back then. Project Wycheproof can help you test your crypto solutions against known problems and attacks. Featuring 80 tests probing at 40 known bugs, here’s a snip from the introduction:

    Project Wycheproof has tests for the most popular crypto algorithms, including

  • AES-EAXAES-GCM
  • AES-GCM
  • DH
  • DHIES
  • DSA
  • ECDH
  • ECDSA
  • ECIES
  • RSA
  • The tests detect whether a library is vulnerable to many attacks, including

  • Invalid curve attacks
  • Biased nonces in digital signature schemes
  • Of course, all Bleichenbacher’s attacks
  • And many more — we have over 80 test cases
  • Interesting stuff with exciting potential!


    Tags: , ,
    Permalink: 20161220.kicking.the.crypto.tires

    Mon, 17 Feb 2014

    Installing INN’s Project Largo in a Docker containter

    Prereqruisites: Docker, Git, SSHFS.

    Today we’re going to look at using Docker to create a WordPress installation with the Project Largo parent theme and a child theme stub for us to play with.

    Hart Hoover has established an image for getting a WordPress installation up and running using Docker. For whatever reason, it didn’t work for me out-of-box but we’re going to use his work to get started.

    Let’s make a place to work and move into that directory:
    cd ~
    mkdir project.largo.wordpress.docker
    cd project.largo.wordpress.docker

    We’ll clone the Docker/Wordpress project. For me, it couldn’t untar the latest WordPress. So we’ll download it outside the container, untar it and modify the Dockerfile to simply pull in a copy:
    git clone https://github.com/hhoover/docker-wordpress.git
    cd docker-wordpress/
    ME=$(whoami)
    wget http://wordpress.org/latest.tar.gz
    tar xvf latest.tar.gz
    sed -i 's/ADD http:\/\/wordpress.org\/latest.tar.gz \/wordpress.tar.gz/ADD \.\/wordpress \/wordpress/' Dockerfile
    sed -i '/RUN tar xvzf \/wordpress\.tar\.gz/d' Dockerfile

    Then, build the project which may take some time.
    sudo docker build -t $ME/wordpress .

    If you’ve not the images ready for Docker, the process should begin with something like:
    Step 0 : FROM boxcar/raring
    Pulling repository boxcar/raring
    32737f8072d0: Downloading [> ] 2.228 MB/149.7 MB 12m29s

    And end something like:
    Step 20 : CMD ["/bin/bash", "/start.sh"]
    ---> Running in db53e215e2fc
    ---> 3f3f6489c700
    Successfully built 3f3f6489c700

    Once the project is built, we will start it and forward ports from the container to the host system, so that the Docker container’s site can be accessed through port 8000 of the host system. So, if you want to see it from the computer that you’ve installed it on, you could go to ‘HTTP://127.0.0.1:8000’. Alternatively, if your host system is already running a webserver, we could use SSHFS to mount the container’s files within the web-space of the host system.

    In this example, however, we’ll just forward the ports and mount the project locally (using SSHFS) so we can easily edit the files perhaps using a graphical IDE such as NetBeans or Eclipse.

    Okay, time to start our Docker image and find its IP address (so we can mount its files):
    DID=$(docker run -p 8000:80 -d $ME/wordpress)
    DIP=$(docker inspect $DID | grep IPAddress | cut -d '"' -f 4)
    docker logs $DID| grep 'ssh user password:' --color

    Copy the SSH password and we will make a local directory to access the WordPress installation of our containter.
    cd ~
    mkdir largo.mount.from.docker.container
    sshfs user@$DIP:/var/www $HOME/largo.mount.from.docker.container
    cd largo.mount.from.docker.container
    PROJECT=$(pwd -P)

    Now, we can visit the WordPress installation and finish setting up. From the host machine, it should be ‘HTTP://127.0.0.1:8000’. There you can configure Title, Username, Password, et cet. and finish installing WordPress.

    Now, let’s get us some Largo! Since this is a test project, we’ll sacrifice security to make things easy. Our Docker WordPress site isn’t ready for us to easily install the Largo parent theme, so we’ll make the web directory writable by everybody. Generally, this is not a practice I would condone. It’s okay while we’re experimenting but permissions are very important on live systems!

    Lastly, we’ll download and install Largo and the Largo child theme stub.
    ssh user@$DIP 'sudo chmod -R 777 /var/www'
    wget https://github.com/INN/Largo/archive/master.zip -O $PROJECT/wp-content/themes/largo.zip
    unzip $PROJECT/wp-content/themes/largo.zip -d $PROJECT/wp-content/themes/
    mv $PROJECT/wp-content/themes/Largo-master $PROJECT/wp-content/themes/largo
    wget http://largoproject.wpengine.netdna-cdn.com/wp-content/uploads/2012/08/largo-child.zip -O $PROJECT/wp-content/themes/largo-child.zip
    unzip $PROJECT/wp-content/themes/largo-child.zip -d $PROJECT/wp-content/themes
    rm -rf $PROJECT/wp-content/themes/__MACOSX/

    We are now ready to customize our Project Largo child theme!


    Tags: , , , , , ,
    Permalink: 20140217.project.largo.docker

    Thu, 13 Jun 2013

    Blogitechture continued… Simplify with Vim

    Last we were discussing the structure and design of your own CLI-centric blog platform, we had some crude methods of starting and resuming posts before publishing.

    Today, let’s explore a little more into setting up a bloging-friendly environment because we need to either make the experience of blogging easy or we’ll grow tired of the hassle and lose interest.

    We can reasonably anticipate that we won’t want to beleaguered with repetitious typing of HTML bits. If we’re going to apply paragraph tags, hyperlinks, codeblocks, etc. with any frequency, that task is best to be simplified. Using Vim as our preferred editor, we will use Tim Pope’s brilliant plug-ins ‘surround’ and ‘repeat’, combined with abbreviations to take away the tedium.

    The plug-ins just need dropped into your Vim plugin directory (~/.vim/plugin/). The directory may not exist if you don’t have any plug-ins yet. That’s no problem, though. Let’s grab the plugins:

    cd ~/.vim/
    wget "http://www.vim.org/scripts/download_script.php?src_id=19287" -O surround.zip
    wget "http://www.vim.org/scripts/download_script.php?src_id=19285" -O repeat.zip

    Expand the archives into the appropriate directories:

    unzip surround.zip
    unzip repeat.zip

    Ta-da! Your Vim is now configured to quickly wrap (surround) in any variety of markup. When working on a blog, you might use <p> tags a lot by putting your cursor amid the paragraph and typing yss<p>. The plug-in will wrap it with opening and closing paragraph tags. Move to your next paragraph and then press . to repeat.

    That out of the way, let’s take advantage of Vim’s abbreviations for some customization. In our .vimrc file, we can define a few characters that Vim will expand according to their definition. For example, you might use:
    ab <gclb> <code class="prettyprint lang-bsh linenums:1">
    Then, any time you type <gclb> and bress <enter>, you’ll get:
    <code class="prettyprint lang-bsh linenums:1">

    The next time that we take a look at blogitecture, we will focus on making the posts convenient to manage from our CLI.


    Tags: , , , ,
    Permalink: 20130613.blogitechture.continued

    Tue, 04 Jun 2013

    Painless protection with Yubico’s Yubikey

    Recently, I ordered a Yubikey and, in the comments section of the order, I promised to write about the product. At the time, I assumed that there was going to be something about which to write: (at least a few) steps of setting up and configuration or a registration process. They’ve made the task of writing about it difficult, by making the process of using it so easy.

    Plug it in. The light turns solid green and you push the button when you need to enter the key. That’s the whole thing!

    Physically, the device has a hole for a keychain or it can slip easily into your wallet. It draws power from the USB port on the computer, so there’s none stored in the device, meaning it should be completely unfazed if you accidentally get it wet.

    Let’s take a look at the device.

    > lsusb | grep Yubico

    Bus 005 Device 004: ID 1050:0010 Yubico.com Yubikey

    We see that it is on Bus 5, Device 4. How about a closer look?

    > lsusb -v -s5:4

    Bus 005 Device 004: ID 1050:0010 Yubico.com Yubikey
    Couldn't open device, some information will be missing
    Device Descriptor:
      bLength                18
      bDescriptorType         1
      bcdUSB               2.00
      bDeviceClass            0 (Defined at Interface level)
      bDeviceSubClass         0 
      bDeviceProtocol         0 
      bMaxPacketSize0         8
      idVendor           0x1050 Yubico.com
      idProduct          0x0010 Yubikey
      bcdDevice            2.41
      iManufacturer           1 
      iProduct                2 
      iSerial                 0 
      bNumConfigurations      1
      Configuration Descriptor:
        bLength                 9
        bDescriptorType         2
        wTotalLength           34
        bNumInterfaces          1
        bConfigurationValue     1
        iConfiguration          0 
        bmAttributes         0x80
          (Bus Powered)
        MaxPower               30mA
        Interface Descriptor:
          bLength                 9
          bDescriptorType         4
          bInterfaceNumber        0
          bAlternateSetting       0
          bNumEndpoints           1
          bInterfaceClass         3 Human Interface Device
          bInterfaceSubClass      1 Boot Interface Subclass
          bInterfaceProtocol      1 Keyboard
          iInterface              0 
            HID Device Descriptor:
              bLength                 9
              bDescriptorType        33
              bcdHID               1.11
              bCountryCode            0 Not supported
              bNumDescriptors         1
              bDescriptorType        34 Report
              wDescriptorLength      71
             Report Descriptors: 
               ** UNAVAILABLE **
          Endpoint Descriptor:
            bLength                 7
            bDescriptorType         5
            bEndpointAddress     0x81  EP 1 IN
            bmAttributes            3
              Transfer Type            Interrupt
              Synch Type               None
              Usage Type               Data
            wMaxPacketSize     0x0008  1x 8 bytes
            bInterval              10
    

    There’s not a great deal to be seen here. As it tells you right on Yubico’s site, the device presents as a keyboard and it “types” out its key when you press the button, adding another long and complex password to combine with the long and complex password that you’re already using.

    Keep in mind that this device is unable to protect you from keyloggers, some of which are hardware-based. It’s critically important that you are very, very careful about where you’re sticking your Yubikey. Even Yubico cannot protect us from ourselves.


    Tags: , , , ,
    Permalink: 20130604.yay.yubico.yubikey

    Thu, 30 May 2013

    Making ixquick your default search engine

    In this writer’s opinion, it is vitally important that we take reasonable measures now to help insure anonymity, lest we create a situation where privacy no longer exists, and the simple want of, becomes suspicious.

    Here’s how to configure your browser to automatically use a search engine that respects your privacy.

    Chrome:

    1. Click Settings.
    2. Click “Set pages” in the “On startup” section.
    3. Enter https://ixquick.com/eng/ in the “Add a new page” text field.
    4. Click OK.
    5. Click “Manage search engines…”
    6. At the bottom of the “Search Engines” dialog, click in the “Add a new search engine” field.
    7. Enter
      ixquick
      ixquick.com
      https://ixquick.com/do/search?lui=english&language=english&cat=web&query=%s
    8. Click “Make Default”.
    9. Click “Done”.

    Firefox:

    1. Click the Tools Menu.
    2. Click Options.
    3. Click the General tab.
    4. In “When Firefox Starts” dropdown, select “Show my home page”.
    5. Enter https://ixquick.com/eng/ in the “Home Page” text field.
    6. Click one of the English options here.
    7. Check box for “Start using it right away.”
    8. Click “Add”.

    Opera:

    1. Click “Manage Search Engines
    2. Click “Add”
    3. Enter
      Name: ixquick
      Keyword: x
      Address: https://ixquick.com/do/search?lui=english&language=english&cat=web&query=%s
    4. Check “Use as default search engine”
    5. Click “OK”

    Internet Explorer:

        _     ___  _ __        ___   _ _____ ___ 
       | |   / _ \| |\ \      / / | | |_   _|__ \
       | |  | | | | | \ \ /\ / /| | | | | |   / /
       | |__| |_| | |__\ V  V / | |_| | | |  |_| 
       |_____\___/|_____\_/\_/   \___/  |_|  (_) 
      
      
      (This is not a good strategy for privacy.)

    Congratulations!

    \o/

    You are now one step closer to not having every motion on the Internet recorded.

    This is a relatively small measure, though. You can improve your resistance to prying eyes (e.g., browser fingerprinting) by using the Torbrowser Bundle, or even better, Tails, and routing your web usage through Tor, i2p, or FreeNet.

    If you would like more on subjects like anonymyzing, privacy and security then drop me a line via email or Bitmessage me: BM-2D9tDkYEJSTnEkGDKf7xYA5rUj2ihETxVR


    Tags: , , , , , , , , , , , , , ,
    Permalink: 20130530.hey.you.get.offa.my.data

    Wed, 15 May 2013

    Git: an untracked mess?

    There may be times when you find your Git repository burdened with scads of untracked files left aside while twiddling, testing bug patches, or what-have-youse.

    For the especially scatter-brained among us, these things can go unchecked until a day when the useful bits of a git status scroll off the screen due to utterly unimportant stuff. Well, hopefully unimportant.

    But we’d better not just cleave away everything that we haven’t checked in. You wonder:
    What if there’s something important in one of those files?

    You are so right!

    Let’s fix this!

    Firstly, we want a solution that’s reproducible. Only want to invent this wheel once, right?

    Let’s begin with the play-by-play:

    Git, we want a list of what isn’t tracked: git ls-files -o --exclude-standard -z

    We’ll back these files up in our home directory (~), using CPIO but we don’t want a poorly-named directory or finding anything will become its own obstacle. So we’ll take use the current date (date +%Y-%m-%d), directory (pwd) and branch we’re using (git branch) and we’ll twist all of it into a meaningful, but appropriate, directory name using sed. git ls-files -o --exclude-standard -z | cpio -pmdu ~/untracked-git-backup-`date +%Y-%m-%d`.`pwd | sed 's,^\(.*/\)\?\([^/]*\),\2,'`.`git branch | grep "*" | sed "s/* //"`/

    Then Tell Git to remove the untracked files and directories: git clean -d -f

    Ahhhh… Much better. Is there anything left out? Perhaps. What if we decide that moving these files away was a mistake? The kind of mistake that breaks something. If we realize right away, it’s easily-enough undone. But what if we break something and don’t notice for a week or two? It’d probably be best if we had an automated script to put things back the way they were. Let’s do that.

    Simple enough. We’ll just take the opposite commands and echo them into a script to be used in case of emergency.

    Create the restore script (restore.sh), to excuse faulty memory: echo "(cd ~/untracked-git-backup-`date +%Y-%m-%d`.`pwd | sed 's,^\(.*/\)\?\([^/]*\),\2,'`.`git branch | grep "*" | sed "s/* //"`/; find . -type f \( ! -iname 'restore.sh' \) | cpio -pdm `pwd`)" > ~/untracked-git-backup-`date +%Y-%m-%d`.`pwd | sed 's,^\(.*/\)\?\([^/]*\),\2,'`.`git branch | grep "*" | sed "s/* //"`/restore.sh

    Make the restore script executable: chmod u+x ~/untracked-git-backup-`date +%Y-%m-%d`.`pwd | sed 's,^\(.*/\)\?\([^/]*\),\2,'`.`git branch | grep "*" | sed "s/* //"`/restore.sh

    Lastly, the magic, compressed into one line that will stop if any command does not report success: a='untracked-git-backup-'`date +%Y-%m-%d`.`pwd | sed 's,^\(.*/\)\?\([^/]*\),\2,'`.`git branch | grep "*" | sed "s/* //"`; git ls-files -o --exclude-standard -z | cpio -pmdu ~/$a/ && git clean -d -f && echo "(cd ~/$a/; find . -type f \( ! -iname 'restore.sh' \) | cpio -pdm `pwd`)" > ~/$a/restore.sh && chmod +x ~/$a/restore.sh; unset a


    Tags: , , , ,
    Permalink: 20130515.git.untracked.mess