Converting from Apache1-style to (Debian-style) Apache2-style vhosts

Yeah, some of us are still doing that migration.

Anyway, historically Apache vhosts are all in one file at /etc/apache/httpd.conf or if you're really lucky something like /etc/apache/vhosts.conf.

Apache2 in Debian uses two directories - /etc/apache2/sites-available and /etc/apache2/sites-enabled. sites-available contains one file for each vhost and in order to enable them they're linked to from sites-enabled. This is all fairly nice an elegant and human friendly, but tedious to migrate to from Apache1.

Since this one's coincided with a feeling that I should know more awk here's how I just did this one:

cp /etc/apache/vhosts.conf /etc/apache2/sites-available
awk '/^"vhost" n }' vhosts.conf
for i in $(ls vhost*); do name=$(grep -i ^ServerName $i | awk '{print $2}'); mv $i $name ; done
rm /etc/apache2/sites-available/vhosts.conf

Yeah, the name should be doable in the initial awk, but by that point I sort-of just needed to get it done.

Allowing uploads of arbitrary files in MediaWiki

I did RTFM and I did what it said, and still my Mediawiki complained when I tried to upload executable files and things with funny file extensions or mime types. if $wgFileExtensions is empty but $wgEnableUploads = true and $wgStrictFileExtensions = false it should just let me upload anything. I can't think what other behaviour one would expect there, but set like that I can't upload my dodgy files.

So I've removed the code it uses to check.

Here's a pair of diffs if you'd also like to do this. These are on version 1.17.0 but I suspect it's not changed very much.

This just comments out the two blocks of code in UploadBase.php which check whether files are considered safe and warn if they're not - it prevents the checking and the warning:

  1. wiki:/home/wiki/public_html# diff includes/upload/UploadBase.php includes/upload/UploadBase.php.bak
  2. 447,455c447,454
  3. < // ## Avi Commented this out so that we can upload whatever we like to our server. That was nice of him
  4. < // // Check whether the file extension is on the unwanted list
  5. < // global $wgCheckFileExtensions, $wgFileExtensions;
  6. < // if ( $wgCheckFileExtensions ) {
  7. < // if ( !$this->checkFileExtension( $this->mFinalExtension, $wgFileExtensions ) ) {
  8. < // $warnings['filetype-unwanted-type'] = $this->mFinalExtension;
  9. < // }
  10. < // }
  11. < //
  12. ---
  13. > // Check whether the file extension is on the unwanted list
  14. > global $wgCheckFileExtensions, $wgFileExtensions;
  15. > if ( $wgCheckFileExtensions ) {
  16. > if ( !$this->checkFileExtension( $this->mFinalExtension, $wgFileExtensions ) ) {
  17. > $warnings['filetype-unwanted-type'] = $this->mFinalExtension;
  18. > }
  19. > }
  20. >
  21. 557,570c556,569
  22. < // ## Avi Commented this out so that we can upload whatever we like to our server. That was nice of him
  23. < // /* Don't allow users to override the blacklist (check file extension) */
  24. < // global $wgCheckFileExtensions, $wgStrictFileExtensions;
  25. < // global $wgFileExtensions, $wgFileBlacklist;
  26. < // if ( $this->mFinalExtension == '' ) {
  27. < // $this->mTitleError = self::FILETYPE_MISSING;
  28. < // return $this->mTitle = null;
  29. < // } elseif ( $this->checkFileExtensionList( $ext, $wgFileBlacklist ) ||
  30. < // ( $wgCheckFileExtensions && $wgStrictFileExtensions &&
  31. < // !$this->checkFileExtension( $this->mFinalExtension, $wgFileExtensions ) ) ) {
  32. < // $this->mTitleError = self::FILETYPE_BADTYPE;
  33. < // return $this->mTitle = null;
  34. < // }
  35. < //
  36. ---
  37. >
  38. > /* Don't allow users to override the blacklist (check file extension) */
  39. > global $wgCheckFileExtensions, $wgStrictFileExtensions;
  40. > global $wgFileExtensions, $wgFileBlacklist;
  41. > if ( $this->mFinalExtension == '' ) {
  42. > $this->mTitleError = self::FILETYPE_MISSING;
  43. > return $this->mTitle = null;
  44. > } elseif ( $this->checkFileExtensionList( $ext, $wgFileBlacklist ) ||
  45. > ( $wgCheckFileExtensions && $wgStrictFileExtensions &&
  46. > !$this->checkFileExtension( $this->mFinalExtension, $wgFileExtensions ) ) ) {
  47. > $this->mTitleError = self::FILETYPE_BADTYPE;
  48. > return $this->mTitle = null;
  49. > }
  50. >

And this just stops Setup.php making-safe the $wgFileExtensions array by removing whatever's in $wgFileBlacklist from it, which I think wouldn't complain had I not already done Bad Things to those two variables, but it's late and it can't hurt to turn this off, too:

  1. wiki:/home/wiki/public_html# diff includes/Setup.php includes/Setup.php.bak
  2. 296,298c296,297
  3. < // ## Avi Commented this out so we can upload whatever we like to our server. That was nice of him
  4. < //# Blacklisted file extensions shouldn't appear on the "allowed" list
  5. < //$wgFileExtensions = array_diff ( $wgFileExtensions, $wgFileBlacklist );
  6. ---
  7. > # Blacklisted file extensions shouldn't appear on the "allowed" list
  8. > $wgFileExtensions = array_diff ( $wgFileExtensions, $wgFileBlacklist );
  9.  

Fail2Ban and date formats

Fail2Ban is utterly daft in at least one respect. Here's me testing a regex on a date format it doesn't recognise:

# fail2ban-regex '2010-12-14 15:12:31 - 80.87.131.48' ' - <HOST>$'
Found a match but no valid date/time found for 2010-12-14 15:12:31 - 80.87.131.48. Please contact the author in order to get support for this format
Sorry, no match

And on one that it does:

fail2ban-regex '2010/12/14 15:12:31 - 80.87.131.48' ' - <HOST>$'

Success, the following data were found:
Date: Tue Dec 14 15:12:31 2010
IP  : 80.87.131.48

Date template hits:
0 hit: Month Day Hour:Minute:Second
0 hit: Weekday Month Day Hour:Minute:Second Year
1 hit: Year/Month/Day Hour:Minute:Second
0 hit: Day/Month/Year:Hour:Minute:Second
0 hit: TAI64N
0 hit: Epoch

Benchmark. Executing 1000...
Performance
Avg: 0.10257935523986816 ms
Max: 0.125885009765625 ms (Run 8)
Min: 0.10085105895996094 ms (Run 780)

Ignoring for the moment the fact that it doesn't recognise 2010-12-14 15:12:31 (Seriously?)1 , the only way to get that list of date formats is by happening to pick a correct one. As soon as you no longer need a list of date formats you may use, it presents you with one.

What?

So, as an attempted fix for this situation, see above for a list of compatible date formats.

  1. It's worth noting, too, that the author is of the opinion that specifying your own date format is too much like hard work, so if you want support for any date format other than those already supported, you've to patch it yourself. Which is obviously way easier than just having a date regex in the config file []

Getting root on a UK T-Mobile Galaxy S

It's a bit weird. The process was really easy, but none of the tutorials I found worked; each stopped working at one point or another. So, assuming other people will hit the same barriers and want a Just Works way to get root, I've gone through my terminal history for the bits that worked. Obviously, this is just what worked for me; I can't guarantee it'll work anywhere else though if you're at all familiar with the process it'll probably look right. The only real stumbling blocks I hit were getting a recovery menu (unfamiliarity with adb) and picking a ROM that I could trust. There was nowhere near enough diligence in that bit of the process, though. Bad Avi. Also, for those still expecting disclaimers, this voids warranties.

I can't find a reboot-and-hold-down-X-key method of rebooting into recovery mode (the boot menu) that works, and it seems to be different for each variant of this device in any case. Using adb, the Android debugger, does work, and contrary to several scare stories doesn't require proprietary Samsung drivers.

adb's really easy to make work. First, install a jdk. You might do this differently if you're not running Debian:

root@debian:~# apt-get install sun-java6-jdk

While we're here, if you want to suggest a more interesting hostname, go for it. I'm having a bit of an imagination failure in that department.

Next, you need to grab the tarball of the Android SDK, extract it somewhere and make its tools subdir part of your $PATH. You probably should do most of this as some user that isn't root, but I was rather excited at the time:

root@debian:~# mkdir adb && cd adb
root@debian:~/#wget -q http://dl.google.com/android/android-sdk_r07-linux_x86.tgz
root@debian:~/# tar -xzf android-sdk_r07-linux_x86.tgz 
root@debian:~/adb# ls android-sdk-linux_x86
add-ons  platforms  SDK Readme.txt  tools
root@debian:~/adb# export PATH=${PATH}:/root/android-sdk-linux_x86/tools

Now (or perhaps while you're waiting for the tarball to arrive), enable USB debugging on your phone. It's under Settings -> Applications -> Development for some reason. Check the box next to "USB debugging". You'll need to have the USB cable unplugged. Plug it back in again, then, returning to your shell with adb in its path, check for the presence of your device:

root@debian:~/adb# adb devices
List of devices attached 
90006e8ba84e    device

If yours doesn't show up, I'm not really sure what to do. Google?

Now, you need to have the update.zip somewhere. I've uploaded the one I've used to here but this'll work for any of them.

root@debian~/adb# wget -q http://aviswebsite.co.uk/stuff/galaxys_root/update.zip

If UMS works on yours, copy update.zip with your favourite file copying method. Mine didn't, so I used adb. There are two sdcards in the Galaxy, an internal one and an external (removable) one. The internal one is mounted at /sdcard, the traditional location of removable ones, and the external one at /sdcard/sd. You want to put update.zip in /sdcard, not /sdcard/sd.

root@debian:~/adb# adb push update.zip /sdcard/update.zip

You then use adb to reboot into the recovery menu:
root@debian:~/adb# adb reboot recovery

Select "Apply sdcard:update.zip" and wait while it installs it, then "reboot system now". You now have root. The quickest way I can think to test it is to download and install the 'superuser' application from the market, then test it with adb:

root@debian:~/adb# adb shell
$ su
#

You'll get prompted (on the phone) to allow an unknown application root access, and then you'll have root. Congratulations, your phone is now yours. :)

Now, I'm off to follow the rest of How to make the vibrant software not suck, 'cause it's shocking out of the box.

Whoo! Theme update!

I've updated the theme, and applied my handy modifications that make it more grey. Here's the diff on the css file, if you're wondering what I did (and for next time when I forget). I despise CSS, so it's all nice and easy.

The changes are to give the <pre> tags a grey (#EEE) background colour, and to make the posts individual white boxes on grey, rather than an all-white page.

  1.  
  2. avi@avi:whiteasmilk_1.8$ diff style.css style.css.original
  3. 18,35d17
  4. < /* Added to make the code blocks pretty. Stupid CSS implementations mean this will break
  5. < any form of validity in order to actually make it work. Stupid browser people.
  6. < I can't remember where I got this from, but if you reckon I might've found it on your site
  7. < let me know and if I believe you I'll stick a URL here
  8. < */
  9. < pre {
  10. < border 0
  11. < padding: 0.2em 0.5em;
  12. < background-color:#EEE;
  13. < white-space:pre-wrap;
  14. < white-space: -moz-pre-wrap; /* Mozilla, since 1999 */
  15. < white-space: -pre-wrap; /* Opera 4-6 */
  16. < white-space: -o-pre-wrap; /* Opera 7 */
  17. < word-wrap: break-word; /* Internet Explorer 5.5+ */
  18. < }
  19. <
  20. <
  21. <
  22. 40c22
  23. < body {background-color:#c9c9c9;}
  24. ---
  25. > body {background-color:white;}
  26. 323,326c305
  27. < border-bottom:15px solid #c9c9c9;
  28. < padding-left:10px;
  29. < padding-right:10px;
  30. < background-color:#fff;
  31. ---
  32. > border-bottom:1px solid #999;

Giving Android a swap file

I don't know if it's because I'm doing more with it than I used to, or the rose-tinted specs that come with the novelty value have worn off, or if the later updates have been designed for more powerful hardware, but my G1's been lagging a bit recently, so I figured I'd have a look at what I can do to make it faster. Turns out it's quite receptive to the idea of a swap file.
You'll need root access for this - everything below is to be run as root. Following is how I did it on my cyanogen'd G1, but I don't see why it wouldn't work on anything else - it makes use of standard linux tools. This will likely advance the death of your sdcard - it increases the reads and writes by some large margin. I don't know enough about sdcards to know how much quicker it will die, but modern ones will likely deal with it better than older ones.

Anyway, in your terminal emulator:

  1. dd if=/dev/zero of=/sdcard/swap bs=1M count=100

You can call it whatever you like. The 'count' value (100 above) is how many MB of swap space you want. The above will create 100MB of swap.
This will take a few seconds (of the order of 50), so if it's 'hung' it's probably not crashed.

Then we make it into a swap file and activate it as swap space:

  1. mkswap /sdcard/swap
  2. swapon /sdcard/swap

Finally, we want to change the swappiness of the system. When the system memory is completely full, and more is requested, the kernel needs to work out how to create some space. It, generally, can do one of two things: drop some filesystem cache, or move some application memory to swap.
The swappiness value dictates the priority of either - a value of 0 means it will always try to drop fs cache rather than use swap, and a value of 100 means it will always try to use swap and preserve the fs cache. Only values between 0 and 100 inclusive are allowed, and neither guarantees any action - you can still find yourself swapping out at 0, for example - just the priority afforded each solution.
The default value for the Linux kernel is 60, which is generally not appropriate for something running completely from flash media (where the fs cache isn't so important). So I'm dropping it to 20, which seems to work for me. If it doesn't for you, try some other values, it's an on-the-fly change.

  1. echo 20 > /proc/sys/vm/swappiness

On rebooting the phone, the above settings wont hold. The swap file will still be available (assuming the SD card is still inserted), but on reboot you'll need to run

  1. swapon /sdcard/swap
  2. echo 20 > /proc/sys/vm/swappiness

On the plus side, you'll be rebooting less.

Cyanogen on my G1

I've just upgraded to Cyanogen on my G1 and it's lovely. Well, I got root, which is basically what I always wanted.

I basically followed the instructions on the Cyanogen wiki and everything worked exactly as described, I've nothing really to add here except to say it's brilliantly easy and everyone should do it.
One small note, though, the final reboot into Cyanogen takes a long time. Mine took just shy of thirty minutes, and I've read of others taking anything above about 15.

The first obvious benefits are root access, exchange activesync support, five workspaces (as against the stock 3) and a bunch of useful apps already installed, including a nifty power control widget which replaces my collection of 5 distinct widgets. I'll write more on it when I've used it more.
In case, as I did, you've been pondering this and wondering what the end result is like, it's almost exactly as previously, but with some new features, there's some screenshots on the webpage. I've not yet found anything to have been degraded by it, and it all looks and feels the same, but a bit more polished in areas. Though it only ships with one ringtone.

Some notes from configuring rTorrent

In anticipation of the new *buntus tomorrow, I'm configuring one of my servers as a torrent node for it, and for reasons unknown I've settled on rtorrent which is in the repos. The documentation is a little lacking (but does tell you how to do things like download torrents. Read it), and the most popular HowTo is quite verbose.

~/.torrentrc is the config file for rtorrent. The one from the Debian repos puts an example in /usr/share/doc/rtorrent/examples/rtorrent.rc, and apparently so does *buntu. You probably want to edit this. Path variables can be absolute or relative, and are generally relative and inside ./ by default. I've two directories, ~/torrents/ for the torrents and ~/.rtorrent/ for rtorrent's working directories.
Some variables I found to be handy:

  1. scgi_local="~/.rtorrent/socket/rpc.socket"

Defines the socket file for scgi communication, which you only really need if you want external stats from it. I did, but haven't yet got round to using them.

  1. session = ~/.rtorrent/sessio

The session directory, which lets multiple instances of rtorrent remember where they left off. Cannot be shared between instances.

Scheduling
rtorrent supports command scheduling, the syntax for which is occasionally documented. The syntax is approximately:

  1. schedule = a,b,c,d

Where
a: What you want to call this scheduled command1.
b: How many seconds after rtorrent starts you want to first execute the command.
c: The interval for subsequent executions.
d: the command you want to execute.
This comes in most handy for auotmatically starting torrents in a directory:

  1. schedule = watch_directory,5,5,load_start=~/torrents/*.torrent

Exactly what that does is left as an excercise to the reader (the example .rtorrent.rc explains, too).

  1. I don't know where this comes in handy later on []

Moving WinXP’s My Documents

The My Documents directory location is stored in the registry at HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders in a key called personal.

The following script is what I use to remap this to u:/documents:

  1. ;; create some dirs:
  2. u:
  3. mkdir documents
  4. mkdir outlook
  5.  
  6. ;; Move My Documents folder:
  7. ;; delete current setting
  8. reg.exe delete "HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders" /v Personal /f
  9. ;; create new setting
  10. reg.exe Add "HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders" /v Personal /t REG_SZ /d u:/documents

Using Synergy to share a keyboard and mouse across PCs

Synergy is a really neat way of using multiple computers at the same time, like a more convenient KVM switch (you do need to be physically close to all of them).

It basically allows you to have a monitor for each PC on your desk, and one keyboard and mouse with which to monitor them. Switching between PCs involves just moving the mouse pointer onto the relevant screen. It uses 'screens' rather than monitors, so you don't need to let it know if you've got a complicated multi-monitor setup on a host (or even if you add or remove monitors from one), and it allows for having screens of different sizes and for, say, the bottom half of one screen to line up with the top half of the next.

It's a server/client model - you have one server box into which you plug the keyboard and mouse. The rest connect over the network to it (in cleartext, don't do this where you don't trust the network).

I have two hosts. My work laptop is running Windows XP and has an external monitor plugged in to it. My testing PC is running Debian testing and is a PC proper. Here's my desk:
desk
jup-linux2 is the left monitor (attached to the PC to its left, running Debian), jup-rmt07 is the laptop on the right, which is also attached to the Sony screen in the middle. Since I take the laptop home with me occasionally, jup-linux2 is configured as a server, jup-rmt07 as the client (less to unplug).

Synergy is in the debian repositories, so it's just an apt-get install synergy. This provides two binaries, /usr/bin/synergyc and /usr/bin/synergys, which are the client and the server respectively.
To install it in Windows, you'll want to grab it from their Sourceforge page

Configuring and starting the server
So, having installed, we can configure! The configuration file can be arbitrarily named, mine's cunningly called ~/.synergy.conf and appears verbatim at the bottom of this. Synergy's really rather configurable, but I've never found much need for more than the basics, so I've an incredibly simple setup.

First, we define some 'screens'. Here we can also configure optons for screens, especially as regards the transference of caps-lock and num-lock statuses. I've no special requests, so I just list the hosts whose screens I want Synergy to manage:

section: screens
	jup-rmt07:
	jup-linux2:
end

Second, we define the links. For each monitor, we state what is at any edge of it. This is used to decide where to put the mouse pointer on leaving the screen, so it needs to be done in both directions - the fact that jup-linux2 is to the left of jup-rmt07 does not imply to synergy that jup-rmt07 is to the right of jup-linux2:

section: links
jup-rmt07:
	left = jup-linux2
jup-linux2:
	right = jup-rmt07
end 

If I have two screens at different heights, I can tell synergy that the top 30% of jup-linux2 lines up with the bottom 40% of jup-rmt07, for example:

jup-linux2
	right(70-100) = jup-linux2(0-40)
jup-rmt07
	left(0-40) = jup-rmt07(70-100)

Again, you always always always need to define the screen in both directions. The file is parsed by synergy to see what to do on leaving that particular screen - when you're in jup-rmt07 and move towards the left of the screen, it's only going to do anything if there's a left defined for that screen, irrespective of how many rights point there.
If my screens are above and below each other, up and down are used. A screen can have as many other screens round it as you like, by assigning percentages of edges to different screens.

Configuring the server under Windows involves a different process to use the same principles. Essentially, you build the same text file as above, but in a clicky gui. It's a little odd, but quite simple.

configuring Synergy server under Windows

To start the synergy server, we now run

  1. synergys --config ~/.synergy.conf

Replacing '~/.synergy.conf' with the path to wherever the config file is saved.

Configuring and starting the clients

So, we have a server. Now, clients.
On Windows, synergy is only one executable, so we start that, select the 'Use another computer's shared keyboard and mouse (client)' option, stick the server's hostname or IP in the box, and click 'start'.
If you have a *nix client, the command to connect to a server at jup-rmt07 is synergyc jup-rmt07. synergyc provides a few options for changing the behaviour.

Starting Synergy automatically
Finally, I want them to start automagically on boot/login. For the linux host, this is relatively easy, add the following to your crontab:

  1. @reboot synergys --config ~/.synergy.conf

And it'll be started on boot.
To start it under Windows, click the 'AutoStart' button. This will let you configure it to start it on login or, if you have the requisite permissions, start on boot.