I guess the previous bitcoin post was a bit of a wake up call again. About 14 days after that post one card's fan stopped spinning and died. Fortunately I was alerted and shut things down before any damage was done. The card was returned for RMA. I'd hope for a new one back - considering the fan stopping could have damaged the GPU, however I was at the mercy of the supplier. And I'm still at their mercy as this friday will be 4 weeks since it was returned and still no word whether it will be repaired, replaced or refunded. A lot changes in 4 weeks. BTC2USD for example, is down from 12 to 4.5. Hmmm. I did get the PCI-E extenders and they do work a treat.

Thursday this week is RUOKDay 2011. So I've been listening to TripleJ's hack program a bit lately and it reminded me of about this time last year when an episode dealt with RUOKDay which was very interesting to me at the time. It raised a very important point which wasn't immediately obvious. Don't jump in and ask someone if they're ok unless you truly and honestly are ready for any answer.

Speaking of listening to things, at work we've taken a liking to listening to the CFox. They seem to play lots of music from when I was in high school, which is cool, mixed in with some new stuff and more metallica/nirvana than most stations would play. The ad's are funny too, being from Vancouver it's all Canada-ay. After noticing they seem to play the same tracks multiple times in a day (like all commercial stations) we started tracking it. Pumped up kicks had it's day, and is dwindling now, it's still funny when it's heard around the place though. It makes me laugh.

Minecraft. What can I say. There's a reason I didn't play WoW and this could be it. So far I've only lost one whole evening to it (and built a Nyan cat). It's fun to do stuff in, but is a massive time sink. I also built a weighted companion cube, 35x35x35 too, only another 5 or so hours. Oops.

Storage. I've got the sata port multipliers I wanted, out of China which went as smoothly as could have hoped. I did take precautions on paying by credit card by obtaining a prepaid mastercard to pay with, which was fine. The Woolworths ones have the best fees of the ones I looked at, and was easy to buy and use. Highly recommended. So far I've only done basic testing with them on the doomed (bitcoin) AMD board. I need to get a wriggle on with this project, because the new tv shows all start this week.

"Not all those who wander are lost." - J.R.R Tolkien

Wow another few weeks fly by. Lets see. I put my Wii onto ebay and it sold for just over $200. I've spent a few days learning python and mucking around with it in modpython and then modwsgi. I'll be playing with it and mongoDB for a small project that I'm building to scale up. I finished reading The Art of Being Minimalist, and started on Minimalist Business, both are highly a recommended read.

The DVD ripping project has progressed further. I've finished all movies and made a good stab at the TV shows. This lead to a much more in-depth understanding of de-interlacing filters than I would have preferred. As the perfectionist in me decided it needed to be right I had to find the best settings. Many of the disk's I have are poorly mastered that there is little you can do to fix errors, however some come up brilliantly with just the right settings. I noticed that usually NTSC/USA sourced material (even region 4 PAL versions of USA content) come up OK, but PAL sourced material (ABC/BBC) stuff really needs a lot of work to make it not look like a total dogs breakfast. Some disk's were even interlaced and then de-interlaced with the wrong field order prior to being re-interlaced to go onto DVD. The result was essentially unfix-able. De-interlacing video correctly is like voodoo, and as much as I have no respect for so called professionals who screw it up, it really does take skill, knowledge, practice and some luck to get it right. That aside, there is no excuse for screwing up the final product especially if it's being mass produced and sold. When I pay full price for something, I expect it to be right, not half assed. Free stuff can be half assed, because it's free, fine. It's breaking the old adage, you get what you pay for. It's things like that which make people think the vodafone network has good coverage or speed.

That's similar to the reason I stopped buying DVD boxed sets. You get screwed by supporting early, and rewarded for jumping in late or after the boats left. Don't ever buy a show season by season, because after the show's finished they'll release a box of the lot, which will be better, for less money. Or while the show is airing, they'll change the cover/box artwork so your neat set doesn't match. I've been burned enough times to know that it's not a once off or my bad luck, it's standard procedure, so my response is to vote with my feet and not play that game.

While all this was going on, more house cleaning was done, I like it when tasks complement each other. The feeling of achieving a result is great. So I filled more bins full of crap and they have been dumped. I'll soon be putting up a list of DVDs which are essentially free to a good home for a token amount of cash back.

We're nearly through the silly season now. So much craziness. As the year is about to wrap up, perhaps a review of last year's new years resolutions. I aimed low this year, and managed to achieve. I simply wanted to "Not eat maccas all year" or really, to see how long I could last without going. Well as it turns out I haven't eaten there all year (since mid December 2009), and only drank coffee from Mc Cafe twice, in October, while in Esperance prior to the Tour De Freedom 1000. Even that was my fault, as I hadn't yet found the Coffee Cat who despite being busy and slow (it was holiday time) manages to produce something that could pass for coffee in Perth. Even if it's not just a coffee cup.

A while back I bought some OCZ Vertex 2 (sandforce based) ssd's, which dropped in price 2 weeks later (as expected). I put one into my desktop when I rebuilt it, which was great, now there's definitely no bottleneck on IO, and even though it's only got 4gb of ram (shared video too - so more like 3.5gb) it now has fast swap. Obviously you don't want to swap ever, but if you do start dipping into swap you'd prefer it to be fast and not impacted by other random IO on the disk, so the ssd is great.

The other 2 ssd's I bought to use as a cache on my primary filer box. ZFS lets you have external caches for both read and write. The write cache is like the journal on a classic file system and is called the ZFS Intent Log (or ZIL). When the ZIL is external to the zpool, it's commonly called a separate log, or slog device. As the file system is built with failure in mind, you need to be aware of the various failure situations if you lose different devices. Losing the ZIL is bad, but no longer catastrophic (it used to be). Now you'll just lose all uncommitted changes to the disk, which is fine, and it won't corrupt the pool. Obviously losing data is bad, and it's always been recommended that the ZIL be on mirrored storage. To select the size of the ZIL there are some calculations related to how much data can be written in 10 seconds (and it's flushed out at least every 10 seconds), and also taking into account system RAM size too. Aiming way too high (but catering for growth maybe) I set mine at 8gb, mirrored.

The rest of the ssd's were to become the read cache, known as the L2ARC cache. The ARC (adjustable replacement cache) cache is a purely in ram cache, and L2ARC is the second level version to be used on fast secondary storage devices like 15K drives or ssds. Objects in this cache are still checksummed, so a device going bad can't corrupt anything (if cache is bad, just read off the primary storage). Due to this there is no point mirroring the read cache and by adding multiple devices you essentially stripe your cache, which is good. So the 2 ssd's were sliced up (Solaris partitions) with an 8gb slice for the mirrored ZIL, and the rest for the L2ARC. Using 60gb ssd's I've now got over 90gb of high speed read cache, the theory is this cache could read at over 500MB/s, in practice it's hard to tell. At least the ssd's are rated at 50000 IOPS for 4k random writes.

Half the idea behind all this was to improve power management. Say you're watching a film, the whole file can be pre-read into cache and then while watching, it can be served purely from the cache while the main disks have spun down. ZFS apparently has some very neat caching algorithms in it, and I'm probably not seeing the best behaviour because the box doesn't have sufficient system ram (only 2gb), but in it's defence it is a 5 year old box. A rebuild (apart from disks) is long overdue.

So to actually do all this, once the disk is sliced up (using format) you can simply add the log and cache vdev's as follows (with your device names):

zpool add <poolname> log mirror c2d0s1 c3d0s1
zpool add <poolname> cache c2d0s3 c3d0s3

Or if you were building the zpool from scratch (say with 8 disks) and all the above craziness (as you do):

zpool create <poolname> raidz2 c0t0d0 c0t1d0 c0t2d0 c0t3d0 c0t4d0 c0t5d0 c0t6d0 c0t7d0 \
     log mirror c2d0s1 c3d0s1 cache c2d0s3 c3d0s3

Which would leave you with a pool along these lines (once you've put heaps of data on it):

# zpool status
 pool: yawn
 state: ONLINE
 scrub: scrub completed after 14h31m with 0 errors on Wed Nov 17 01:41:46 2010
config:

        NAME        STATE     READ WRITE CKSUM
        yawn        ONLINE       0     0     0
          raidz2-0  ONLINE       0     0     0
            c0t0d0  ONLINE       0     0     0
            c0t1d0  ONLINE       0     0     0
            c0t2d0  ONLINE       0     0     0
            c0t3d0  ONLINE       0     0     0
            c0t4d0  ONLINE       0     0     0
            c0t5d0  ONLINE       0     0     0
            c0t6d0  ONLINE       0     0     0
            c0t7d0  ONLINE       0     0     0
        logs
          mirror-1  ONLINE       0     0     0
            c2d0s1  ONLINE       0     0     0
            c3d0s1  ONLINE       0     0     0
        cache
          c2d0s3    ONLINE       0     0     0
          c3d0s3    ONLINE       0     0     0

errors: No known data errors

# zpool iostat -v
               capacity     operations    bandwidth
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
yawn        12.7T  1.81T     26      5  3.20M   482K
  raidz2    12.7T  1.81T     26      5  3.20M   440K
    c0t0d0      -      -      6      1   550K  73.8K
    c0t1d0      -      -      6      1   550K  73.8K
    c0t2d0      -      -      6      1   550K  73.8K
    c0t3d0      -      -      6      1   550K  73.8K
    c0t4d0      -      -      6      1   550K  73.8K
    c0t5d0      -      -      6      1   550K  73.8K
    c0t6d0      -      -      6      1   550K  73.8K
    c0t7d0      -      -      6      1   550K  73.8K
  mirror    11.9M  7.99G      0      0      0  41.4K
    c2d0s1      -      -      0      0      0  41.4K
    c3d0s1      -      -      0      0      0  41.4K
cache           -      -      -      -      -      -
  c2d0s3    46.8G     8M      0      0  16.8K  98.6K
  c3d0s3    46.8G     8M      0      0  16.8K  98.3K
----------  -----  -----  -----  -----  -----  -----

Later if you ever need to, you can remove these cache/slog devices, say you wanted to use the ssd's elsewhere or needed the sata ports for other spinning disks. Cache can be removed from any version, and slog/zil can be removed as long as you're on zpool version 19 or above. You just have to be careful which command you use, like when adding devices as they each have different meaning.

zpool remove - removes a cache device or top level vdev (mirror of logs only). Mirror vdev name comes from the zpool status output.

zpool remove yawn c3d0s3
zpool remove yawn mirror-1

zpool detach - detaches a disk from a mirror vdev (can return it to a single device vdev if you remove the second last disk). Example below detaches c3d0s1 from a mirror, if it's not a mirror the command will error out. Detach works on any mirror vdev, not just logs.

zpool detach yawn c3d0s1

zpool attach - makes a mirror out of an existing vdev (single device, or mirror) Example attaches new device c3d0s1 as a mirror of already present device c2d0s1. Attach also works on any vdev, not just logs.

zpool attach yawn c2d0s1 c3d0s1

After using ZFS for over 2 years now I've come to really appreciate it. There's a definite learning curve to it, but no more than equivalent on Linux. Linux Raid + LVM + + how to resize/reshape and so on. Actually zfs might be simpler, as all the commands are clearly documented in one place and behave all the same. The best part of ZFS is knowing your data is not rotting away on disk, and the very easy incremental replication, and snapshots, and Solaris's smoking fast NFS server, and and and

So I recently spent 10 days in Timor Leste, 4 nights camping out and about.

Let me prefix this with I've never been to Indonesia at all (not even Bali) and I guess my only real exposure to the third world would be seeing aspects of it in South Africa over 10 years ago. Seeing from afar, not being part of it.

Overall people were friendly, well meaning and helpful. We didn't see any crime or even traffic accidents (despite the Indonesian style traffic flow). I was there for a 5 day bike race, across the country and back, so I got to see a lot of the countryside, villages and amazing views. During the race everywhere we went had streets lined with locals all cheering us on. My impression was there is a very strong sense of patriotism and unity for the country, despite there still being some rebel factions and it's history being covered in bloodshed. It seems the vast majority are proud to call Timor Leste their home. Before going I wasn't aware of how important Timor Leste was during WW2 especially for us Australians. While there we went the memorial at Dare (which had an amazing view of Dili from the hills), and on our way through Balibo we visited the house where the 5 journalists were killed in 1975.

Initially I guess I was hesitant as to why I was there in the first place, but once we were out on the road dodging cars and motorbikes it all came back to me. The 5 days of the race were awesome, although very tough at times and even though I've done a hard 5 day road ride before, this was something else. We climbed up to over 1900m elevation, we had to walk up rough sections with a grade of >28% and descend single track with gradients over 30%. Battling the heat and humidity on the coast and the wind and rain through the hills we raced on. I'll upload my photos to my gallery and put up some links to the gps data once I've sorted out the details. I also need to update mycyclinglog as it's a bit out of date again.

During my time there I even met President José Ramos-Horta and shook his hand. During his addresses to the riders he was humorous and very down to earth, and after watching Balibo I saw further insight into how strongly he cares for his country. From my limited and outside view he's an excellent choice for a leader. Lets not say anything about Australia's political situation, actually, no just one thing. We tried really hard to tie the election (bah at two party preferred system) and the AFL grand final did manage a tie. Movin right along.

After BikeSnobNYC linked to some minimalists and made fun of their purple tshirts, I read some of their blogs and sort of got inspired. I guess it's something I've been on the edge of for a while and all these various bloggers just say it like it is. So maybe it wasn't a delusion or bad idea, just something I hadn't thought through to a conclusion point. So as a start, I've started on a much needed spring clean and I've been far more ruthless than before with the stuff that's been here but never used in ages.

Quick storage update. I bought some OCZ vertex2 SSD's for use as mirrored ZIL and L2ARC cache on my Solaris based storage box. While mucking around with them I also played with an embedded Solaris build kit, which after some modifications actually works neat for me. I'll probably switch from Eon to my own base image when I install the SSD's into the primary box, a non obvious benefit of this is kernel power management works on base Solaris, but not on Eon (with my controller/drives/whatever). I've been rethinking my storage solution over the last 6 or so months with a goal of drastically reducing power consumption and bumping the performance dramatically too. Ultimately the SSD's will end up in a newer, smaller box with fewer disks, and the big kahuna box will not be on 24x7 anymore (and it's mirror obviously won't be either). Until that happens though, the new Solaris image hopefully will save some power with kernel power management and the SSD's will improve both read and write performance.

Riding off the back of that is a new desktop idea too. Since I switched to Linux full time a few months back, the atom+nvidia ion system has performed well, but not quite well enough for my use. So I'm toying with a real upgrade to that and will consolidate my VM's onto this box, so it'll have lots of ram. In the mean time, I'll try an SSD in the atom and see if that improves it much/at all. Most of the time I think it's IO bound, except when you know it's not - video playback (h264 is 99% ok thanks to the ion chip but everything else is so close but so far). The rest of the time when you know it's CPU bound, there's nothing you can do because although it's a dual core with hyperthreading it's still not capable of out-of-order execution and is only 1.6ghz (clocked to 1.9ghz I think). Still, for general use, it's an awesome box, and uses around 45w from the wall.

Oh no, with this I might be over my quota for words this week. All I've ever wanted was an honest week's pay for an honest day's work.