with a side of crazy
Ok so with the surge in price of bitcoins in june-july spiking to USD30 each and then crashing back to about USD12 it got some decent media coverage.
Perhaps I was in a vunerable situation, looking for something hobby like and a decent distraction. See I needed a distraction from storage as that's a problem still sort of wrecking on the rocks waiting to sink. Mix into this the lack of a vmware box and I've sort of lost the plot again.
So I sank some cash into a box to mine bitcoins, which was fine, or would have been fine if the hardware didn't have issues. Basically I rushed into it and opted for something available right now vs shopping around for a quality product. So the Gigabyte board I have has a fault which took about 2 weeks to isolate, and one of 2 dimms is also faulty. Now I really hate returning stuff, and after dealing with gigabytes useless and slow support decided to never buy a product from them ever again, and I still have to convince the shop that there is a fault. So that will be downtime, and in bitcoin mining - all downtime costs money. Real money.
A further complication came from a software hang issue caused by ATI Catalyst drivers, OpenCL code and an accelerated video layer (or GPU call) on Windows. Basically a major bug in the ATI drivers causes a hang of the machine if a video plays while OpenCL code is running. This was after realising the driver bundle with control center caused crashes itself, so switching to the driver only pack at least let me isolate the problem. Oh and the fix has been committed for the driver release due out AFTER the next one. So October apparently. Thank's AMD/ATI. I'll never buy your products ever again too. For reference, this card was released in NOVEMBER last year.
The memory fault isn't such a problem now, but once this fad wears off (or becomes unprofitable due to the price of power going up due to failure of the government (to cater for population growth or restrict immigration)) I was hoping to use this box as a vmware host. But alas, it's again apparent that I'm not allowed to have things go well or to plan.
Further proof of this was the build up to the final Harry Potter film. My glasses broke, and not having spares anymore (long story) left me without sight. Skip forward a week and I'm still without, and now without more money too. Health insurance is such a scam. It barely covered 25% of the cost of replacement glasses. Sigh.
I'm not allowed to win. Someone please remind me why is it I try at all.
Perhaps I should move to some far away land which is cheaper than here. Perth's now in the top 10 most expensive cities to live, and it's higher up the list than New York. What the hell am I staying here for (apart from America being filled with americans).
I started using gmail when spam became intolerable and spam assassin stopped working enough for me. Maintaining my own email server became too greater maintenance for not enough gain. Gmail's spam filter is excellent and just works. I like things that just work. A decent web mail client wasn't around at that stage or not for free. The ajax'y ones weren't that good yet and gmail's was again, excellent. So I did it, I switched. I even put up with having to forward all my mail to a single email account and handling it all through there. I also put up with the from header being set to it, exposing the actual account name which I never liked. Not a perfect solution but it worked. 80/20 rule be gone, this was more 95/5. Initially it was a tactical solution which just never got changed.
Then when google apps came along, even better, my whole email solution could be google hosted, however I still used forwards to the master gmail account (because the google apps account wasn't a full featured google account at the time) and I used other services. Since then google apps accounts are now 99% full featured (95/5 again) so I moved all my data from my gmail google account to my apps google account, and all was happy again. My from headers are right now, still no spam, it just works.
However by this stage I had already been sucked into the cloud. As much as I don't trust google (or trust them only slightly more than facebook/twitter) I do use it for important stuff - my email. The contents of my account is important, and due to this I was backing it up for a long time. Though not for a while now. When I moved between accounts I realised the backup tools all suck - hint: they don't backup everything and they battle to restore everything if restore is even an option.
From the privacy perspective; Email can be faked easily enough, or lost in spam, so the level of trust/denial is quite low anyway, just like sms. Being one in billions on the service helps you to blend in.
I believe eventually everything posted online will become public. Through poor information security, coding mistakes, disgruntled/curious employees, cracking and so on. If you're hiding information behind a sites privacy settings to keep something private, it's just a time delay lock, not a safe. It's happened before and it'll happen again, information leaks. Don't trust anyone.
So the answer is to either to play with fake information even if it limits the usefulness of it in the first place, or not play the game at all.
Right now I'm toying with two courses of corrective action:
Wipe the slate clean and start a new fully embracing the cloud and what it provides. AKA the whole hog.
Pull back the error of my ways and figure out a new way forward/around. AKA the long way.
I've ignored the middle ground; Combine existing with the rest of the cloud but not embracing by using fake details; because that dilutes the potential and raises more questions than it answers.
Or simply continue on as I have been and ignore my internal monologue's screaming. That's the do nothing option.
Mental note: must not forget to include a random image.
<3 the philosiraptor. As much as I like things that just work. It's getting harder. And no I haven't bought a mac, yet.
So the zfs experiment continues. Upon the release of b129 I set off into the unknown on a voyage of dedupe. Which at first had the promise of lower disk usage, faster IO speeds and a warm fuzzy feeling deep down that you only get from awesome ideas becoming reality. ahem
Most sources say you need more ram, and that is true, what they don't say is how much ram for what size data set, which might be more useful to home users like me. My boxes have 2gb of ram each, and that is not enough for dedupe, no way near. Not if you have a 6 TB of randomish data. I might retry when I get to 8gb ram but not before. You see, if it can't keep the whole of the dedupe table in ram ALL the time, any write to a dedupe enabled volume will result in reads for the rest of the table, or at least seeks. So what I saw was a gradual slowdown while writing to the volume, I was determined to let it finish, to see what savings I would make, and then scrap it due to performance, but after waiting 16 days for the copy, I cancelled it.
The only way I found to even see the contents/size of the dudupe table (DDT) is: zdb -DD
DDT-sha256-zap-duplicate: 416471 entries, size 402 on disk, 160 in core DDT-sha256-zap-unique: 47986855 entries, size 388 on disk, 170 in core DDT histogram (aggregated over all DDTs): bucket allocated referenced ______ ______________________________ ______________________________ refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE ------ ------ ----- ----- ----- ------ ----- ----- ----- 1 45.8M 5.69T 5.66T 5.66T 45.8M 5.69T 5.66T 5.66T 2 394K 43.0G 40.3G 40.3G 821K 89.0G 83.0G 83.1G 4 9.90K 527M 397M 402M 47.0K 2.35G 1.76G 1.79G 8 2.06K 125M 82.4M 83.4M 21.1K 1.20G 795M 806M 16 391 13.7M 8.54M 8.76M 7.26K 272M 162M 166M 32 69 1.17M 776K 822K 3.08K 51.3M 32.7M 34.8M 64 17 522K 355K 368K 1.43K 36.9M 25.1M 26.2M 128 6 130K 7K 11.2K 1.07K 31.3M 1.50M 2.23M 256 2 1K 1K 2.48K 833 416K 416K 1.01M 512 4 2K 2K 4.47K 2.88K 1.44M 1.44M 3.32M 2K 1 512 512 1.24K 2.79K 1.39M 1.39M 3.46M Total 46.2M 5.73T 5.70T 5.70T 46.7M 5.78T 5.74T 5.74T dedup = 1.01, compress = 1.01, copies = 1.00, dedup * compress / copies = 1.01
Saving's of around 80gb with dedupe and compression (backup box so no real world performance requirement) is just not worth the need for 3-n times the ram and possibly an ssd for the l2arc cache to speed things up. Yep, the suggestion and observed behaviour was to hook up a cheap small (30gb) SSD for cache to accelerate it. I don't mind that so much for a primary but this is my backup/2nd copy box so it's not really ideal. Certainly not for 80gb of savings, or at current prices around $5 of disk.
My second attempt is now underway, this time I've sliced up my data sets into more volumes, and by more that means smaller average size, so this time around 2TB max per volume, which from experience at work I've learned is a good rule of thumb. So now I can enable compress+dedupe on only specific bits, hopefully where the most savings is to be made, and then the rest is just stored raw. This way the savings might be similar, but without the major write speed penalty. I've also realised for the production box if I want screaming performance, I'll throw an ssd on there, but that means more sata ports, which means a major change. I also need to work on power management too.
One thing that has gone right this time, is I'm now using CF->IDE adaptors and booting off that. This way the OS think's it's on a 2gb hdd, so booting doesn't have the complexity of usb boot and also uses less power and doesn't take up a sata port. Of course new boards don't have pata anymore so I might need to get a CF->sata one in future.
Another thing that must be said, Solaris's CIFS server is fast.