macdiggs.com

Ok so today I have noticed that ranking ramp-up for a specific person still continues. I have blocked one device, but the other found another one, it seems..

Both devices are mobile phones of the same carrier - KDDI. Looks like some manager (the site is a talents' directory site) decided to rank his favorite pet up, no matter what, but it also looks like that person is only doing this from mobile phone, and his carrier stands unchanged. So, I decided to tighten block logic (if you can call that logic) on that specific person's profile.

Before, I was only blocking a specific User-Agent:

RewriteCond %{HTTP_USER_AGENT} "KDDI\-KC35 UP\.Browser/6\.2\.0\.5" [NC]

RewriteRule ^.*$ - [F,L]

On one of my servers, I was under some weird "ranking up!" attack, which basically was just a loop of requests to one user's profile (making that users access count higher, and therefore ranking higher in the access top)

Here's what my Apache logs were telling me:

218.25.251.170 - - [17/Oct/2008:11:02:26 +0900] "GET /ono/profile HTTP/1.1" 200 2363 "-" "KDDI-KC35 UP.Browser/6.2.0.5 (GUI) MMP/2.0"
218.25.251.170 - - [17/Oct/2008:11:02:30 +0900] "GET /ono/profile HTTP/1.1" 200 2363 "-" "KDDI-KC35 UP.Browser/6.2.0.5 (GUI) MMP/2.0"
218.25.251.170 - - [17/Oct/2008:11:02:34 +0900] "GET /ono/profile HTTP/1.1" 200 2363 "-" "KDDI-KC35 UP.Browser/6.2.0.5 (GUI) MMP/2.0"

SetEnvIfNoCase User-Agent Mozilla getout

<Directory "/var/www/html/myserver">

Order allow,deny

Allow from all

Deny from env=getout

</Directory>

Sometimes, I really love just to HTML-code an interesting concept. That's exactly what I was asked to do by the fine folks at Windup Design.

The site is now up: www.cottageinternational.com

Have a look. Great concept by Windup Design, and nice careful coding by yours truly ;)

This is mostly of interest to my russian-speaking friends, hence Russian is used :)

В последнее время в Японии идут рекламные ролики продукта, созданного в России, очень полезного и все такое. Называется чудо-продукт - "кефир". Продается почему-то в мини-емкостях (видимо чтобы не отравились). Сегодня решил попробовать (кстати не дешево! порядка $1.7 за баночку 100мл - это ж 17 грин за литр!)

Надеясь наконец-то попить вовеки-веков кефира, открываю банку..

Офигенный кефир придумали японцы… (справа - это я там в крышечку налил немного "йада" ). Вот такие изобретательные.. Ждем японского взгляда на следующий полезный для здоровья российский продукт :)

На банке кстати написано "НАСТОЯЩИЙ кефир" :)

PS: да-да, оно прозрачное. желто-зеленая прозрачная жидкость ;) Видимо просто сыворотку заливают. Без молока.. йад в общем.

I have some very weird problems going on with some Ruby on Rails-driven sites on my server.. Basically, everything works perfect after my mongrel clusters just launched, but as time passes (and it could be hours or even days), "serialize" part of my apps start returning absolutely wrong values, don't return them at all, or just don't store values correctly.

It is all fixed by restarting Mongrels, but I wonder if anybody knows what's up with this problem? I tried different versions of ruby (currently running 1.8.6p287), and don't want to upgrade to 1.8.7 yet (have too many systems, some running on Rails 1.2…)

WTF is going on?…

Well I exchanged my third iPhone for another one at Apple Store here in Nagoya. This is my fourth device in just a couple of months after iPhone introduction here in Japan.

A little history on replacements..

My first iPhone was fine for first 2 weeks and then developed a dead pixel near the edge of the screen. Not extremely annoying, but once you know it's there, well.. it just somehow take up good half of the screen and you just keep staring at it all day long. Got this iPhone replaced at Apple Store, without any problems.

My next iPhone was.. well.. I was bad because I seemed to have fried its modem's firmware after jailbraking and then updating it to 2.0.1 firmware, without first reverting to unjailbroken state. Still, I guess I was lucky that neither Apple nor Softbank got a clue problems like that might (and possible ony) happen if you jailbrake the device. I got the unit replaced at Softbank shop.

OK.. My third iPhone was fine and all, but ever after getting my hands on it I noticed that display's glass sort of floats about 0.5mm above the edge of the metal bevel. And since I had my previous 2 units with glass screen on the level with the bevel, I was worried about dust getting inside. But oh well it was my third unit, so I decided just to just be tolerant about this issue. And everything was fine for about a month, I even stopped paying attention to the fact that there was a slight problem with the build quality of my iPhone. Until one day I noticed a white dot on my screen again. Almost right in the middle of the screen, and pretty well annoying this time. "Oh, a dead pixel again?" - I thought. And brought my iPhone to Apple Store, yet again.

After a quick examination, it turned out that the "dead pixel" was actually a speck of dust under the screen glass! How the hell it got there at all, not even telling getting right into the center of the screen, is a little beyond me. However, I think the misaligned screen was the source of the problem.

Anyways, this time Apple Store's genius rejected to replace my iPhone, telling me that the device is not dust proof. It took me about half an hour to finally convince the guy at the counter to replace my iPhone. For the very last time, I was told.

So, since it was my supposedly (and hopefully) last time of needing to replace the unit, I asked to bring me at least 2 devices so I could actually choose the one with the best build quality. In the end, I ended up choosing from 4 units!

I think it's pretty easy to guess why I had to choose from that many units. The build quality just.. I dunno.. not the best:

2 of the units I was presented with had misaligned, floating screens, just like my iPhone with dust speck under the glass screen, another one actually had a dust speck preinstalled under the screen glass - for my convinience I suppose :)

The fourth unit was flawless.. And I really, really hope it stays this way for months and years to come..

So, what's this babble is all about? Have a look yourself.. I touched 7 iPhone units, of which I owned/own four, for period from a few weeks to just over a month. 3 of them had misaligned screens (of which one got dust speck under the screen after a month of usage - and no, I'm not doing extreme sports and such), 1 of the NEW units already had dusk speck under the screen, one unit developed bad pixel after 2 weeks of usage. So, out of 7 units, only 2 were absolutely fine for any prolonged period (if you can call 2-3 weeks a prolonged period) .

I'd say, either Apple needs to probably do something about build quality of the iPhones, or I need to get my luck with the electronic devices back..

We had a call from a client for whom I have built my first "commercial" Rails application. The client was telling the pages on blog part of their site started loading pretty slow.

After some quick investigation, I found out that it was somehow related to the access logging part of the application. Monitoring the SQL requests, I noticed requirest like "select * from accesses where accesses.site_id=1" taking huge time to get processes (well of course..). And I don't remember I was writing selections for all access records for a particular site anyways.

So who (or what?) was doing these huge selections? Turned out it was a pretty harmful code inside the Post class (which represents a blog post):

def log_access()
oneAccess = Access.new()
self.accesses << oneAccess
self.site.accesses << oneAccess
end

This just created a new access record and then attaches it to site, and to the post's accessses list. Well that's fine, and attaching access to the post's accesses list is actually pretty fast, but when it comes to the code:

self.site.accesses << oneAccess

things start getting hairy.. Basically, a post site's accesses are being completely read into memory, and THEN a new access record is attached to the list. And that's what generated the forementioned "select * from accesses where accesses.site_id=1" queries.

Fix is pretty easy of course, now that I know Rails better than few years ago ;)

def log_access()
oneAccess = Access.create(:post_id => self.id, :site_id => self.site_id)
end

So, the lesson is  - always be careful about making things the "smart" way, unless you understand what actually is going under the hood…

I got a new iMac just over a month ago now, and guess what? Its hard disk decided to go to disk heaven or hell much sooner than I expected or then was planned by the Creator.

Basically, I had problems which were looking very much like hard disk failures - my Mac suddenly started freezing when doing spotlight indexes and TimeMachine backups, and after a few reboots, I couldn't make it work for over 10 minutes without freezing at all. Again, it looked very like a hardware failure, but you never know without tests, and I didn't have time to run surface scans that time, so just restored everything from TimeMachine (it rocks!), reformatted my hard disk on "slow" zero-writing mode (hoping that bad sectors will be detected and filtered out).

Well.. probably they did get filtered out, but in just under a week after the first failure, I had another, very similar one. Just this time my Mac stopped booting at all. Well this time I ran surface tests, and both TechTool and Drive Ganius stopped just at the beginning of the disk, unable to proceed, seemingly unable to block bad sectors even..

So, I decided to bring my iMac to my local Apple store, which is Apple Store in Nagoya, Japan. Even though the store is "local", it is still 20km by car (or just above one hour - yeah.. traffic). I thought - "well, its just a little hard disk - Apple Store geniuses should be able to fix stuff like that pretty fast). Man was I wrong..

Apple Store geniuses are actually pretty nice guys - they replaced my first iPhone which had one dead pixel, and almost replaced my second iPhone which I bricked during jailbraking (sorry! just don't tell anyone).

So yeah.. they helpfully offered me to replace my hard drive to free. With a catch - it'll take them 4-5 days, on a fastest scenario. Wha..?! 4-5 days to replace a goddamn harddrive? Probably the next easiest service operation after adding memory? Well yeah, sorry, but we have other customers waiting for their repairs.

Well, that's all cool, by I have several deadlines coming my way and really-really need the goddamn computer working ASAP, which means TODAY. Oh sure, there's one option which can actually shorten waiting time - I was told - just sign up for Apple's ProCare service, and you'll get priority service (no waiting time). Yeah.. very good, I have to get $100 package to get my Mac repaired fast, even though it broke just after 2 week from purchase! (and no, they can't replace the machine, as it was built-to-order - wtf? I just added memory and a hard drive to a stock configuration.. oh well).

So, well, no problems, I won't be happy but I can shell out $100 for ProCare. But wait, they said, we have to check if we have any hard disks for replacement in stock! Oh sorry, we don't have any - we'll order then from "center", and THEN you'll get your ProCare priority service. So.. hard disk should arrive tomorrow, or probably the day after tomorrow. No longer.

Well by that time I was pretty much annoyed and unhappy, so even though I decided to replace the goddamn drive myself (I hold an Apple Service Engineer certificate, though I got it back in the year 2000) - I ordered the drive from Apple Store as well. I dunno.. just in case.

So, without knowing anything about how to replace hard drive in the newer iMacs (early 2008, glass-aluminum model), I got back home at a little past noon, and by 5pm I already had new hard disk churning inside my Mac. I don't think they even can find out that I broke warranty, except for the hard drive I have installed, is not Apple Certified.

That was on Monday. I was happily running after my deadlines for a week, and honestly, sort of forgot about all this nuisance (I called Apple on Tuesday to cancel my hard drive order). Now guess what? On Saturday, almost a WEEK after I ordered the hard drive, I am getting a call from a genius, letting me know that (drumroll!!) my hard drive finally arrived! (thank-you-very-much-but-i-dont-remember-you).

Well.. the morale is.. Apple Store is a mess. When they don't have any replacement, you can wait forever for a basic repairs to be done for you. I mean.. almost a week for a replacement hard drive to just arrive from some "center" (Martian customs didn't give it green light? importing them from Brazilia? I dunno… a WEEK!). And if you're not on Pro Care, good luck waiting for several more days in line, with as unlucky chups as yourself.

I hear here and there that Apple support is top-notch et al.. but you just try and ask them a little more difficult question (did that) and they'll stuck and will only babble that they don't have anything on the topic in their database… Have a hardware failure - and get ready to wait and wait. And wait. Unless you pay for Pro Care. But get ready to wait even then..

Oh well. I'll post my experience replacing early 2008 iMac hard drive as soon as I have some more time. It was actually pretty easy, once I got all the required tools.

I'm getting bombarded with "RHN Errata Alert" messages from Red Hat (our company is a legitimate paid user of their OS) for last THREE days already…..

What the fuck?! Can't they just make a bloody knowledge article and link there? This is just getting annoyed. And am I supposed to update my systems every hour, when a new alert comes? Or should I wait till the flood from Red Hat stops?? Man… what an annoyance… :/

I've been setting up a very simple backup of one site's MySQL database to another server today.

What I needed to be done is to have the MySQL database files to get archived, compressed and transferred to my other server, and be named after the actual backup date and time. And that operation should happen every night.

Here's the full code for those who are in hurry (a cool one-liner heh :), and more detailed explanation of steps taken will follow.

ssh root@remoteserver.example.com 'mysqladmin flush-tables –socket=/tmp/mysql.sock; rm -f remote_db.tar* && tar cf remote_db.tar /var/lib/mysql && bzip2 remote_db.tar' && scp root@remoteserver.example.com:~/remote_db.tar.bz2 /home/mike/remote-backups/`date +%y%m%d_%H%M%S`.tar.bz2

And the explanation.

First, the server I'm backup up from is "remoteserver.example.com" and the forementioned command is invoked from the server I am backing up to. Also, I have my public ssh key installed for root account on the remote server, so I don't need to enter password to log into the remote server. You can read more about setting up SSH keys here.

First step is to log into the remote server, which is achieved simply by running

ssh root@remoteserver.example.com

However, actually instead of logging into the server, I only need to execute some commands on the remote server. This can be accomplished by giving a string of commands to execute as a second (last parameter to ssh command), for example:

ssh root@remoteserver.example.com 'ls -lh'

will give you a listing of files of remote server root's home directory.

So, now that we are connected to the remote server, we actually just need to prepare backup files which we will later transfer from remote to local server. Though the command is actually a one-liner, I'll split the lines for easier understanding, and give them numbers. Also, please note that the command concatenation with the && symbol does the following - it runs next command in chain only of previous command executed successfully.

1. mysqladmin flush-tables –socket=/tmp/mysql.sock
2. rm -f remote_db.tar*
3. tar cf remote_db.tar /var/lib/mysql
4. bzip2 remote_db.tar

#1 flushes mysql tables (so everything that is possibly in memory cache is writted to disk).

#2 removes any previous backup files (that's easy)

#3 this archives the whole MySQL data directory (this path can differ from mine, depending on your installaion parameters!). Also, please note that this approach is NOT SAFE! You can easily get a corrupted backup if any data changes during the archival process. The reason why I'm doing it myself is because I have 100% guarantee that the database will not be updated (the backup is for some inter-corporate system, and I have 100% guarantee that nobody is accessing the database at 3 o'clock in the morning when my backup task is running). You might want to lock tables during backup and unlock them once it is complete. Also, I'm backing everything up this way because I need a drop-in backup - anything happens, and I can just drop the backed up DB in place of the old one.

#4 and the final step is just to compress the backup archive to lessen the time required for transfer. You can compress it in one step actually, using 'tar cjf yourfile.tar ….'. The reason I'm running it in two steps is in order to lessen the time require for creating a database snapshot, so there's even less probability of database being modified during archivation process (tar takes ~5 seconds, tar with bzipping takes almost a minute).

Now, we have the backup file prepared on remote server, and all we need to do is to transfer it to our local server. That's an easy task.

Using the scp command for that task (you can read a little more about it here)

scp root@remoteserver.example.com:~/remote_db.tar.bz2 /home/mike/remote-backups/`date +%y%m%d_%H%M%S`.tar.bz2

There's a little trick here though! I am naming backup files after the date and time I copied them to the local server.

Well that's all. Hopefully somebody find info here helpful :)

PS: Also notice, there is no error checking and notifications if something goes wrong in this script! So feel free to enchance the functionality yourself.

PPS: Oh and I almost forgot! Of course in order to do daily (or hourly, or any periodic backups for that matter), you need to add this one-liner to crontab on your backup server!