Archive for the 'Tech blog' Category

Subversion RPMs for RedHat Linux 7.3

Tuesday, January 22nd, 2008

I have no idea why you would need them, but I just spent the best half of day trying to compile Subversion from the source of some ancient RHL 7.3 machine - the operation which usually takes a few minutes, but I was getting errors no matter what. Luckily, I found RPM repository which contained everything I needed to install Subversion of that ancient RedHat machine.Here's the link:http://summersoft.fay.ar.us/pub/subversion/1.1.4/redhat-7.x/bin/ 

SA's best X'Mas present

Friday, December 21st, 2007

.. is a new server of course :)

Though I'm not much of a full-fledged SA, and just have to do it because nobody else in the company where I work at, can.. Still, there was a very warm and fuzzy feeling when I was unpacking our new Dell PowerEdge 1900 servers (well, only one of them because I'll do setup next month).

These actually are the first "real" servers which I'll be setting up and administering myself (there were lots of built-it-yourself servers before.. I wonder how they match against the "real" ones)

I'll be setting up separate application and database servers.. The specs are as follows:

Application: Intel Xeon Quad 2.33MHz, 8Gb RAM, RAID10 (6×250Gb), DRAC5 remote access card, Intel's Dual 1Gb Ethernet, RedHat 4.5

Database: Intel Xeon Quad 1.86MHz, 4Gb RAM, RAID10 (6×250Gb), DRAC5, Intel Dual 1Gbit Ethernet, RedHat 4.5

 

Well we don't yet even have a server room, closet, or anything.. Lots of things to setup, even more stuff to learn.

Rails active_record_store & Segmentation fault

Wednesday, December 19th, 2007

Here's a quick tip if you're getting errors like this one in your Rails application:

/usr/local/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/session/active_record_store.rb:84: [BUG] Segmentation fault

The most probable reason you're getting the Segmentation fault and your server crashes is because you're trying to store too much data in sessions table of your application (I assume you are using active_record_store as a session store).

The problem with "too much data" is that by default, the session table creation rake task created the following migration in Rails 1.x:

class AddSessions < ActiveRecord::Migration
def self.up
create_table :sessions do |t|
t.column :session_id, :string
t.column :data, :text
t.column :updated_at, :datetime
end
add_index :sessions, :session_id
end

def self.down
drop_table :sessions
end
end

Please note the data field defined as text. This means that it can only store up to 64Kb of data. And that also means that if you're trying to store more than 64Kb in your session.

In order to fix the problem, you just need to manually change the column type before you run migration which creates session store, or just create a new migration which changes parameters of the data column in existing sessions table:

Should look something like that (Rails 2 syntax):

class CreateSessions < ActiveRecord::Migration
def self.up
drop_table :sessions

create_table :sessions do |t|
t.string :session_id, :null => false
t.column :data, :binary, :limit => 10.megabyte
t.timestamps
end

add_index :sessions, :session_id
add_index :sessions, :updated_at
end

def self.down
drop_table :sessions
end
end

Empty your sessions table, restart your server and you're done. No more segmentation faults. Of course you shouldn't store that much data in a session in the first place, but well.. things happen. Now I'm off to fix my code which stores Megs of data in a session store ;)

Function output caching with PHP

Monday, December 10th, 2007

I've been working on optimizing of a system written in PHP, which started to perform slowly, and I had to implement content caching so I could, say, calculate access stats once a day and then just output cached content to the user, or have top page regenerated every 15 minutes.. Stuff like that.

I thought I'd share my approach (which may not be the best around, but is a working one and can save somebody some time). So here goes….

First, we need to create a table to store cached information (I store cached information as well as system-wide settings in that table)

CREATE TABLE `settings` (
`key` varchar(250) NOT NULL default ",
`data` text NOT NULL,
PRIMARY KEY (`key`)
)

Next, we need accessors to the data in the settings table (just put these functions somewhere accessible by your code):

function settingsGet($key)
{
$key = addslashes($key);
$qstring = "select * from settings where `key`='$key'";
$result = queryDB($qstring);
$stuff = mysql_fetch_array($result);
return $stuff[data];
}
function settingsSet($key, $value)
{
$key = addslashes($key);
$value = addslashes($value);

$qstring = "select * from settings where `key`='$key'";
$result = queryDB($qstring);
if (mysql_num_rows($result)>0) {
//updating
$qstring = "update settings set data='$value' where `key`='$key'";
} else {
//inserting new
$qstring = "insert into settings (`key`, data) values ('$key', '$value')";
}
queryDB($qstring);
}

Next, we'll setup function to get and set cached data, which in turn use settings table accessors we created above. Again, just place them anywhere accessible by your code.

function get_cachedData($data_id, $expireSeconds = 3600) {
//expiring every hour by default

$data = unserialize(settingsGet("data_cache_{$data_id}"));

$lag = time() - $data["ts"];

if ($lag > $expireSeconds) {
return false;
}
return $data[value];
}
function set_cachedData($data_id, $value) {
$ts = time();
$data = array();
$data["ts"] = $ts;
$data["value"] = $value;
settingsSet("data_cache_{$data_id}", serialize($data));
}

Basically, we just use (hard-coded) prefix "data_cache_" to store data in the settings table (useful if you store something else in the settings table as well), as well as "data_id" variable by which you identify the "name" or id of data you wish to cache.

We will also need some nice wrappers:

function startCaching($data_id) {
if ($data = get_cachedData($data_id)) {
echo $data; return true;
}
ob_start();
return false;
}

function endCaching($data_id) {
$data = ob_get_contents();
ob_end_clean();
set_cachedData($data_id, $data);
echo $data;
}

Usage example:

Before caching:

function print_access_rating() {
/*
some code which requires a lot of time to get generated
*/
}

After caching was added:

function print_access_rating() {
if (startCaching("xxxxxxxx)) return;
/*
some code which requires a lot of time to get generated
*/
endCaching("xxxxxxxx");
}

The "xxxxxxxx" would identify the chunk of data you wish to cache. Say, you want to cache latest news section on your top page. For the forementioned usage example you can use print_access_rating as the data_id parameter. Or you can just pass the __FUNCTION__ php 's variable which is always set to the name of current function, if you want.

That's basically all. Please notice that you can use optional second argument of get_cachedData function to set number of seconds for which to cache a particular chunk of data.

Don't know how usable it will be for anyone except for myself, but here it is.. :) Modify to suit your needs.

Rails 2.0 is out so…

Friday, December 7th, 2007

..just kill me because I absolutely have no time to dig into it doh! For these lucky who do have time, follow here Somebody stop the planet and give me a couple of months of rest with just me and the computer (and the Internet, of course) :) 

Turn off / tune logger level in Rails

Friday, November 9th, 2007

If you haven't noticed already, Rails-based application's logs can grow up pretty fast with default settings even in production mode. I just have one system which generated about 500Megs of production.log in just about a month. This is not a very good situation when you're on a shared host with limited hard disk space. So unless you are totally anal about what's going when on your production server (and well.. I don't know why you should be, since stuff is supposed to be at least partially stable when you actually deploy it) - you might want to limit logging, or eliminate it completely. Here's how.By default, logging level in production mode is set to :info, meaning every request will be logged, and that's not your Apache log. A single entry is something like that:

Processing FrontController#news (for 124.110.11.17 at 2007-11-09 17:55:11) [GET]  Session ID: f14f25c56d59f278194146955357c198  Parameters: {"action"=>"news", "id"=>"249", "controller"=>"front"}Cookie set: visitor_id=177; path=/; expires=Sat, 08 Nov 2008 14:55:11 GMTRendering  within layouts/kyoroman/frontRendering front/newsCompleted in 0.56727 (1 reqs/sec) | Rendering: 0.29647 (52%) | DB: 0.00000 (0%) | 200 OK [http://kyo-roman.com/news/249]  

Pretty long for a single request huh? And do you really really need this info in your logs?If not, just open your config/production.rb file and add the following line:

 config.log_level = :error  

This way only errors will be logged. You can use :fatal instead of :error and have even fewer stuff being logged.But still, even with :fatal settings, SOME stuff will be logged, and this stuff are really fatal dumps like:

ActionController::UnknownAction (No action responded to unknown_request):    /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.13.2/lib/action_controller/filters.rb:632:in `call_filter'    /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.13.2/lib/action_controller/filters.rb:638:in `call_filter'    /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.13.2/lib/action_controller/filters.rb:438:in `call'    /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.13.2/lib/action_controller/filters.rb:637:in `call_filter'    /usr/local/lib/ruby/gems/1.8/gems/actionpack-1.13.2/lib/action_controller/filters.rb:638:in `call_filter'  

…………………. many many lines ……………… 

So you might want to just disable the whole logging totally. In a hasta-la-vista-baby approach. In order to do that, just add the following 2 lines to the end of your environment.rb file:

ActiveRecord::Base.logger = nil

ActionController::Base.logger = nil 

 

 And that's it. You'll get zero-sized logs. Just remember - ALL logs will be empty, development, production and testing :)  Now.. I think there might be a better way to control logging, so if anyone knows a better way please share it in the comments. 

Why some cron jobs fails to execute

Friday, November 2nd, 2007

There's one simple thing every system administrator should take into account, when settings up cron jobs, and which I - to my shame - didn't know until now.

When you setup a cron job, especially for a root user, I guess, your command runs in a very limited environment (basically, not paths and other variables set). So if you have a command in /usr/local/bin directory and can "test" it executes just fine from command line - it doesn't mean the same exact command will be executed successfully when running as a cron job!

Same goes for setting up and "testing" cron jobs with Webmin. You can click the "Run Now" button in Webmin's cron jobs manager, and the command will run just fine, generating output you expect - but when running as a cron tab job, your command might NOT work at all.

The problem, as outlined above, is that not all paths you expect to be set are actually set for cron jobs.

How to confirm?

You can just read root's email (or cron job owner's email). In terminal just type "mail" and then enter a number of message you want to read. Here's the output for a problematic setup:

& 5000
Message 5000:
From root@………  Thu Nov  1 00:02:01 2007
Date: Thu, 1 Nov 2007 00:02:01 +0900
From: root@……….. (Cron Daemon)
To: root@……….
Subject: Cron <root@www> monit validate  #relaunches monit if needed
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>
X-Cron-Env: <USER=root>

/bin/sh: monit: command not found

The solution? Just specify full path to the command!

In my case, monit is located in  /usr/local/bin/ directory, which clearly is not defined in the PATH. So, it's /usr/local/bin/monit validate and you're set.

So, use full paths to your commands, and inside shell scrips (or set paths), and don't really trust Webmin' Cron Run Now command.

Rails and readline problems on Mac OS X

Friday, October 19th, 2007

If you build Ruby on Mac OS X yourself, you may face the following problem once you have downloaded the language's source from ruby-lang.org, compiled/installed it and tried to run your Rails app:

 mike$ script/console

Loading development environment.

dyld: NSLinkModule() error

dyld: Symbol not found: _rl_filename_completion_function

  Referenced from: /usr/local/lib/ruby/1.8/i686-darwin9.0.0b5/readline.bundle

  Expected in: flat namespace

 

Trace/BPT trap 

 

 In order to fix the problem, you need to re-build readline from the source and then rebuild Ruby, linking it to the new readline's installation dir. Here's how you do it:

wget ftp://ftp.gnu.org/gnu/readline/readline-5.1.tar.gz 

tar -xzf readline-5.1.tar.gz

cd readline-5.1

./configure –prefix=/usr/local && make && sudo make install

 

and then rebuild Ruby (you do it in your Ruby source directory of course)

make distclean && configure –with-readline-dir=/usr/local && make && make install   

Once done, everything should be back on track. 

New portal goes online

Tuesday, October 2nd, 2007

Being in development (mostly contents part, not the system itself) for a several months, a kimono-company portal which uses the multi-site system I have developed, have finally went live today :)

The Kyo-roman portal is a portal system for a company which sells kimonos here in Japan - the headquarters and filials (and that's 17 of them!) each got own web-site, with blog and news/events periodicals, centralized articles approval solution and other perks like visual editor and video uploads. Latest news from filial sites are all gathered on the top page of the headquarters' site (hey, it's a portal site, after all :).

Currently, sites of company's filials are pretty minimalistic, but hopefully that will change in the future (it's up to users to add new content from now on).

The system uses Ruby on Rails as development system (specifically, version early version of my Rails-based multi-site system) and is deployed on CentOS-based hosting, Apache 2 + Mongrel cluster.

Well I know this might sound pretty boring (bah.. ANOTHER system online, so what?), but this is actually the first large-scale deployment of the system I've been working on for over a year, so well.. this is kinda exciting for me :)

One more Rails project online

Thursday, September 20th, 2007

I have another project went live.

RightViewPro is a site for baseball fans, selling some pretty interesting baseball systems. The site is written completely in Rails, features rich admin interface, fully customizable thru admin panel, and features online shop with physical and downloadable products.

Cheers goes to WindUp Design team with whom I was working on the project.