30.01.2008

Generating daily usage statistics

Sometimes you'll want to know how many logins there're on your web site and how the daily usage profile looks like. Usually the first attempt to evaluate this is to copy some log files or unload some database table which holds the login information and then import it to Excel or a similar spreadsheet generator.

But in many cases it can be done by the database directly. Here's a DB2 example. Assuming there's a table "events.logins" holding the login events with a date and a time column, then this query would bring up a daily usage profile based on the average data of the last 7 days:

select
tmptable.hr as Hour_of_day,
tmptable.cnt as Num_Logins,
substr(
repeat('#', tmptable.cnt * 60 / (

select
max( maxquery.cnt ) as max
from (

select
hour(time) as hr,
count(hour(time)) as cnt
from
events.logins
where
date >= (current_date - 7 days)
group by
hour(time)
) maxquery

)
),1,60) as Logins_Profile
from (

select
hour(time) as hr,
count(hour(time)) as cnt
from
events.logins
where
date >= (current_date - 7 days)
group by
hour(time)

) tmptable
order by
tmptable.hr
;

This query includes two temporary tables. The upper one is used to evaluate the maximum logins per hour, which is necessary to normalize all counts. The second is generating the raw data, means the hour of the day and the number of logins. The result looks like this:

HOUR_OF_DAY NUM_LOGINS  LOGINS_PROFILE
----------- ----------- ------------------------------------------------------------
0 36 ##
1 43 ##
2 52 ###
3 55 ###
4 61 ####
5 73 ####
6 119 ########
7 294 ####################
8 625 ##########################################
9 846 #########################################################
10 880 ############################################################
11 812 #######################################################
12 663 #############################################
13 673 #############################################
14 685 ##############################################
15 662 #############################################
16 548 #####################################
17 359 ########################
18 190 #############
19 123 ########
20 112 #######
21 87 #####
22 75 #####
23 48 ###

24 record(s) selected.

Diese Seite zu Mister Wong hinzufügen

11.01.2008

Spam vs. CO2

Some are discussing CO2-free email services, means they'd like to power their servers with electricity from regenerative sources. The goal is to bring this to market as a product called "green email" or similar. But can an email be CO2-free at all? It's not only the mail server of the sender but also that of the recipient plus dozens of relays, routers, switches and other systems in between. They all would have to be powered by regenerative sources - a very ambitious aim.

But there's another aim in this. Reducing greenhouse gas emissions by changes in the email system is possible - by reducing spam. Latest articles are saying that about 90% of all emails sent world wide are unwanted, mostly spam. So here lies a big potential of saving resources. Fight spam and save the world a little bit.

Diese Seite zu Mister Wong hinzufügen

03.01.2008

Green IT

... is one of the current buzz words in IT business. Every hardware vendor is offering some special to help his customers reducing their CO2 emissions and thus saving the planet. Where the saying originally was "sex sells", in the digital world it became "green IT sells".

Even in view of so much heroism the fact must be taken in doubt that the world is full of modern, environmental Clark Kents. Although in a poll of the german IT magazine Computerwoche 42% said their motivation was to protect the environment, in my opinion this is just a desirable side effect. At least 32% probably told the truth and said the reason for their engagement in green IT was to reduce the costs of operation (electricity for operation and cooling etc.).

So in the end one could modify another saying: Money makes the world go round - and helps saving it.

Diese Seite zu Mister Wong hinzufügen

01.01.2008

Final shutdown

Recently we had to abandon an external Linux system which was operated by a hosting provider. Although just about five years old and still fine for our requirements, the provider wanted to take the hardware out of service. Of course we wanted to make sure that the system, once given back, was clean and none of our data could be exposed to third parties.

Our team named this project the "Operation_Regenbogen" (operation rainbow) which originally in the last days of WWII was an order to all german U-boats to sink their vessels. In some ways our externally hosted Linux system was an U-boat. For years it has been doing its service far away from home, silently, realiable, with communications encrypted. And now it had to be made sure that its secrets are kept. But how to do this properly on a server you don't have a direct attached console to?

First, we unmounted all the unnecessary partitions and cleared them with "dd if=/dev/zero of=/dev/hdaX bs=4M". After switching off swap and clearing the swap partition the same way, all unneeded daemons were stopped and all TCP ports closed - except ssh of course. Then came the difficult part: Would a dd from zero device to hda device work?

We decided for the safe way. After cleaning up the remaining filesystems / and /boot, especially the /var subfolders, we first cleared the free space by filling a dummy file with zeros: dd if=/dev/zero of=/dummyfile bs=1M ; rm /dummyfile . Then we gave the order to sink the boat with a "dd if=/dev/zero of=/dev/hda bs=4M".

While pinging the server, we waited for things to happen. Then some time later, actually the prompt came back. So the boat was inoperable but still afloat. Via the last ssh terminal we could still run some simple commands like "ls" which of course resulted in an I/O error. All data on harddisk was zero, no secrets left to be compromised.

In the end we tried to shutdown the system but failed. "shutdown" and "halt" just produced an "I/O error" message, and so did an "init". When we tried to start a "kill" command, something dropped the ssh connection at last. Finally we were locked out and lost control of the system.

Our pings still have been answered for four days. On the fifth day, the hardware was retired and somebody had switched off the server. The boat had gone, now lying on the ground of the sea of oblivion.

Diese Seite zu Mister Wong hinzufügen