Planet Debian Administration

drbd8 lenny

2010-03-15T11:47:00+00:00

Update: They fixed it :) It is safe to upgrade again

Don't upgrade your drbd8 / linux kernel with the current security updates. It will break your setup!

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=573531
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=573829
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=573490

There is a fix pending in proposed updates

On-Line

2010-03-13T19:16:00+00:00

At the moment I'm back on line at home. It's amazing how much you miss not being on-line. Traditional media like the wireless* and TV are only so useful, plus their science and technology coverage is utterly dire...

Anyhow we signed for the new house, and the solicitor will do the contract exchange dance next week (we hope) with completion following a few weeks or so later. Then we'll be off-line again while BT drag their feet on that move...

* Yes I know wireless is a very old word for radio, but I think it's a nicer sounding word.

add a IP on lo:0 ,then all arp packages can't

2010-03-12T03:54:00+00:00

when add a IP on lo:0 ,then all arp packages can't send out from this computer.

e.g.
#ifconfig lo:0 192.168.1.160 up
then
#arp -a
no any entry.
and the "eth0" can't send any arp packages.
why ?
help me , I'm testing LVS-DR cluster.

Dear Lazyweb - Apache Log and slow start

2010-03-08T10:41:00+00:00

Problems restarting Apache on one server.

Receiving a huge amount of traffic due to some Chinese click jacker who can't spell their own domain name on one virtualhost. When Apache restarts it compiles some Perl for FastCGI which causes a delay (and fcgid hits its process limits till the compiling is all done - which is as expected for a busy FastCGI and not usually an issue). But this morning it was all too much (it was running some other stuff as well) the slow response caused Apache to hit MaxClients, and it sat swamped responding to requests very slowly.

The huge Chinese traffic was just getting a 301 redirect. I stopped that virtual server and it all started normally. Since the new error response was larger than the 301, it occurred to me that the change was really to log less. So I disabled logging on the virtual host getting the spurious traffic from China, and suddenly the box has enough horse power to deal with the requests.

I didn't see obvious evidence of disk I/O problems. So anyone see a bottleneck on Apache log files before? Hostname look-up is disabled. The server does hold open a few hundred log files, which is probably not ideal. Didn't see any file handle warnings. The URLs logged are chunky, 3 or 4 line (200+ characters between GET and PROTOCOL).

Failing that anyone who can make money out of lots of horny Chinese people, you can be the destination of that 301 redirect for a small donation ;)

On the other hand this is the first non-IE6 traffic from this click-jacker so presumably he "fixed" his Javascript to work with modern browsers :(

Pptp tracking modules

2010-03-06T11:08:00+00:00

Are you running a multihomed pptpd server?

Be sure you have these modules loaded:

ip_conntrack_ftp
ip_nat_ftp
ip_conntrack_pptp
ip_nat_pptp

Otherwise, you will have weird problems with vpns not coming up right.

If anyone ever figures our how to set the default dns search domain, I would really like to know about it.

How to patch debian package

2010-03-06T04:05:00+00:00

I have downloaded a Debian package source files. And modified some of .c files and added my new (foo.c) plus my pre-compiled file (bar.so) that the new Debian package will need.
what files (rules, make files) I need to change to build the install the package so that the new .so file will include my foo.c and picked up my pre-compiled bar.so file?
Thanks,
jalayari

Permission to access NIC

2010-03-02T13:02:00+00:00

Hi all, Did any one know how to give access to a normal user to access the ethernet device for both read and write?? Is there any way to do that?

Which browser should I choose?

2010-03-02T01:47:00+00:00

With Microsoft being forced to offer a choice of browser to EU citizens it is timely that Microsoft announced yet another security issue with Internet Explorer 8 on the 1st of March which they'll eventually patch.

The answer to which browser you should choose is Firefox.

Why?

It is fast.
It is comparatively secure.
Websites are widely tested with it.
It respects your freedom to know exactly how it works, and what it does with your data.

None of the other browsers meet all four of these criteria, as far as I am aware none of the other browsers meet criteria 4. For a shock try reading the Chrome EULA.

Off-Line

2010-02-24T15:03:00+00:00

We moved house this month and we're still off-line at home. That means I can only email and do web stuff from work, which is much more limited and I don't do as much.

If all all goes well my ISP will hook us up again with ADSL just in time for the next move into the house we intend to buy. Joy of joys! Once we own our own home it's time to get structured CAT-6 installed throughout and build a proper mini-server room...

Icedove gotcha - Emoticons include ^2

2010-02-19T22:22:00+00:00

Icedove, aka Thunderbird, will substitute a superscripted 2 for "^2".

To disable this you need to disable the "Emoticon" feature (Edit -> Preferences -> Display under plain text).

This seems so wrong. My quick attempts to get a full list of "emoticons" (or other ways we **** with your text) failed miserably.

Anyone know if there is a list outside the source code, or which bit of code to read?

MySQL datetime from date, time

2010-02-16T22:16:00+00:00

Say you have a MySQL table called "traffic" representing http requests through a web proxy with a date field (i.e. 2010-02-13) and a time field (i.e. 22:40:23) for each request. Then say you want to select records with a range using a datetime (i.e. 2010-02-13 22:40:23).

Let's say we want to see records for all traffic between 4:30 in the afternoon of January 1st and 5 in the afternoon on January 2nd, a 24 1/2 hour period.

If you write a select statement like this:

SELECT * FROM traffic WHERE date BETWEEN '2010-01-01' AND '2010-01-02' AND time BETWEEN '16:30:00' AND '17:00:00';
7573 rows in set (1.63 sec)

you will get all of the records from January 1st for the half hour between 16:30 and 17:00, and all of the records from January 2nd for the half hour between 16:30 and 17:00.

That's not what we wanted.

However, we can use a function to combine the date and time:

SELECT * FROM traffic WHERE DATE_ADD(date, INTERVAL time HOUR_SECOND) BETWEEN '2010-01-01 16:30:00' AND '2010-01-02 17:00:00';

The DATE_ADD function lets us turn the separate date and time fields into a single datetime field. Yay! It works great in testing with a small sample database, but there is one small caveat:

mysql> SELECT COUNT(*) FROM traffic;
+-----------+
| COUNT(*) |
+-----------+
| 186208916 |
+-----------+
1 row in set (0.00 sec)

That is 186 million rows in the table, representing proxy traffic for the past ten months or so. Going the DATE_ADD route means that MySQL will attempt the conversion on all 186 million rows of the table, and then compare the results to the BETWEEN statement. This takes a LONG TIME. Like a half hour on my hardware. Given that this SELECT statement is used in a cgi script that feeds a web page, a half hour is a bit long to ask the user to wait.

But we can fix this a little:

SELECT * FROM traffic WHERE date BETWEEN '2010-01-01' AND '2010-01-02' AND DATE_ADD(date, INTERVAL time HOUR_SECOND) BETWEEN '2010-01-01 16:30:00' AND '2010-01-02 17:30:00';
146948 rows in set (2.72 sec)

Now MySQL limits the results using the date value, and only does the calculation on the records for January 1st and January 2nd. Much faster at just under three seconds!

But is this really the most efficient way to do this? I can't change the database schema so the individual date and time fields must remain the way they are. What I'm looking for is a way to request a date/time range in a more efficient way than the one above. Any suggestions?

RSyslog RELP and Stunnel

2010-02-07T03:41:00+00:00

This will show you a secure, guaranteed client logging server setup using RSyslog with RELP (Reliable Event Logging Protocol) and Stunnel.

Background:
I have used references (listed below) from the RSyslog site (REF1) and Stunnel site (REF2) and put them together in this quick and easy setup for getting it all to work together. Make sure you do this exactly, do not mix "localhost" with "127.0.0.1" or it will not work.

Here are the main portions we will be working with:
-RSyslog using the RELP protocol provides the means of guaranteeing delivery of system log messages (REF3).

-RSyslog using RELP does not at this time support encryption (REF4), we will add this with Stunnel. Stunnel has many more options for authentication then we will be using here. Please explore the Stunnel faq (REF5) for more information. As it stands this configuration will allow any number of clients to connect to the server, and in uncontrolled environments is not good. It will be left as an exercise to the reader to enforce a limit, so read the Stunnel faq (REF5) for a better understanding how.

-Using the power of RSyslog templates we will separate all log-files out on a client by client, and day by day basis for anyone that reports to the server (REF6).

The chain of communication looks like this:

client_rsyslog_send(127.0.0.1:60514)
<-client-> client_accept_from(127.0.0.1:60514)|client_connect_to(<server_address>:60000)
<-network->
server_accept_from(:60000)|server_connect_to(127.0.0.1:60001)
<-server->
server_rsyslog_listen(127.0.0.1:60001)

###################
## Setup Stunnel ##
###################

## Stunnel4 - Client Setup
Install Stunnel4
%aptitude install stunnel4

Edit the '/etc/default/stunnel4' to start the service on system startup
ENABLED=1

Edit the "/etc/stunnel/stunnel.conf", Make the following changes:
- Comment the line 'cert = xxxxx'
- Remove comment for 'client = yes'
- Comment out the '[pop3s]', '[ssmtp]', and '[imaps]' sections.
- Add the following section, substitute your server address:
# Will accept connections to ports on local
# host and forward them to server
[rsyslog]
accept = 127.0.0.1:60514
# Server the stunnel client will connect to
connect = <server-address>:60000

Restart the Stunnel service:
%/etc/init.d/stunnel4 restart

Check the configuration was successful:
%netstat -aln

This should list 127.0.0.1:60514

## Stunnel - Server Setup
Install Stunnel
%aptitude install stunnel4

Edit the '/etc/default/stunnel4' to start the service on system startup
ENABLED=1

Edit the "/etc/stunnel/stunnel.conf", Make the following changes:
- Comment out the '[pop3s]', '[ssmtp]', and '[imaps]' sections.
- Change 'cert=/etc/stunnel/mail.pem' to 'cert=/etc/stunnel/stunnel.pem'
- Add the following
# Will accept external connections and forward them to the localhost
[ssyslog]
accept = 60000
connect = 127.0.0.1:60001

See the Stunnel faq (mentioned above) for a more detailed explanation of this file. For this EXAMPLE the following will suffice.
Execute the following to create the '/etc/stunnel/stunnel.pem' file:
openssl req -new -x509 -days 365 -nodes -out stunnel.pem -keyout /etc/stunnel/stunnel.pem

Restart the Stunnel service:
%/etc/init.d/stunnel4 restart

Check the configuration was successful:
%netstat -aln

This should list 0.0.0.0:60001, and 0.0.0.0:60000

#######################
### Setup RSyslog ###
#######################

## RSyslog - Client Configuration
To get RELP support we need to add the backports repository for lenny
In "/etc/apt/sources.list", add the following:
deb http://www.backports.org/debian lenny-backports main

Update apt:
%aptitude update

Install rsyslog-relp
%aptitude install rsyslog-relp

In "/etc/rsyslog.conf" add the following lines in the "MODULES" section:
# Load the relp module
$ModLoad omrelp
In "/etc/rsyslog.conf" add the following lines in the "RULES" section:
# Dump all messages to the remote logging server through the localport
*.* :omrelp:127.0.0.1:60514
Restart the RSyslog service
/etc/init.d/rsyslog restart

## RSyslog - Server Configuration
To get RELP support we need to add the backports repository for lenny
In "/etc/apt/sources.list", add the following:
deb http://www.backports.org/debian lenny-backports main

Update apt:
%aptitude update

Install rsyslog-relp
%aptitude install rsyslog-relp

In "/etc/rsyslog.conf" add the following lines in the "MODULES" section:
#Setup the Relp server config
$ModLoad imrelp.so
$InputRELPServerRun 60001

In "/etc/rsyslog.conf" add the following lines in the "RULES" section:
# Receive Logs from the clients
# put them in respective directories in a long format.
$template HostAudit, "/var/log/rsyslog/%HOSTNAME%_%$MONTH%_%$DAY%_%$YEAR%-audit_log"'
*.* ?HostAudit"
Restart the RSyslog service
/etc/init.d/rsyslog restart

To test the setup on the client send a log message:
%logger testing the stunnel-rsyslogrelp setup
On the server in '/var/log/rsyslog/' should be a file with your server name and the current date. Tail it with 'tail -f <file_name>' and watch the log messages populate.

REFERENCES:
(REF1) http://www.rsyslog.com
(REF2) http://www.stunnel.org
(REF3) http://www.rsyslog.com/doc-imrelp.html
(REF4) http://www.rsyslog.com/doc-rsyslog_tls.html
(REF5) http://www.stunnel.org/faq/certs.html
(REF6) http://www.rsyslog.com/doc-rsyslog_conf_templates.html

LVM on software RAID.... trials and tribulations

2010-02-02T18:02:00+00:00

Been working out how I'd like to configure our replacement server's disks.

Given the pain of the last server I'd like to use LVM, and not use the fakeraid controllers built into the servers.

Test server had flat BIOS battery, which means it forgot it was configured to use RAID. The BIOS of the DELL SC1425 writes metadata to disks when configured as RAID, which it doesn't then read when it boots in other modes to check if it should be in RAID mode - hmm - maybe I'm missing something here but it looks like a BIOS battery going flat puts your data at unnecessary risk to me.

Whilst the "dmraid" tools can be used to manipulate some of this metadata, typically the system won't let you erase it once booted. Can anyone explain why to me? Eventually I enabled RAID in the BIOS and deleted the RAID array it then found. Then I reinstalled.

The box also had a prior LVM config. The Debian installer is good at finding and reading any old LVM config on a box, and then not wiping/reseting this information. This got me an "Incorrect metadata area header checksum", which I eventually reverted to the slow but effective "dd if=/dev/zero of=/dev/sdX" to make sure any state information on the disks was truly dead.

Trying to set up LVM on software RAID requires a separate /boot for the installers health at least (I'm not fighting it any harder). Once you configure RAID the installer says it needs (and does need) a reboot so that the installer can take this on board and then configure LVM. There are some related bug reports already, but this seems to be as designed.

Aside from the above it "just worked" ;) This isn't the first time that the Debian installer has tripped me by preserving existing state on disk drives, I'm thinking I should write zero's over all disks before use, tedious as it is.

Twitter results in Google....

2010-02-02T17:14:00+00:00

..... and the first search result I had with Twitter results in was someone suggesting we redirect IE6 users to a certain well known Christmas Island domain name.

Whilst I understand the sentiment, I didn't think it was the most useful second result on the Internet to display for that search term.

Must try harder Google - (Matt Cutts, Marissa Mayer, do you guys have an alert for your own names I wonder?) and I couldn't find a button to immediately switch this spew off.

Paypal RANT

2010-01-29T21:33:00+00:00

Today was too full of Paypal pain.

Amongst other things to activate my PayPal security key, I first had to delete all my Paypal cookies, as otherwise I was only offered the option to activate SMS based security.

But in my guise of being positive here is what I think Paypal needs to do.

1) Stop shuffling customers between websites, pick a domain and stick to it. Don't send me to Paypal.co.uk simply to forward me to paypal.com/uk, and then to paypal-marketing.somewhere else. I care about my online security I have to whitelist these manually for active content, even if I didn't I have to keep an eye on the toolbar to make sure I'm not being phished.

2) Stop advertising "http:"; URLs, use HTTPS only. Why encourage people to visit using an insecure system, when a secure one is only one letter more to type.

3) Stop promoting proprietary email verification and use well established standards for same, not new ones invented by people who couldn't encrypt their spaghetti letter soup. Everyone who understands security is using OpenPGP for email, do catch up it will be simpler in the end.

4) Make it harder to access accounts when a security key is not present.

I have a lot more suggestions but that'll get them started.

You have requested an outdated version of PayPal.

2010-01-29T16:20:00+00:00

I confess, I long for the days when Paypal's search gave results for pages that exist, and contained the word you actually searched for......

TCP weirdness, IMAP, wireshark, and perdition

2010-01-21T19:37:00+00:00

This is the story of a weirdly unfriendly/non-compliant IMAP server, and some nice interactions that arose from a debugging session around it.

Over the holidays, i got to do some computer/network debugging for friends and family. One old friend (I'll call him Fred) had a series of problems i managed to help work through, but was ultimately basically stumped based on the weird behavior of an IMAP server. Here's the details (names of the innocent and guilty have been changed), just in case it helps other folks in at least diagnosing similar situations. the diagnosis The initial symptom was that Fred's computer was "very slow". Sadly, this was a Windows™ machine, so my list of tricks for diagnosing sluggishness is limited. I went through a series of questions, uninstalling things, etc, until we figured it would be better to just have him do his usual work while i watched, kibitzing on what seemed acceptable and what seemed slow. Quite soon, we hit a very specific failure: Fred's Thunderbird installation (version 2, FWIW) was sometimes hanging for a very long period of time during message retrieval. This was not exhaustion of the CPU, disk, RAM, or other local resource. It was pure network delay, and it was a frequent (if unpredictable) frustrating hiccup in his workflow.

One thought i had was Thunderbird's per-server max_cached_connections setting, which can sometimes cause a TB instance to hang if a remote server thinks Thunderbird is being too aggressive. After sorting out why Thunderbird was resetting the values after we'd set them to 0 (grr, thanks for the confusing UI, folks!), we set it to 1, but still had the same occasional, lengthy (about 2 minutes) hang when transfering messages between folders (including the trash folder!), or when reading new messages. Sending mail was quite fast, except for occasional (similarly lengthy) hangs writing the copy to the sent folder. So IMAP was the problem (not SMTP), and the 2-minute timeouts smelled like an issue with the networking layer to me.

At this point, i busted out wireshark, the trusty packet sniffer, which fortunately works as well on Windows as it does on GNU/Linux. Since Fred was doing his IMAP traffic in the clear, i could actually see when and where in the IMAP session the hang was happening. (BTW, Fred's IMAP traffic is no longer in the clear: after all this happened, i switched him to IMAPS (IMAP wrapped in a TLS session), because although the IMAP server in question actually supports the STARTTLS directive, it fails to advertise it in response to the CAPABILITIES query, so Thunderbird refuses to try it. arrgh.)

The basic sequence of Thunderbird's side of an initial IMAP conversation (using plain authentication, anyway) looks something like this:

1 capability
2 login "user" "pass"
3 lsub "" "*"
4 list "" "INBOX"
5 select "INBOX"
6 UID fetch 1:* (FLAGS)

What i found with this server was that if i issued commands 1 through 5, and then left the connection idle for over 5 minutes, then the next command (even if it was just a 6 NOOP or 6 LOGOUT) would cause the IMAP server to issue a TCP reset. No IMAP error message or anything, just a failure at the TCP level. But a nice, fast, responsive failure -- any IMAP client could recover nicely from that by just immediately opening a new connection. I don't mind busy servers killing inactive connections after a reasonable timeout. If it was just this, though, Thunderbird should have continued to be responsive. the deep weirdness But if i issued commands 1 through 6 in rapid succession (the only difference is that extra 6 UID fetch 1:* (FLAGS) command), and then let the connection idle for 5 minutes, then sent the next command: no response of any kind would come from the remote server (not even a TCP ACK or TCP RST). In this circumstance, my client OS's TCP stack would re-send the data repeatedly (staggered at appropriate intervals), until finally the client-side TCP timeout would trigger, and the OS would report the failure to the app, which could turn around and do a simple connection restart to finish up the desired operation. This was the underlying situation causing Fred's Thunderbird client to hang.

In both cases above (with or without the 6th command), the magic window for the idle cutoff was a little more than 300 seconds (5 minutes) of idleness. If the client issued a NOOP at 4 minutes, 45 seconds from the last NOOP, it could keep a connection active indefinitely.

Furthermore, i could replicate the exact same behavior when i used IMAPS -- the state of the IMAP session itself was somehow modifying the TCP session behavior characteristics, whether it was wrapped in a TLS tunnel or not.

One interesting thing about this set of data is that it rules out most common problems in the network connectivity between the two machines. Since none of the hops between the two endpoints know anything about the IMAP state (especially under TLS), and some of the failures are reported properly (e.g. the TCP RST in the 5-command scenario), it's probably safe to say that the various routers, NAT devices, and such were not themselves responsible for the failures.

So what's going on on that IMAP server? The service itself does not announce the flavor of IMAP server, though it does respond to a successful login with You are so in, and to a logout with IMAP server logging out, mate. A bit of digging on the 'net suggests that they are running a perdition IMAP proxy. (clearly written by an Aussie, mate!) But why does it not advertise its STARTTLS capability, even though it is capable? And why do some idle connections end up timing out without so much as an RST, when other idle connections give at least a clean break at the TCP level?

Is there something about issuing the UID command that causes perdition to hand off the connection to some other service, which in turn doesn't do proper TCP error handling? I don't really know anything about the internals of perdition, so i'm just guessing here. the workaround I ultimately recommended to Fred to reduce the number of cached connections to 1, and to set Thunderbird's interval to check for new mail down to 4 minutes. Hopefully, this will keep his one connection active enough that nothing will timeout, and will keep the interference to his workflow to a minimum.

It's an unsatisfactory solution to me, because the behavior of the remote server still seems so non-standard. However, i don't have any sort of control over the remote server, so there's not too much i can do to provide a real fix (other than point the server admins (and perdition developers?) at this writeup).

I don't even know the types of backend server that their perdition proxy is balancing between, so i'm pretty lost for better diagnostics even, let alone a real resolution. some notes I couldn't have figured out the exact details listed above just using Thunderbird on Windows. Fortunately, i had a machine with a decent OS available, and was able to cobble together a fake IMAP client from a couple files (imapstart contained the lines above, and imapfinish contained 8 LOGOUT), bash, and socat.

Here's the bash snippet i used as a fake IMAP client:

spoolout() { while read foo; do sleep 1 && printf "%s\r\n" "$foo" ; done }

( sleep 2 && spoolout < imapstart && sleep 4 && spoolout < imapfinish && sleep 500 ) | socat STDIO TCP4:imap.fubar.example.net:143

To do the test under IMAPS, i just replaced TCP4:imap.fubar.example.net:143 with OPENSSL:imap.fubar.example.net:993.

And of course, i had wireshark handy on the GNU/Linux machine as well, so i could analyze the generated packets over there.

One thing to note about user empowerment: Fred isn't a tech geek, but he can be curious about the technology he relies on if the situation is right. He was with me through the whole process, didn't get antsy, and never tried to get me to "just fix it" while he did something else. I like that, and wish i got to have that kind of interaction more (though i certainly don't begrudge people the time if they do need to get other things done). I was nervous about breaking out wireshark and scaring him off with it, but it turned out it actually was a good conversation starter about what was actually happening on the network, and how IP and TCP traffic worked.

Giving a crash course like that in a quarter of an hour, i can't expect him to retain any concrete specifics, of course. But i think the process was useful in de-mystifying how computers talk to each other somewhat. It's not magic, there are just a lot of finicky pieces that need to fit together a certain way. And Wireshark turned out to be a really nice window into that process, especially when it displays packets during a real-time capture. I usually prefer to do packet captures with tcpdump and analyze them as a non-privileged user afterward for security reasons. But in this case, i felt the positives of user engagement (how often do you get to show someone how their machine actually works?) far outweighed the risks.

As an added bonus, it also helped Fred really understand what i meant when i said that it was a bad idea to use IMAP in the clear. He could actually see his username and password in the network traffic!

This might be worth keeping in mind as an idea for a demonstration for workshops or hacklabs for folks who are curious about networking -- do a live packet capture of the local network, project it, and just start asking questions about it. Wireshark contains such a wealth of obscure packet dissectors (and today's heterogenous public/open networks are so remarkably chatty and filled with weird stuff) that you're bound to run into things that most (or all!) people in the room don't know about, so it could be a good learning activity for groups of all skill levels.

mod-fcgi terminated by calling exit()

2010-01-18T11:32:00+00:00

Error:

[Mon Jan 18 11:24:53 2010] [notice] mod_fcgid: process /.../MyApp/script/MyApp_fastcgi.pl(10514) exit(communication error), terminated by calling exit(), return code: 255

Was caused by changed file permissions on the script MyApp_fastcgi.pl

The simple ones are the hardest to find sometimes.

Profiling Perl Web application under Apache

2010-01-15T13:05:00+00:00

We have a web application written in Catalyst running using mod_fcgid on Lenny which we wish to improve performance on, the folks are #catalyst recommended NYTProf as a profiling tool.

Attempts to persuade the application itself to run under libapache2-mod-perl2 failed miserably (well actually succeeded first time, then failed repeatedly). The behaviour of mod-perl seems to be inconsistent from run to run, with Apache starting or not starting apparently randomly (race condition?).

So set about trying to gather data when it runs under mod_fcgid.

Modified the start of the "scripts/myapp_fastcgi.pl" script to read "#!/usr/bin/perl -w -d:NYTProf", and modified the permissions of "scripts" so that www-data could write the profile data there.

Added "MaxProcessCount 1" to the virtualhost so that only one fcgid process is used. Modified "/etc/apache2/apache.conf" so MaxKeepAliveRequests is 0, as when the process restarts the nytprof.out is overwritten.

Added a method to the Catalyst app that causes the FCGI process to exit cleanly, so that the NYTProf file is not truncated. I'd hoped "kill" of the fcgi process would work here but no joy so far.

The fastcgi process is still restarting occasionally. Any ideas why anyone?

I can add the process ID to the output file if this gets too painful, but it is not so often I can't collect useful data about performance (the fcgid will restart every hour due to configurable timeout, but I don't get anywhere near an hour when collecting profile data before the process restarts).

So far everything points to performance of the YAML libraries as our first bottleneck in this application, although we may simply be reading and saving YAML files more often than we have to.

PHP gettext

2010-01-11T00:19:00+00:00

Tried to use a package (phpesp) that uses PHP gettext support.

There is a comment in the PHP gettext pages that Debian needs package locales-all for this to work.

Now I think:

It just works (i.e. I didn't strictly need to fiddle)

You want "locales-all" because software using gettext almost certainly can/will support locales you don't have installed if you don't.

phpESP tests gettext support with a locale I didn't have installed.

php-gettext is a separate package implementation a PHP based gettext like approach which you shouldn't need because PHP gettext works if you use Debian.

It feels like there is a bug report due when I have to read comments on the PHP website to make stuff work in Debian. Maybe it is a RTP for phpESP that is missing? Guess I'll let you know if we decided to use it in anger.

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=421268

Well trodden paths.....

As someone pointed out at the Devon and Cornwall GNU/Linux user group mentioned Google Docs has a form editor that does much the same thing much more easily, but them my freedom to fiddle would disappear. Still credit to Google for a nice implementation.

Brief updates and some advice

2010-01-07T04:04:00+00:00

So, while Richard is busy doing research and arranging or facilitating several Debian workshops, here I am, gulping down Idlis and Dosas by the dozen. It's been a fun visit, and I got to visit several places in my native state, but since my visit to Mumbai is rather short, I won't be able to say hello to several friends this time (Sorry Kartik). Maybe this would happen only when I am back in India on a permanent basis; let's see...

Here are some general guidelines for travelers which I'd like to share:

They say it doesn't snow in Dallas. Don't believe them. I had to stay overnight at Dallas, and then embark on a trip in the other direction (trans-Pacific) to get home. It wasn't a really fun journey.
If you're ever rerouted due to flight delays, ensure you travel by a similar route, or else, ensure that you're not a vegetarian/vegan. If you fly to Tokyo and Hong Kong, for instance, be sure that you've specified your meal choices well in advance. Of course, you can live on fruits and bread, but you'll really be doing yourself a favour by not missing your flights in the first place; I don't know how I have wronged the weather gods this time.
Ensure that you don't keep too many valuables (which Customs might be suspicious of) in your checked-in baggage. This is important, especially to get through situations where you find that your baggage doesn't arrive with you (my baggage came a day late, and, thankfully, cleared customs quickly since my clothes were duty free, thanks to the many heavy duty cycles they've gone through in washing machines).

It's been a really fun trip, and I am sure I'll enjoy the rest of the trip, till I get back to my old routine and resume work.

Finally, in keeping with the spirit of a blog I try to understand (albeit, in vain, I must accept), I duplicate the concept of a post from that blog, here is an up-to-date list of airports in which I have sat, but never left except by way of airplane:

More updates later. Bye for now.

Brief updates and some advice

2010-01-07T04:01:00+00:00

Here are some general guidelines for travelers which I'd like to share:

They say it doesn't snow in Dallas. Don't believe them. I had to stay overnight at Dallas, and then embark on a trip in the other direction (trans-Pacific) to get home. It wasn't a really fun journey.
If you're ever rerouted due to flight delays, ensure you travel by a similar route, or else, ensure that you're not a vegetarian/vegan. If you fly to Tokyo and Hong Kong, for instance, be sure that you've specified your meal choices well in advance. Of course, you can live on fruits and bread, but you'll really be doing yourself a favour by not missing your flights in the first place; I don't know how I have wronged the weather gods this time.
Ensure that you don't keep too many valuables (which Customs might be suspicious of) in your checked-in baggage. This is important, especially to get through situations where you find that your baggage doesn't arrive with you (my baggage came a day late, and, thankfully, cleared customs quickly since my clothes were duty free, thanks to the many heavy duty cycles they've gone through in washing machines).

It's been a really fun trip, and I am sure I'll enjoy the rest of the trip, till I get back to my old routine and resume work.

More updates later. Bye for now.

Linux/Ubuntu on an Asus UL30A (QX130V)

2010-01-06T03:26:00+00:00

Preliminary note:
I know this web site is actually about 'Debian Administration' so if this article, which also deals with the underlying hardware and linux driver issues, appears offtopic please notify me so I can move it elsewhere.

All information in this article refers to the Asus UL30A-QX130V with BIOS v211.

The first thing I did is what I always do on a new system with Windows preinstalled: install the latest firmware update, since it is convenient to do it this way before replacing Windows by Linux since most vendors still fail to provide BIOS flashing support for linux (actually the only vendor I know to support it on some of their products is DELL) and unsupported ways can fail, especially on laptops (my mom's laptop refused to even power on after flashing its BIOS with coreboots' "flashrom" the other day, ouch).

I ended up installing Ubuntu 9.10 (Karmic) on the UL30A using the Alternate installer. This is not really straight forward since you need to get the Alternate Installer image onto a USB stick first (or setup net booting) and the recommended way to do this - http://www.pendrivelinux.com/create-a-ubuntu-9-10-live-usb-in-windows/ - will not work with the Alternate Installer. However, one of the methods provided at http://help.ubuntu.com/community/USB%20Installation%20Media should work. Alternatively, if you have another Ubuntu desktop, you can use the 'USB start media creator' (usb-creator-gtk) application recent versions provide. The version in Karmic (have not tested other versions of usb-creator-gtk) works fine with the Alternate Installer ISO.

When installing Ubuntu, you should select manual partitioning, since you do not really want a 12G swap partition (WTF!) it would setup by default. You may (or not) also want to preserve the first partition (a hidden partition with an NTFS file system on it) which is the Express Gate partition. If you remove it Express Gate will no longer work - which is fine for me, but your mileage may vary. Apparently Splashtop (the original non-Asus-branded title of this software) is not really installed on a SSD in this case, it is just invoked by a NAND-flash contained boot loader which will print an error message on screen and reboot if you power the laptop up using the left hand side Express Gate button after having removed the Express Gate partition.

Once I had completed the installation, I ended up with a kernel image which was of a later patch level than the modules I had installed. This negatively impacted the loadable kernel modules, i.e. neither ethernet nor wireless worked. A dirty workaround of adding a symlink to the old modules worked around it. A better way is probably to make sure you have the modules installed for the kernel you are using whenever you upgrade it.

I also installed the linux-backport-modules package so that I could have access to the latest modules. I did not actually test whether they are neccessary, though. Most critical stuff (chipset, CPU, graphics, keyboard, NIC, USB) seemed to work fine without them, too.

I finally ended up with this:

# cat /proc/version
Linux version 2.6.31-16-generic (buildd@crested) (gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu8) ) #53-Ubuntu SMP Tue Dec 8 04:02:15 UTC 2009
#

# lspci -nn
00:00.0 Host bridge [0600]: Intel Corporation Mobile 4 Series Chipset Memory Controller Hub [8086:2a40] (rev 07)
00:02.0 VGA compatible controller [0300]: Intel Corporation Mobile 4 Series Chipset Integrated Graphics Controller [8086:2a42] (rev 07)
00:02.1 Display controller [0380]: Intel Corporation Mobile 4 Series Chipset Integrated Graphics Controller [8086:2a43] (rev 07)
00:1a.0 USB Controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 [8086:2937] (rev 03)
00:1a.1 USB Controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 [8086:2938] (rev 03)
00:1a.2 USB Controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6 [8086:2939] (rev 03)
00:1a.7 USB Controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 [8086:293c] (rev 03)
00:1b.0 Audio device [0403]: Intel Corporation 82801I (ICH9 Family) HD Audio Controller [8086:293e] (rev 03)
00:1c.0 PCI bridge [0604]: Intel Corporation 82801I (ICH9 Family) PCI Express Port 1 [8086:2940] (rev 03)
00:1c.1 PCI bridge [0604]: Intel Corporation 82801I (ICH9 Family) PCI Express Port 2 [8086:2942] (rev 03)
00:1c.5 PCI bridge [0604]: Intel Corporation 82801I (ICH9 Family) PCI Express Port 6 [8086:294a] (rev 03)
00:1d.0 USB Controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 [8086:2934] (rev 03)
00:1d.1 USB Controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 [8086:2935] (rev 03)
00:1d.2 USB Controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 [8086:2936] (rev 03)
00:1d.7 USB Controller [0c03]: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 [8086:293a] (rev 03)
00:1e.0 PCI bridge [0604]: Intel Corporation 82801 Mobile PCI Bridge [8086:2448] (rev 93)
00:1f.0 ISA bridge [0601]: Intel Corporation ICH9M-E LPC Interface Controller [8086:2917] (rev 03)
00:1f.2 SATA controller [0106]: Intel Corporation ICH9M/M-E SATA AHCI Controller [8086:2929] (rev 03)
02:00.0 Network controller [0280]: Intel Corporation WiFi Link 100 Series [8086:0083]
03:00.0 Ethernet controller [0200]: Attansic Technology Corp. Atheros AR8132 / L1c Gigabit Ethernet Adapter [1969:1062] (rev c0)
#

# lsusb
Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 002: ID 058f:6366 Alcor Micro Corp. 
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 008 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 003: ID 04f2:b036 Chicony Electronics Co., Ltd 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 002: ID 0b05:1751 ASUSTek Computer, Inc. 
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
#

Everything worked fine out of the box with the following exceptions:

* the Elantech touchpad is incorrectly detected as a Logitech USB mouse: https://bugs.launchpad.net/ubuntu/+bug/418282

* the webcam image is flipped upside down, apparently due to an issue in the lib4l-0 version shipping with Karmic; the Lucid (Ubuntu 10.04) version as available at http://packages.ubuntu.com/lucid/libv4l-0 may fix this, if it does not try to follow the instructions provided by Hans Goede around page 21 of http://ubuntuforums.org/showthread.php?t=838210; a possible workaround (which I have not tested) is also mentioned at http://ubuntuforums.org/showthread.php?p=8256802; some applications (such as 'cheese') also allow for flipping the image upside down

* the SD 5-in-1 card reader is detected but fails to detect any media (I tested with two different SDHC media only), so they cannot be mounted; according to the Asus driver download website this device is an "Alcor AU6433 Card Reader"; the vendor website is at http://www.alcormicro.com/en_content/c_product/product_01b.php?CategoryID=7&IndexID=1; this bug is being tracked at https://bugs.launchpad.net/ubuntu/+bug/366478

* some people seem to have issues with the Bluetooth device not being detected; This can be a BIOS bug which according to the BIOS changelog on the Asus support website has been fixed as of version 209: "Fix the issue that MF210 3G card and Bluetooth cannot be detected." I am currently using BIOS v211 and Bluetooth seems to work out of the box (though I have no other Bluetooth device to test with).

Reviews:
http://www.liliputing.com/2009/10/asus-ul30a-review.html
http://www.laptopmag.com/review/laptops/asus-ul30a.aspx
http://hothardware.com/Articles/Asus-133-UL30A-CULV-Notebook-Review/
http://gryniewicz.com/blogs/dang/2009/12/18/linux-on-asus-ul30a-x5/
http://www.linlap.com/wiki/asus+ul30a

More reviews at tuxmobil.org, linux-on-laptops.com and linuxcompatible.org

Manual:
http://dlcdnet.asus.com/pub/ASUS/nb/UL30A/E4790.zip

Downloads:
http://support.asus.com.tw/download/download.aspx?model=UL30A&os=8

Troubleshooting:
http://vip.asus.com/forum/topic.aspx?board_id=3&model=UL30A
https://launchpad.net/~asus-ul30
http://wiki.ubuntu.com/LaptopTestingTeam/AsusUL30A

Debian HCL entry:
http://kmuto.jp/debian/hcl/ASUS/UL30A-QX130V

Hacking ExpressGate / Splashtop:
http://www.phoronix.com/forums/showthread.php?t=11610

Installing Debian onto an Asus A6Q00KM

2009-12-30T10:51:00+00:00

We're in France at the in-laws. Today I'm installing Debian Lenny onto my mother-in-law-(equivalent)'s Asus notebook.

The first problem was getting the new RAM I bought for her into the unit. There are three possible holes in the base and non of them are marked, but I was lucky and guessed the right one first time. The first DIMM went in okay, the second needed re-seating and some coersion to stay in place and be detected by the BIOS.

The second problem was that the Asus's ACPI BIOS was rubbish and the Debian installer stopped dead in the water. I added acpi=off pci=nommconf and the installer is running happily at the moment. It's pulling most of the files it needs off the 5.0.3 DVD, the rest directly off the fr.debian.org mirror, 14 minutes to go....

Death of a firewall

2009-12-28T11:47:00+00:00

The disk drive overheated, the screen was full of I/O errors and a panic message, but the kernel carried on routing packets for another 2 hours before it succumbed. By which time I was in the computer room, and had built a replacement server just in case the disk drive was dead.

I need a job where I get paid extra for a call out at 03:30 over the holiday period, preferably in extra sleep.

January 2010 Bug-Squashing Party NYC

2009-12-21T20:54:00+00:00

We're going to have a Bug-Squashing Party at the end of January 2010 in New York City. If you live in or around the tri-state area (or want to visit), are interested in learning about the process, meeting other debian folk, or just squashing some bugs in good company, you should come out and join us!

Where:: Brooklyn, New York, USA
When:: January 29th, 30th, and maybe 31st of 2010
Why:: Because them bugs need squashing!

If you plan on coming, please either sign up on the wiki page, or at least mail one of the good folks listed there, or pop into #debian-nyc on irc.oftc.net's IRC network.

dd, netcat, and disk throughput

2009-12-21T06:21:00+00:00

I was trying to dump a large Logical Volume (LV) over ethernet from one machine to another. I found some behavior which surprised me. fun constraints

I have only a fairly minimal debian installation on each machine (which fortunately includes netcat-traditional)
The two machines are connected directly by a single (gigabit) ethernet cable, with no other network connection. So no pulling in extra packages.
I have serial console access to both machines, but no physical access.
The LV being transfered is 973GB in size according to lvs (fairly large, that is), and contains a LUKS volume, which itself contains a basically-full filesystem -- transferring just the "used" bytes is not going to save space/time.
I want to be able to check on how the transfer is doing while it's happening.
I want the LV to show up as an LV on the target system, and don't have tons of extra room on the target to play around with (so no dumping it to the filesystem as a disk image first).

(how do i get myself into these messes?)

This entry has been truncated read the full entry.

Big email providers don't care!

2009-12-21T00:51:00+00:00

3 years ago I wrote a short rant highlighting that many big email providers fail to make even preliminary efforts to improve the deliver-ability of their email.

3 years on I note the situation is now worse. The RFC-Ignorant list I referred to then, now has additional records for hotmail. There are now other widely implemented statistical systems using this list. Thus making it more important to sort these details out.

The topic came onto my radar again when someone testing their own email address hit an issue, one of the contributing factors was a 7 year old listing of their provider (Verizon) on RFC-Ignorant.

If you have a lot of trouble with email being falsely marked as spam, perhaps you are using the wrong provider.

They may be "too big to block", but they aren't "too big" to leave.

Friends don't let other friends use {hotmail|Yahoo!|orange.fr|....}

Getting Debian onto a Novatech V13

2009-12-20T17:39:00+00:00

I decided to get my self a new notebook. Novatech do a V13 unit (Clevo W83T) unit for £340 and it comes naked, so no Windows tax to pay.

It doesn't have a CD/DVD drive so I have to boot it some other way to get the installer going. I tried to PXE boot it, and that works but the Lenny or Squeeze kernels doesn't have the drivers for the JMC25X ethernet card it has so that's not much use at the moment...

I tried an external USB/IDE hard disk. That has loads of space on it but GRUB fails with a Geom problem.

I can get it to boot okay with an ext2 formatted USB key with extlinux, however that kenel panics when it can't find a root filesystem...

I tried a FAT formatted USB key that I used on my dad's Viglen last year and that just gives a blank screen.

I must be doing something silly....

For Hindi Debian users

2009-12-14T03:56:00+00:00

(English translation below. Indian Debian community, please spread the word!)

?????? ?? ????? ???? ???? ???? ???????! ???? ???? ??? ????, ?? ?????? ?? ?? ??? ?? ?????? ?????? ??? ?????? ??, ?? KDE, GNOME ?? ???? ?? ????????? ???? ??? ????? ??? ?????? ???. ?? ?? ????? ?? ?????????????? ????? ?????? ?? ???? ?? ???? ??. ????? ?????? ???????? (Debian Installer) ?? ??????? ?? ?????? Lenny ?? ???? ?? ?? ??? ???? ? ??? ???, ?? ??? ?? ???? ??????. ?? ?? ?? ??? ???? ???, ?? ????? ??? ?????? ?????? ?? ???? ?????, ????? ????? ???. ?????, ??? ?????? ?? ????? ???? ???? ???? ??????? ?? ?????? ???? ??? ??, ??? ???? ??? ??? ??, ?? ????? ?????? ???????? ?? ??????? ?? ?????? ??? ???? ?????? ????. ??? ???? ?? ?? ???????? ???? ?? ??? ??? ?????? ?? ????? ???????????? ?? ???????? ?? ??????? ?? ??????!

?????? ??? ??? ?????? ?? ??? ????? ??? ?????? ???? ?? ???????? ???? ??; ???? ????? ?? ????????? ????? ?? ???? ??? ???? ????? ??? ???? ??????? ????.

???? ??????? ?? ??? ?????? ??????: akumar@??????.???? (???????? ???) ;-)

???????! Translation

Fellow Debian users (who know Hindi)! Maybe you aren't aware yet, but several parts of Debian have been localized and are available in several Indian languages, and KDE and GNOME and several programs are available natively in those languages. This has been possible only due to dedicated volunteers spending time in translating the programs. I have been translating the strings in the Debian Installer to Hindi, since Lenny, and will continue with the translations. But like others, I am short on both time, and my knowledge of technical Hindi words and phrases is limited. So, I'd request fellow users to chip in with some translations to the Debian Installer strings, if they have the time. That way, not only will you earn fame and glory, you'll also earn the gratitude of the Hindi Debian user community.

You don't need to be an expert in the language and grammar to help out with the translations. All you need is some working Hindi knowledge and a bit of time.

For more information, email me at akumar followed by debian.org

Thank you!