Tuesday, June 27, 2006

RSS Support for Neengine

Yay!!! I just completed validating the RSS generated by Neengine,
therefore Neengine now had RSS Feed support. You'll notice the RSS feed
icon now if you use a real browser (*hint* Firefox).

Also, Thelda now properly mirrors to a specified blogger.com url. I'm
thinking of writing an e17 module that will call thelda and periodically
post blogs from a folder to Neengine... hmmmm... I'll see about that
(seems essienitaessien.com is becoming a technology franchise :)

Anyways, I don't think I should be repeating any RSS Howto here, so I'll
just point to some of the many fine articles available if you want to
know how to do this too:

<ul>
<li><a
href="http://www.peterfreitag.com/item/465.cfm">http://www.peterfreitag.com</a></li>
<li><a
href="http://www.kbcafe.com/rss/rssfeedstate.html">http://www.kbcafe.com/rss/rssfeedstate.html</a></li>
</ul>

The first of the links is a quick primer and you can get up and running
in no time. The second one contains some more information that is
interesting to know.

Oh, and I quickly learned that you REALLY WANT TO VALIDATE your feed.
You can do this over at <a href="http://feedvalidator.org">Feed
Validator</a>

Ok... with that out of the way, I think I to really work on the comment
spam validation next. This means, I have to implement or borrow a <a
href="http://en.wikipedia.org/wiki/Captcha">CAPTCHA</a> implementation.
That's all that's really holding Neengine back from being ready for a
v1.0 I think. Anyways, I'll have to work on that this week.

In other news, I'm beginning work on a Forums module. I will probably be
testing it here on essienitaessien.com, but its to be used in client web
communities.

Time to go clean-up the soul... aka... reading C code and possibly
refactoring.

return;

Friday, June 23, 2006

My kingdom for an SMS Port

I'm currently awake, putting the finishing touches to the MIDP
application I've been working on since last week. I posted last week
about the rapid speed with which I quickly whipped up a 90% working
demo. At that point, what was left was finally sending collected
information via SMS to a remote application server listening via SMS.

To do this, you'd use the J2ME Wireless Messaging API (WMA). The api
seems rather simple and I looked at the code in the WMA Demo example
that comes with the j2me Wireless Tool Kit. I adapted the main sending
functionality there into a function of mine to send text... and that's
where my problems began (I know that in retrospect).

Apparently, according to the SMS specification, more than one SMS
application can recieve at the same phone number simoultaneously, just
like TCP/IP sockets. To differentiate the different applications, the
sms url addressing scheme looks just like the usual http scheme we're
all used to, complete with the port numbers, so I can send an SMS to a
phone number at sms://+123456:10 . When the SMS gets to the phone, an
application listening at sms port 10 will recieve this. All well and
good it seems, but not so for me... when I suddenly discovered that SMS
allows for port numbers, I started wondering what would happen if I
don't specify the port number? Or which port the default sms recipient
application listens at? Basically, b/cos I knew that in HTTP, not
specifying a port number means you're specifying port 80, I got worried
and tried to find out. I didn't find this out directly, instead what I
found out was that if your app is LISTENING, and it couldn't careless
about the port the message comes in on, it should be set to listed at
port 0.

Armed with this half baked information, I went on to assume that sending
to port 0 then should be okay, if you're not interested in sending to
any particular port! Well, needless to say, my first version didn't
work. I didn't get an exception or anything, the application just
behaved as if it were sending, and quit rapidly, but nothing happened.
On the Sony-Ericsson phone I was initially testing on, the phone kept
asking me if I wanted the application to RECIEVE messages! This was odd,
since i was trying to SEND, anyways, I just thought the phone was being
stupid and that all was ok, and I went on... it didn't work.

I decided to try on a Nokia 6600, but when I tried to move the jar over
via bluetooth, it didn't even install. In my "infinite" wisdom ;), I
believed we were having problems with code signing (hilarious huh?).
Anyways, I convinced my boss to start the process of obtaining a
verisign certificate last week, and I went on to relax and wait.

By Monday this week, I realized the certificate would probably not be
getting to us b4 Wednesday, when I was due for a second client demo. On
tuesday then, I decided to go stay over with some of my friends (Bugzy,
Nnamo, ROM and co :P) who also write j2me apps on and off. That nite,
Bugzy and Myself first tried to deploy his j2me apps on another 6600 and
they deployed... that was when I realized the Nokia issue was due to the
MIDP version. I had been building for MIDP2.0 all the while, so I
changed my setting to build for MIDP 1.0, CLDC 1.0 with WMA 1.0, and
rebuilt the app, fixing what needed fixing, then I deployed to the Nokia
properly... and boom!!! It worked!!! The sms left the Nokia phone... we
were ecstatic.

We then deployed to a sony-ericsson and met disappointment... it didn't
send. After some tinkering, we changed the port number from 0 to 1, just
for the sake of trying something different... and w00t!!! It worked!!!
PHEW! I tidied up the code for the demo and went for the demo yesterday.
It went well BUT there was a problem, the SMS message arrived at the
application with some 8 extra bytes infront of the commands we were
sending! Incidentally, sending the message to another phone, there were
no extraneous bytes... so what could be wrong :(.

I was just frustrated and annoyed, but hey... I had to figure it out. I
started scouring materials, and probably read over 10 WMA tutorials
today, that all said the same thing, and who's example code looked
exactly like my current code, though they were all using port 5000. I
decided to use port 5000, but nada... no show. Very depressing. Finally,
my collegue at the office, Lanre suggested we try to setup NowSMS on his
box, since he has a trial version and see if that could help. Grudgingly
I agreed.

10 minutes latter, we have NowSMS listening for messages, and I send a
command from the application, and check the incoming messages to NowSMS
and what do I see? THE SMS COMES IN AS A BINARY MESSAGE AND NOT AS A
TEXT MESSAGE!!!

That is sooooo suprisingly amazing... but finally I at least know why
the bytes. I try to read thru the specs to see where I may have made a
mistake, and finally, I come to a conclusion that J2ME WMA
implementation has a bug, since I had specified
MessageConnection.TEXT_MESSAGE!!! Finally, I do what I do sometimes when
all logical explanations have flown out the window... I deleted the sms
sending function I had, and proceeded to write another one. I just
cleared the page, and without thinking of my former implementation I
wrote a smaller one, and just tested again... and BOOM!!! IT WORKS!!! I
see the messages into NowSMS come in as TEXT and not as BINARY!!!
Amazing... I look thru my former code and the new one... and the
difference? Nothing... hey.. then why is this worki... uhh... wait... in
my new code, I FORGOT to add port numbers!!!!! You mean...? Yup. The
problem was the port numbers I was adding. I SHOULD NOT have added them
AT ALL. Infact, if I hadn't added them at all, the very first time I
tested my app on the sonyericsson, it would have worked!!!!!

Well, needless to say, this was an anti-climax. I was just too relieved
and rushed to do the demo, and everything works well finally. I'm
currently awake putting finishing touches to the look, input
verification, etc.

The moral lesson of this long twisted story? Uhhh... I'm not too sure
what that should be, but I guess "when all fails, do something
illogical." is not a bad moral lesson :)

Seriously though, I still don't know why I hadn't tried without port
numbers loooooooooooooong ago. Now at least, I know better, and I'm
putting this here incase someone else has the same problem.

Anyways... njoi

OLPC board in da haus!

Two days ago Khaled of the OLPC project was at our office. They're in
Nigeria trying to wrap up the arrangement with the Nigerian government.
He came with some friends from Alteq (Alteq is an IT company in Nigeria
operating out of Abuja, who also use Open Source software end-to-end to
provide their solutions - Really cool guys).

Anyways, Khaled came to drop one of the prototype boards with me, so I
can play around with it. I immidietly transfered one of the older OLPC
version of fedora I had lying about onto a USB drive and tried to boot
off of the board, but I got a GRUB error... meh!!! I'be been busy with
my MIDP app, so no time to tackle it squarely yet. I should be done with
the darn MIDP application tomorrow anyways, so I should be a bit free to
tackle the board.

Its very small and cute. I should post images here... as soon as I work
in image or the gallery section of essienitaessien.com.

Ahhh well... we had some very interesting conversations and some really
cool ideas flowing back and forth... its always that way when open
source folks meet to talk. You'd think that coming from different parts
of the world, we'd have little to crack wit about... but you'd be sooooo
wrong. Its amazing, but at least for IT/geek types, who spend time on
newsgroups/irc/forums, there has emerged a kind of sub-culture which is
immedietly evident once they meet and start conversing. We really had
fun... its a pity they had to head back to Abuja that day. Khaled should
be swinging by Lagos tommorrow or Saturday probably, so we should be
able to catch some drinks this evening without the pressure of work
deadlines :)

I'll err.. try to *hic* keep everybody *hic* posteddddd *hic*

:P

Monday, June 12, 2006

Neengine Update - Fun With Cheetah Templates

Finally, Neengine is looking cleaner and more visually palatable. Thanks
to a few hours on Googling on HTML formatting with Cheetah Templates.
I've been mulling moving over to templetor which seems will come by
default in webpy 2.0, but the '4 space' indentation thing actually irks
me. I can understand it in Python as a language, but as a hard
requirement in a templating system, its a bit irritating. The real
problem though is that right now, the errors caused by getting
indentation wrong are not accurately reported, and will probably lead
you to frustratingly look at other things. Since I'm not motivated to
hack that area, I'll just wait for Aaron to fix it, before I evaluate it
again :D

Anyways, I got to find out that Cheetah can control how special
characters in its place holders are treated, using something called
Output Filtering. With output filtering, you can either specify an
output filter as an argument to the rendering function, or you can do it
in the Template itself, using the #filter directive to cheetah.

By default, web.py treats HTML tags specially, hence rendering them
literally, instead of treating them as HTML. But using the two
directives: #filter ReplaceNone, #filter WebSafe, you can turn this
behaviour on and off. With the 'ReplaceNone' filter, you switch to
Cheetah's default behaviour, which is to only treat Python's 'None' as a
special character, replacing it with an empty space, every other thing
is rendered as is, so HTML tags will be rendered as HTML tags. With the
'WebSafe' filter, all characters that that can have special meanings on
the Web are escaped so as to be rendered visually literal.

I currently switch between these two when rendering Neengine's HTML
contents. Ofcourse, this is after turning linebreaks into and HTML break
tag. Currently, I still have to add transparent support for embedding
HTML into the posts, so it is escaped intelligently. I'm not even going
to think of how I'll do that now, till probably later in the week (if
i'm in the mood).

With that, Neengine is a lot more mature now. The only critical things
left now before I can declare a version 1.0 Neengine is Anti-Spam
validation for comments, so that I can enable back the comments section,
and RSS feed support. Other features like embedding images in posts will
probably come in a major version bump, since this will likely break the
database schema.

For now though, I'm off to hack thin client images :)

*enjoi*

Friday, June 09, 2006

Unix Network Filesystems Suprise

Ever since I started work on this Thin Client distro, I've been kinda
skeptical about the kind of application performance I would get.

I decided to ignore it and work on and cross over the speed barrier when
and if it hits me in the face, and I was really expecting a huge punch.
Coupled with the fact that I was testing the distro out with a USB
bootable version of it, everything was just slow and ungainly.

The first time I tried to mount the network filesystem via samba on the
USB based distro, and run OpenOffice off of it, I almost passed out just
from the time I was waiting for openoffice to show me the damn splash
screen!!! Well... that really discouraged me, but hey... I went on to
put stuff together.

As at today, I just had to test that baby on an IDE device, so I
hijacked an old harddrive, dd'd my image over to it and proceeded to
boot off of it. The first thing I decided to try was Opera (which our
client really wants), and WOW HOT DAMN!!!! I was shocked. The thing was
blazing fast!!! Just to see how Opera running off of a networked
filesystem performed against a networked Firefox as well, I went ahead
to run Firefox too... and tho it was slower to start up than Opera, I
was also impressed.

I went on to run some other applications like QT Designer which I was
expecting to take forever and a half to start... and my o my... It
displayed the splash screen almost immedietly!!!

Finally, I went to try OpenOffice and it took like 25 seconds to show me
the splash screen (which is basically about the same time that MS Office
apps take to show me their splash screen on a Windows Box!!!) Well, at
the end of today... I'm really tripped about Network Filesystem
performances. I actually wrote an initscript helper app 'wltcfsd'
(Wazobia Linux Thin Client FileSystem Daemon) to make configuring and
persisting my network filesystem settings on the thin client easier.

I can say... I'm really loving thin clients. Infact, i'm loving it so
much I'm thinking of setting up all the translators to run off thin
clients at the office :P

Also, I've been thinking of clustering too... check the scenario... Thin
clients, connecting to a huge cluster over a 1000baseT network or
something... WOW!!!! That there is the power of linux.

Ahhh... now I have to go become a couch potatoe... I don't think i'm
partying this weekend. I have me 24 on DVD, I have to finally add
intelligent formatting to Neengine, and I have some refatoring patch I
was thinking about for e17 modules gadcon clients.

Seems I'll just be coding thruout this weekend.

t'sall good

Thin Client Success

Hurray for persistence!!! I've finally gotten the thin-client distro to
a point we can actually deploy.

My main worry (apart from the hardware fiasco i mentioned in a previous
post), was getting the network filesystem right. I was worried about
performance and all those other small things, but eventually, it seems
to a large degree SMBFS mounts solves the immediate need of our first
prospective client.

I'm impressed at the speed of stuff like Opera, Skype, etc, running over
the SMBFS mounted partition. Veeeeerrry cool stuff. What I did was
basically to modify kadischi (the Fedora LiveCD generator), to stop
immediately after building the filesystem and after running
post_install_scripts.

At this point, I run my own customization scripts, and then make a
squashfs image. I also build my initrd system from an updated ALE
(http://datavibe.net/~essiene/ale) initrd system.

I'll have to do a full overhaul on ALE, as its serving me in ways I
didn't even think of previously... another hurray for lazyness by prior
doing!!! :P My initrd image, creates a unionfs of the readonly squashfs
filesystem and some other read-write images which I use to persist
config data, and once I boot, my custom rc.d script runs and mounts my
server drives over SMBFS and I have my full system.

As at today, the config file for the mounting is just that... a simple
txt file. I'm sure the clients will want a GUI config program accessible
from the menu... if i'm not passed out over the weekend, I'll do that up
for Monday :)

For now, I'm wrapping up everything nicely, and I'll probably drive down
to do a techie demo for the techs over at our first prospective
thin-client client (huh? :P)

return;

Tuesday, June 06, 2006

Wazobia Linux Thin Client Edition - Problems

For the past couple of weeks, I've been deriving an embedded distro
based on Wazobia Linux which ofcourse is based off of Fedora Core. I've
tested this out on an HP hardware we layed our hands on for testing
purposes and everything has been working fine.

Recently though, we had to order a prototype of the hardware we intended
to use and I proceeded to boot off of it and WHAM!! BHAM!!! Problems.
I'd gotten so comfortable on my test platform that I'd forgotten that
the test platform was i586 but the target platform is i386. First, I
rebuilt the kernel... and I get it to boot. Next was rebuilding the
initrd image. That one was a huge battle, since I had some other
problems with tty(s) that I didn't show themselves till I tried booting
off of the target platform.

Right now, I realise we have two options. Either we rebuild all the
binaries for the target platform (a lot of drudgery involved :( ) or we
select our test platform (or similar) as the new target platform.
Arggghhhh!!! And i'm supposed to be having a demo later today :(. I
guess I'll just go ahead and demo with the test platform, and worry
about the rest later.

Its really annoying though... ahh well... at least I now have a way
forward. *yawn* return;