Version 2.03 IZ2S Release - SOUND and SHARED LIBRARIES!

I am probably more excited about this release than any other, because it has opened up the possibilities for some things that just weren't possible before. We now have the ability to run applications using shared libraries. And the first application I decided to port over was "cmus". Cmus is a "small and fast text mode music player" which runs in a console. It's been built to support wav's, mp3's, and even vorbis ogg files. And best of all, it can play streams (that means streaming radio!).

This represents a lot of work on the part of myself, and my friends from #zipit on irc.freenode.net. GPSFan was nice enough to share some of his dark secrets with me pertaining to buildroot and ALSA. From there I was able to get the ALSA-utils built, and then rebuilt statically.

You can download version v2.03 here: http://www.oddree.com/zipit/Z2S-v203.zip

Here is a quick video demonstration of the new feature(s):


As always - here is the README file which is also included in the Zip file.

IZS - The "Improved Z2 Shell" (aka Shell with Benefits) v2.03

ABOUT
This collection was built by Ray Dios Haque out of sheer necessity. I wanted something that I could play with that would not effect the stock software that comes on the Z2. The easiest way to do that was to take the Z2 Shell that was built by rossimo from Zipit Wireless and expand upon it. I take no credit for the work of the Zipit community. If I don't claim to have built it - it's because I didn't. Thanks to all of you for your contributions over the years. I hope you enjoy my additions!

INSTALL (borrowed from the original README)
Format your SD card as a FAT or a FAT16 file system. Copy everything from this zip file onto the root of the card (preserving all paths/directories). Insert your SD card into the Zipit and boot it up. Note that you can only boot this image if you are running the Stock OS that came with your Zipit. If you have installed OpenEmbedded or Debian, then you have probably replaced the stock operating system which included a routine to look for the z2script.sh script. For legal purposes, I cannot provide you this stock operating system. Some of the smarter people in the Zipit community might be able to help you restore this OS (see CHAT below).

CHAT
Come chat with us! There is a thriving (?) community of the worlds best Zipit hackers who hang out in #zipit on irc.freenode.net. You can fire up 'irc' from this very image and come hang out.

CHANGES/ADDITIONS
RELEASE 2.03 (ALL NEW!) - December 2009
We have sound!! Thanks GPSFan! (alsaconf, alsamixer, alsactl, amidi, aplay, gawk, getopt, iecset), SHARED Library support (ldd, ldconfig, readelf), cmus (a wicked MP3 Player), matrix (ncmatrix), mpg123, mplayer-10rc2, ntpdate, gzip (gzip, gunzip, gzexe, zcat, zcmp, zdiff, zegrep, zforce, zless, zmore, znew), zz (script to run after ssh'ing in - use . zz (dot space zz))

RELEASE 2.02 - November 2009
cls (clear screen), centerim (ncurses instant messenger with AIM, Yahoo, ICQ support), irssi (ncurses irc client), links (ncurses), links2 (graphical browser!), sed (for scripting), tar, bzip2 (bzip2, bz2recover), coreutils (10MB's of binaries)

RELEASE 2.01 - Initial Release
wget, ircii-20090520 (irc), wireless-tools (ifrename, iwconfig, iwevent, iwgetid, iwlist, iwpriv, iwspy), ftp (ftp, /etc/services), unzip (unzipsfx, unzip, funzip), nmap (nmap, *new* ncat), wpa_passphrase, less (lessecho, lesskey), grep

LIBS CHANGES
Removed the worthless ncurses static libs to make way for all of the *shared* libs. Not listing all of them here. Just look for yourself.

STARTUP SCRIPT MODIFICATION(S)
REV 2.03
- Stopped audio_pxa from loading, and then run /mnt/sd0/modules/alsa/loadsound.sh for ALSA sound modules.
- Added ntp protocol to /etc/services file.
- Added a line which will fix the time and date if there is a network connection available, and then display that time/date to you.
- From here on out, all libs will go into /mnt/sd0/lib which will be linked to both /lib and /usr/lib/.
- Hid all the gory details of wpa_supplicant, while adding some other stuff for verbosity.
REV 2.02
- Modified the PATH statement so that coreutils and /mnt/sd0/bin binaries will override the busybox stuff.
- Created a home directory for the root user which exists on the SD card. It holds your stuff.
- Added a routine that change your MAC address to something random the first time you boot up. This should prevent duplicate MAC problems if you own multiple Z2 devices.
- Added a /var directory that exists on your SD card. Some things (like dropbear) expect to find it. Had to fix a script error in the BETA (thanks rkdavis).
- Added an /etc/issue file which is displayed at logon time and to ssh connections.
REV 2.01
- Added a routine asking if you would like to configure your wireless card. This was not possible before as there was no scanning routine and the image lacked the wpa_passphrase utility. You can now scan for and configure your wireless settings entirely from the zipit (wording updated in 2.02).
- Added "cp /mnt/sd0/etc/services /etc/services" which gives the ftp command the port numbers it needs to function.
- Added "cp -R /mnt/sd0/etc/terminfo /etc", "export TERM=vt102", "export TERMINFO=/etc/terminfo" to help irc understand the screen layout.
- Added "ln -s /mnt/sd0/lib/* /lib" to make ncurses libraries usable, and any future libraries that you or I might add.
- Added "ln -s /mnt/sd0/share /share" for nmap and anything else that might require a "share" directory to be present to function.

OTHER MODIFICATION(S)
- Replaced the stock keyboard driver in /modules with one that was created by GPSFan (Ken) from the Zipit Yahoo Group. This removed all of the problems with keys not working, or repeating while typing. The original drivers remain, but have been renamed to *.orig.

COMING SOON?
- A better mp3player or a front-end for mpg123.
- A script to download and install "packages" (zip files) using wget, unzip, and a "binary repository" on www.oddree.com. A similar system could provide updates when I provide them.
- I still want to get an image viewer working. DFBSee 'almost' works.

IFAQ (InFrequently Asked Questions)
Q. Why are you wasting your time on this? The Z2 Shell is dead!
A. It may be 10x more work to get simple things built for it - the Z2 is very lean, and still very useful.

Q. Why does nmap (or some other binary) run so slow?
A. Because everything has to be built "static" to run on the Z2. That means that where you might normally have hundreds of shared libraries on a linux machine, the Z2 Shell must embed the libraries into each running binary. It makes for some bloated inefficient stuff. But, it works. If you want a better system - start writing it! Otherwise, try to limit the actions of what you are doing. For instance, use an address range or port range with nmap. Don't start long running scans on entire networks. This is a Z2 we're talking about. The resources are pretty lean.

Q. How can I add my own binary/package?
A. There are several ways to go about this. The method I have used to compile everything is "scratchbox". Scratchbox is a program which let's you cross-compile applications. That means that you can build stuff for an ARM platform, even though you have a regular x86 machine. It can be tricky and complicated. Try reading up on what I have posted at http://www.oddree.com.

Q. Why did you *insert snarky comment here*?
A. If you find that I did something stupid and inefficient, do let me know. I don't claim to have any expertise in the area of building software. If you have some suggestions on how to improve things or would like to help with the next release of this improved Z2 Shell, email me at ray@oddree.com. I would love to collaborate with some folks who might be more talented than myself.

Q. Will you build *insert package name here* for me?
A. Probably. Unless you are asking me to build something that has an endless list of dependencies. In which case I will probably tell you to install OpenEmbedded or full fledged Debian. They all ready have all of these packages. I am only expanding upon the old Z2 Shell because I find it lean and useful.

Happy hacking! Next release ... keyboard backlight support? It's hard to chat in the dark. :-( I am seeking help from the Z2-Zen master, aliosa27 for that project.

About The "Improved Z2 Shell?" (IZ2S)

QUICK ABOUT
The Improved Z2 Shell (IZ2S) is an expansion upon the Z2 Shell that Zipit Wireless released for general experimentation.

WHY USE IZ2S?
I built IZ2S out of necessity because the hardware was perfect for what I needed, and the stock OS had some limitations. I started by building add-ons, but some of them were a bit complicated and so it turned into full standing releases. There are lots of reasons to use the IZ2S, but here are a few highlights ...

• IZ2S loads from an SD card just by having it inserted, as this is how the folks at Zipit Wireless intended you to use their shell. You do *NOT* need to flash anything, hook up and special hardware, or perform any activities that might otherwise :brick: your Z2. You can return to the stock OS at any time simply by ejecting the SD card and rebooting.
  
• IZ2S contains most of the features that you find in the stock OS, but improves upon them (such as a much better keyboard driver and the ability to pick your own radio stations).
  
• IZ2S is very lean. It boots in 4 seconds and running multiple applications at once is perfectly fine. You can effectively chat on IRC in one window, instant message your buddies in another, and be listening to your favorite streaming radio station in a third window (thanks to the magic of 'screen').
  
• IZ2S offers applications that are not available for the stock OS, such as a graphical web browser (links2).
  
• Ray is building and releasing new applications all the time, and installing them can be done on the device itself - or by copying files onto your SD card from within MS Windows.
  

LIMITATIONS

• IZ2S is "text based" meaning that everything happens from a terminal console. If that is not your cup of tea, you might consider installing Debian or OpenEmbedded. Keep in mind though - that those installations have their own kernels which will require that you remove the stock OS.
  
• IZ2S has an instant messenger, but it will not offer you cellular texting. For that, you need to return to the stock OS so that you can be connected to your paid Zipit Wireless account.
  
• IZ2S does not yet have any controls over the display brightness/contrast, or the keyboard lights. We are hoping to have these features included soon thanks to the work of the Z2 community.
  
• IZ2S has an "idle screen sleeper" that looks like a bricked device. This seems to be the result of the kernel trying to put a display to sleep, and the display driver doesn't respond correctly. It's a problem that we inherited from the Z2 Shell from Zipit Wireless, and we haven't been able to fix it yet.
  

WHERE CAN I GET IT?
You can download the Improved Z2 Shell by following links from this very site. This site is actually a great big Blog, and I always tag IZ2S releases with the tag "IZ2S". Therefore, you can always find the latest release with this link: http://rayhaque.blogspot.com/search/label/IZ2S. The newest/latest release should always be the first thing that shows up.

New IZ2S Shell - Coming Soon

I am probably days away from releasing another version of my Improved Z2 Shell. Since getting the ALSA sound package working, I have been working on getting a good MP3 player to use. I found that there are quite a few console (ncurses based) MP3 players to chose from - but they all shared a common problem ... *SHARED* libraries and plugins.

I have never really challenged the restriction that comes with the Z2 Shell which is "your binaries must be static". I assumed that this had something to do with the way that the kernel was built - and I was partially right. Since working with GPSFan on getting ALSA going, I have been getting familiar with "buildroot" which makes building the Z2 kernel really easy. Not that I am interested in replacing the kernel (because I don't want to harm my stock OS) but I *am interested* in re-creating the kernel. In that process that builds the kernel, you are given a chance to turn enable options, add modules, and then build software against the final product. I found that uClibc had some issues with "pre-loading shared libraries". This pre-loading process typically also contains a cache which is created at boot time, or on demand (such as after building and adding new libraries). So caching and pre-loading are broken. But ... the ldconfig, and ldd tool could still be built. That means that shared libraries can still be used provided that "ldconfig" has been run before the application that needs the libraries.

With that, I went to work on building cmus for the Z2. I had spent the previous couple of days trying to work the plugin files into "static binaries" which was quite a contradiction being that they were created in the form of .so files. Now, I could drop all that nonsense and build the application as the developers intended. With the binaries built, I began the process of getting libraries copied over and putting the shared libraries where they belonged. To my absolute delight - it worked!

And so, in the coming days I will be releasing the latest and greatest Z2 Shell which includes the highly anticipated ALSA sound modules, and a new line of applications that previously were not possible.

I am also trying to figure out how we can get the keyboard backlight working, but I will definitely need some help with that. Your friend and mine, aliosa27 has offered his assistance and so I am much more confident that we can come up with something now. You may have heard that sweetlilmre' has been successful in getting this working with the latest kernel source, but it will be tricky trying to apply his same tactics to an older kernel version. Additionally, we will have to do the whole process with modules and daemons (without touching the core kernel).

So check back soon folks. In the mean time, I will be BETA testing all this stuff I have thrown together latetly. If you aren't all ready doing so - get into #zipit on irc.freenode.net and join the discussion.

Version 2.02 IZ2S Release (aka Shell with benefits)

I am pleased to present you with my very latest Improved Z2 Shell version 2.02. This release is a grand improvement over the last, and provides you with a new *color* irc client (irssi), a multi-protocol instant messenger (center-im 4.22.8) and a graphical web browser (links2 with DirectFB). And thanks to the wonders of "screen" you can run all three of these new programs and more in the comforts of your Z2 Shell.

Here is the download link: http://www.oddree.com/zipit/Z2S-v202.zip

VIDEO
I have created a video to demonstrate some of these neat new tools and features.

Improved Zipit Z2 Shell v2.02 - Now with graphical web browser! from Ray Haque on Vimeo.

ABOUT
This release represents several weeks of maddening code hacking. After getting Center-IM working I went straight to my goal of creating a graphical web browser for the shell. Initially I went after trying to get something going with "MiniGui". This was used to create a graphical web browser for the original Zipit device. However, the folks in #zipit on irc.freenode.net turned my onto the DirectFB project. It took about two weeks of off and on hacking and poking, but I was eventually able to wrangle all of the dependencies into submission and slap all the parts together into a single static binary. The biggest challenge there was to make the screen display correctly. As you may know, the Z2 uses a screen size of 240x320, which is turned sideways. This meant that DirectFB had to be rotated. But when rotated, it no longer fit the screen! In the end, a kind fellow in the UK (from the DirectFB mailing list) suggested that I alter the application (in this case, links2). From there it wasn't that difficult to alter. I just had to tell links to ignore whatever size DirectFB reported to it, and set it to 320x240. Another challenge was to get the keyboard to be usable in links2. Links2 wants to use the ALT key to control menus. But on the Z2, the ALT key is used for every single non-alpha key! I was exploring some complicated solutions when rkdavis (from #zipit) found that simply using a different keymap, this whole problem was avoidable.

Here are the release notes:

IZS - The "Improved Z2 Shell" (aka Shell with Benefits) v2.02

ABOUT
This collection was built by Ray Dios Haque out of sheer necessity. I wanted something that I could play with that would not effect the stock software that comes on the Z2. The easiest way to do that was to take the Z2 Shell that was built by rossimo from Zipit Wireless and expand upon it. I take no credit for the work of the Zipit community. If I don't claim to have built it - it's because I didn't. Thanks to all of you for your contributions over the years. I hope you enjoy my additions!

INSTALL (borrowed from the original README)
Format your SD card as a FAT or a FAT16 file system. Copy everything from this zip file onto the root of the card (preserving all paths/directories). Insert your SD card into the Zipit and boot it up. Note that you can only boot this image if you are running the Stock OS that came with your Zipit. If you have installed OpenEmbedded or Debian, then you have probably replaced the stock operating system which included a routine to look for the z2script.sh script. For legal purposes, I cannot provide you this stock operating system. Some of the smarter people in the Zipit community might be able to help you restore this OS (see CHAT below).

CHAT
Come chat with us! There is a thriving (?) community of the worlds best Zipit hackers who hang out in #zipit on irc.freenode.net. You can fire up 'irc' from this very image and come hang out.

CHANGES/ADDITIONS
RELEASE 2.02 (ALL NEW!) - November 2009
cls (clear screen), centerim (ncurses instant messenger with AIM, Yahoo, ICQ support), irssi (ncurses irc client), links (ncurses), links2 (graphical browser!), sed (for scripting), tar, bzip2 (bzip2, bz2recover), coreutils (10MB's of binaries)

RELEASE 2.01 - Initial Release
wget, ircii-20090520 (irc), wireless-tools (ifrename, iwconfig, iwevent, iwgetid, iwlist, iwpriv, iwspy), ftp (ftp, /etc/services), unzip (unzipsfx, unzip, funzip), nmap (nmap, *new* ncat), wpa_passphrase, less (lessecho, lesskey), grep

NEW LIBS
ncurses (libform.a, libform_g.a, libmenu.a, libmenu_g.a, libncurses++.a, libncurses.a libncurses_g.a, libpanel.a, libpanel_g.a)

SCRIPT MODIFICATION(S)
REV 2.02
- Modified the PATH statement so that coreutils and /mnt/sd0/bin binaries will override the busybox stuff.
- Created a home directory for the root user which exists on the SD card. It holds your stuff.
- Added a routine that change your MAC address to something random the first time you boot up. This should prevent duplicate MAC problems if you own multiple Z2 devices.
- Added a /var directory that exists on your SD card. Some things (like dropbear) expect to find it. Had to fix a script error in the BETA (thanks rkdavis).
- Added an /etc/issue file which is displayed at logon time and to ssh connections.
REV 2.01
- Added a routine asking if you would like to configure your wireless card. This was not possible before as there was no scanning routine and the image lacked the wpa_passphrase utility. You can now scan for and configure your wireless settings entirely from the zipit (wording updated in 2.02).
- Added "cp /mnt/sd0/etc/services /etc/services" which gives the ftp command the port numbers it needs to function.
- Added "cp -R /mnt/sd0/etc/terminfo /etc", "export TERM=vt102", "export TERMINFO=/etc/terminfo" to help irc understand the screen layout.
- Added "ln -s /mnt/sd0/lib/* /lib" to make ncurses libraries usable, and any future libraries that you or I might add.
- Added "ln -s /mnt/sd0/share /share" for nmap and anything else that might require a "share" directory to be present to function.

OTHER MODIFICATION(S)
- Replaced the stock keyboard driver in /modules with one that was created by GPSFan (Ken) from the Zipit Yahoo Group. This removed all of the problems with keys not working, or repeating while typing. The original drivers remain, but have been renamed to *.orig.

*STILL* COMING SOON?
- A script to download and install "packages" (zip files) using wget, unzip, and a "binary repository" on www.oddree.com. A similar system could provide updates when I provide them.
- I have been working on getting GPSFan's sound modules working on the Z2 Shell. The modules load, but the alsa tools do not build properly with a static flag. I am hoping to get this resolved soon. GPSFan has been very helpful in troubleshooting this.
- I still want to get an image viewer working. DFBSee 'almost' works.

IFAQ (InFrequently Asked Questions)
Q. Why are you wasting your time on this? The Z2 Shell is dead!
A. It may be 10x more work to get simple things built for it - the Z2 is very lean, and still very useful.

Q. Why does nmap (or some other binary) run so slow?
A. Because everything has to be built "static" to run on the Z2. That means that where you might normally have hundreds of shared libraries on a linux machine, the Z2 Shell must embed the libraries into each running binary. It makes for some bloated inefficient stuff. But, it works. If you want a better system - start writing it! Otherwise, try to limit the actions of what you are doing. For instance, use an address range or port range with nmap. Don't start long running scans on entire networks. This is a Z2 we're talking about. The resources are pretty lean.

Q. How can I add my own binary/package?
A. There are several ways to go about this. The method I have used to compile everything is "scratchbox". Scratchbox is a program which let's you cross-compile applications. That means that you can build stuff for an ARM platform, even though you have a regular x86 machine. It can be tricky and complicated. Try reading up on what I have posted at http://www.oddree.com.

Q. Why did you *insert snarky comment here*?
A. If you find that I did something stupid and inefficient, do let me know. I don't claim to have any expertise in the area of building software. If you have some suggestions on how to improve things or would like to help with the next release of this improved Z2 Shell, email me at ray@oddree.com. I would love to collaborate with some folks who might be more talented than myself.

Q. Will you build *insert package name here* for me?
A. Probably. Unless you are asking me to build something that has an endless list of dependencies. In which case I will probably tell you to install OpenEmbedded or full fledged Debian. They all ready have all of these packages. I am only expanding upon the old Z2 Shell because I find it lean and useful.

WHAT'S NEXT?
Right now, I am focusing on getting the sound modules working correctly on the Z2 Shell. Long ago, GPSFan developed some kernel modules and utilities for the Z2. He did so for the replacement shell(s) which are able to load dynamic libraries. I have been working to get those same tools to load in a static environment but I have run into some obstacles. Thanks to GPSFan, we determined that the broken alsa-utils I kept creating were actually due to bugs with the alsa-libs and alsa-utils packages themselves. Meanwhile, I have been successful in compiling an older audio tools called "aumix". While it can recognize devices and change sound settings ... I still have no sound! It's a work in progress! If any of you talented Z2 developers are lingering and lurking out there - get your asses in to #zipit and help me. :-)

centerim for the Zipit Z2!

One of the first things I noticed about the Z2 was that the keyboard driver for the stock application was downright broken. On the top of my list of "things to compile for the shell" was a multi-protocol instant messenger. But that ended up being quite a task.

Most people probably would have given up and just installed a new kernel or a new OS to the device such as Debian or OpenEmbedded. But I actually like the stock operating system, and I don't want to break it (despite the lousy keyboard driver). And so ... I have been enslaved to Scratchbox for the past two weeks trying to build this application for the Z2 Shell. There were some pretty deep pitfalls. Namely that MSN and Jabber require a few encryption pieces which didn't like being built into a static binary. But before I get carried away with the details ...

Here is the binary if you would like to try it: http://oddree.com/zipit/centerim-zipitz2.zip

Here are the release notes:

** centerim-4.22.8 for the Zipit Z2 by Ray Dios Haque **
**     http://www.oddree.com          **

IM ME!  AIM: rayhaque - Yahoo!: ray_haque - ICQ: 12003044 

ABOUT
This is the centerim instant messenger from http://www.centerim.org
compiled for use on the Z2 Shell image for the Zipit Z2.  It was a
nightmare to build this due to the long running list of dependency
libraries and applications.  Note that the enourmous file size is
the result of building in all of these static libraries.  This binary
was built with AIM, Yahoo, and ICQ protocols.  I removed everything
else to keep the size low and my sanity in check.  Also note that 
MSN and Jabber were removed due to their dependency on GPGME which I
could not successfully build in a 'static' manner.  If you want these
protocols added back in, please build it yourself and then tell me
how you pulled it off.  :-)

INSTALL
Extract the contents of bin to the bin folder on your SD card.  These 
files will be located in /mnt/sd0/bin after you have booted.  You will 
not need a path to run them as your PATH variable should include this 
directory.  Just run "centerim" to get started (it takes a few seconds
to load up).

Extract .centerim to the root of your SD card.  It will be mapped as
/mnt/sd0/.centerim.  This folder will house all of your contacts, 
logs, settings, and perhaps your account passwords if you elect to 
save them (default).

USAGE (to get you started)
centerim - starts the messenger.
g - will bring up a menu which you can navigate with the circular 
arrow keys on the zipit.  You can set up your accounts here.
ESC (smiley key) twice - exits whatever menu you are in.
*Read the official documentation at http://www.centerim.org for more*

MODIFICATION
I noticed that the "servers" being used for the AOL and Yahoo plugins
were very old, and didn't answer requests any more.  I have changed 
them to reflect what is used when I run Pidgin.  If you can't get AIM
or Yahoo to connect or log on, it could be that these servers have 
been changed again.  It could also be that the protocols have been 
intentionally broken by the vendors to prevent people from running
applications like this one.  Check www.centerim.org for details.

I also set the margins to what I think is an appropriate height and 
width.  If you don't like them, you can change them in the main menu
(accessed by pressing 'g').  Any changes you make will be saved to the
config file which resides in /mnt/sd0/.centerim.

Earlier in the week I wrote up a blog post about compiling gettext to give you all some indication of why I have pulled out all my hair. In that post, I pasted my "dependency tree" which I created so that I could keep track of where I was in trying to build a library - which needs a library - which needs ... you get the idea.

When I was done, this is what that dependency tree looked like, complete with notes. I am sharing this partly so that you can see what all was involved here - but also in case I am every crazy enough to try and build this again (which could happen if AOL or Yahoo! decide to re-write something with their protocols or logon processes).

I am pasting a picture, because the HTML here keeps slaughtering all of my tabs and white-space. If you actually want to get to the text - I have put it here.



So there you have it - an application that took two weeks of my life away and will probably only be used by me. This will also make it's way into my next "IZ2S" Shell image which should drop in a week or so depending on what else I might build in the near future.

HACK ON!

Magnify Your Zipit!

Mcavity from #zipit on irc.freenode.net has finally found the cure for strained eyes when you are running a terminal console on your Zipit. You just need to magnify it. After doing some shopping around, he found a site that sells a Gameboy screen magnifier for $0.69.



The magnifier is a flip up - flip down model that you can put up and out of the way when you are not using it. This morning he shared some pictures of the attachment in action.



And here it is in "flipped up mode".



Now, with magnified eyes!



Another before and after ...




Mcavity says that this works pretty well, especially for having spent less than a buck on the idea. I don't know that he was thinking large enough though. When he shared his ideas about getting one of these, I kind of had a scene from the movie Brazil in my head.

"Has anyone seen Sam Lowry?"

Fun with the Debian ARM collection + Building with Scratchbox Part I

There have been a few packages that I simply cannot configure for the ARM platform. These past few weeks I have attempted to build some rather hefty applications that have dozens of dependencies. The way things go, I usually get about three or four dependencies deep when I hit one that just won't 'make' without a pile of errors. Being the non-programmer that I am, I do what anyone else would do. And that is to run the error message through Google and then read through whatever mailing list or forum that I come across. At the end of the day, I usually end up with a pile of compiled and installed libraries - which don't do me any good.

That is, until I found the Debian ARM ports collection. This collection of software includes packages, source code, and patches. Naturally, just taking a compiled binary and dropping it onto the Z2 is not going to work. It's not that easy! And I don't mind building things from source code ... when the source code builds correctly. That's were the source code and patches come into play.

Take for example: gettext. This old GNU utility builds pretty easily on most platforms. But the ARM branch of Debian (and the Zipit Z2 Shell) were built with ulibc. ulibc is a work-alike clone to GNU libc (aka glibc). But there are those little nuances that get under your skin. I found myself beating my head against the wall Saturday afternoon because ulibc doesn't support some sort of program error checking routine. The result is that you cannot build gettext without modifying some code which is well beyond my capabilities. The ulibc folks could probably fix this. But they throw the blame at the glibc folks for supporting a routine which really serves no purpose. While they are busy fighting like a bunch of children (this argument is a couple years old) I still don't have a working gettext!! Surely there are people out there who are smarter than me and who can fix this broken code. There are. And *maybe* they have fixed it. Here is where I go next ...

First, I browsed out to Debians Etch package page for the ARM platform. You can find it here: http://packages.debian.org/etch/arm/.

After some browsing around (DANGER: the search at the top of the page leads you away from the ARM platform stuff!!) you might find that gettext is available from the "Development" category. Here is the page where the source code was located: http://packages.debian.org/etch/arm/devel/gettext. Take a look at it now. On this page I can see that gettext has some dependencies which are displayed with a red bullet. While it may be lousy that I have another couple dependencies to install - at least I know I can build them without errors! Time, I have. Development skill - not so much. Assume that we have satisfied all of it's needs (ignoring the libc needs, since Scratchbox provides ulibc for us).

On the right side of the page there are several links for downloading the chunks of code. There is the package itself (some pre-compiled binaries that "might" work, but probably won't). There is the original source code (the tar-ball has the word "orig" in it's title). Most importantly, there is the "patch file" which will always end with "diff.gz". The patch file is what will correct the source code for us so that we can build it properly for our Z2 ARM platform.

If you haven't figured this out yet - it helps to be connected to your Scratchbox development PC in a terminal window, with a browser opened on your regular PC. Then you can copy and paste links directly into scratchbox without weighing it down with whatever else you are running (like the web browser).

So, we would start up Scratchbox and fetch the files we need:

wget http://ftp.de.debian.org/debian/pool/main/g/gettext/gettext_0.16.1.orig.tar.gz
tar -xzf gettext_0.16.1.orig.tar.gz
cd gettext-0.16.1

Now before we start building anything, we need to patch the source code for the Debian ARM platform:

wget http://ftp.de.debian.org/debian/pool/main/g/gettext/gettext_0.16.1-1.diff.gz
gzip -d gettext_0.16.1-1.diff.gz
patch -p1 < gettext_0.16.1-1.diff

At this point we would run './configure', and then when it was complete we would run 'make' or 'make install'.

./configure
make


But it's not that easy. Gettext *STILL* won't install. Even with the patches. Why is that? Well, because the Debian folks didn't use ulibc. They used the full fledged GNU libc, or glibc. You are still going to get those damned error messages. Grrr! But hey, someone else had to have gotten these errors before. And if you start Googling around you will find what fixes it, which are these two pages:

http://osdir.com/ml/linux.lfs.hardened/2007-07/msg00041.html
http://www.linuxfromscratch.org/patches/hlfs/svn/gettext-0.17-uClibc-1.patch

Here are what the patches entail. Note that these patches were created for version 0.16.1, which was the version I was attempting to build. If I was using the latest and greatest gettext from the folks at GNU, I might have to do some further digging or investigating. As it was, I was able to read the patch snippets and make sense of them. Here are those snippets.


--- gettext-0.16.1/gettext-tools/src/msgfmt.c 2006-11-27 17:02:08.000000000
+0000
+++ gettext-0.16.1/gettext-tools/src/msgfmt.c 2007-07-18 20:12:58.000000000
+0000
@@ -194,6 +194,7 @@
static void read_catalog_file_msgfmt (char *filename,
catalog_input_format_ty input_syntax);

+void (*error_print_progname) (void) = NULL;

int
main (int argc, char *argv[])


And ...

--- gettext-0.16.1.orig/gettext-tools/gnulib-lib/error.h 2006-11-27 18:14:50.000000000 +0100
+++ gettext-0.16.1/gettext-tools/gnulib-lib/error.h 2007-06-20 13:29:32.000000000 +0200
@@ -50,7 +50,10 @@ extern void error_at_line (int __status,
/* If NULL, error will flush stdout, then print on stderr the program
name, a colon and a space. Otherwise, error will call this
function without parameters instead. */
-extern DLL_VARIABLE void (*error_print_progname) (void);
+#ifndef __UCLIBC__
+extern DLL_VARIABLE
+#endif
+void (*error_print_progname) (void);

/* This variable is incremented each time `error' is called. */
extern DLL_VARIABLE unsigned int error_message_count;


To apply the snippets you can:
1) Create two files, with these snippets and save the files as somefilename.diff. Then run 'patch -p1 < somefilename.diff'.
2)

Now, since I had all ready started building my application and bombed in the middle somewhere, I should clean everything up and start over.

make clean
make


Ten minutes or so later, you should be done. And if you didn't get any errors. You can now "make install".

That's it! You are one step closer to compiling some giant peice of software. Now, where was I in the list of dependencies when I got stuck trying to solve this? If you get lost or sidetracked, you might never make your way back to where you were when the problems started. For that reason, I often create a running list for myself. If I try to install one program that requires three more, I document them and use simple tab indentation to indicate what is a dependency. Your end result will be a big messy tree that gives you some indication of how many frustrating hours it will take to get this all done. It's also good to document what you have had to do that is "special" to get something to build. If you have to start over later, it might be good to have those notes.

Here is the "tree" as it looked when I was building gettext. An "X" indicated that I completed it. Question marks mean that I think something might be required, even though the Debian pages didn't mention it as a requirement. In other words, there were massive build failures and I had to take a guess at what I was missing. Your notes and abbreviations may vary.

centerim (4.22.5-1~bpo40+1) - http://packages.debian.org/etch-backports/arm/centerim
 libcurl3-gnutls  (>= 7.15.5-1) 
  libgnutls13  (>= 1.4.0-0) 
  X zlib1g  (>= 1:1.2.1) 
  libcomerr2  (>= 1.33-3) 
  libidn11  (>= 0.5.18) 
  libldap2 - (configured with: CPPFLAGS="-I/usr/local/BerkeleyDB.4.2/include"
                LDFLAGS="-L/usr/local/BerkeleyDB.4.2/lib" ./configure,, make depend,, make
                install)
   X ?? (undocumented requirement) gettext (0.16.1-1) http://osdir.com
                              /ml/linux.lfs.hardened/2007-07/msg00041.html
    ?? (undocumented requirement) intltool - installaed intltool-0.40.6
     X XML::Parser
      X ?? expat - installed expat-2.0.1
    ?? (undocumented requirement) iconv - installed libiconv-1.9.2
   X libgnutls13  (>= 1.4.0-0) 
    X libgpg-error0  (>= 1.4) 
    X libgcrypt11 (1.2.3-2)
     X libgpg-error0  (>= 1.2)  
    X liblzo1
    X libopencdk8  (>= 0.5.8) 
     X libgcrypt11  (>= 1.2.2) 
     X libgpg-error0  (>= 1.4) 
     X zlib1g  (>= 1:1.2.1) 
    X libtasn1-3  (>= 0.3.4) 
    X zlib1g  (>= 1:1.2.1) 
   X libsasl2-2 (2.1.22.dfsg1-8+etch1) - installed to /usr/local/lib/sasl2
     X libdb4.2 (4.2.52+dfsg-2)
     X (undocumented requirement) libssl0.9.8 (0.9.8c-4etch9)