I’ve used OpenWrt previously to this project to build some firmwards for the Linksys Router WRT54 range. OpenWrt is an incredibly powerful and small Linux distro. Although debian is probably better suited to the reasonably powerful hardware, I wanted to give OpenWrt a go anyway.
Unless you’re running a MIPS 4Kc processor on your host which I’m guessing you’re not, you’ll either need to cross compile your binaries, or just compile them natively on the device itself. Compiling on the device works fine as long as you have the relevant packages, however if I was going to build a 2.6 kernel, I’d rather do it on an x86 quad core intel host, rather than waiting a week for the device to do it. I also wanted to minimize the writes on the CF card.
OpenWrt comes with a nice buildroot environment which you can read about and download from www.openwrt.org using Subversion.
Here http://downloads.openwrt.org/kamikaze/docs/openwrt.html#x1-310002 is a great HOWTO on getting the build root environment set up on your x86 host.
Also, see: http://wiki.mikrotik.com/wiki/RB500_Linux_SDK – this is a very complete HOWTO, which is why I’ve not covered most of the installation process and just detailed customizations.
You’ll need to select the RB5xx target for the kernel. Also, run:
In your build root top directory, and add USB support (as my one is modded for USB which is not RB5xx default.
While you’re there, browse to the networking options and make sure you have everything you want, specifically the schedulers for traffic shaping.