As the frame buffer layer is not back-ported to Linux-2.4, only the primary graphic card is initialized during the boot process. Secondary graphic cards can only be initialized by an X server, so under Backstreet Ruby you will have a single VGA text console on the primary graphic card.
Ruby for Linux-2.6 supports framebuffer devices and single framebuffer console (which takes over the VGA console), but support for multiple VT's through framebuffer consoles is not yet ready.
To keep VGA console properly working it is important first to be started the XFree instance which will drive the graphic card which is used for VGA console(the graphic card defined as primary in BIOS).