An old-school hack remembered

Way back in the days of yore, when Turbo C++ had just came out, I was using a DEC Rainbow 100B+. For those not old enough to recall, this was a machine that came out in 1985 with software that came out in about 1990. Boreland had done a good job of not using all the IBM BIOS calls, so its compilers and such were useful on the Rainbow. The Rainbow had an 8088 processor in it, but wasn't IBM Compatible (being released at about the same time the IBM PC was). The Rainbow was better in a lot of ways, which is how I wound up with it, but worse in other ways...

However, Boreland hadn't done a perfect job. They had used an unused software interrupt (INT 18h) for memory management of the compiler. Early versions of the Turbo C++ used it in only a few places. Patches quickly appeared in the Rainbow community to allow one to run TurboC++ on the Rainbow. These patches were about a hundred lines of code and just patched the offending INT calls to use a different vector (one that was unused on the Rainbow).

I had seen these patches, and when my job was using C++, I thought it would be cool to be able to compile code for it at home. I went out and bought Turbo C++, only to discover it was a newer version of the compiler than the one the patches were for. They just didn't work. So I wrote the author of the patches, who seemed a little put out with me for asking so soon after the compiler was released. Feeling a little bad for not having given more to the online Rainbow community that had given me so much cool software, I resolved to create patches myself and post them.

To make a long story short, I failed. The code was too complicated in the new version for someone of my meager skills (at the time) to produce patches. The calls were all over the code, and the sequence of bytes that invoked INT 18h were in many places that turned out to be data.

Being young, and still struggling to make ends meet, I wasn't about to let this get in my way. I didn't want to have wasted that $80 on software I couldn't use and now couldn't return. Buying a new machine was out of the budget at the time. So I came up with a clever hack, which I'll describe now.

On the DEC Rainbow, INT 18 was used for video output. So when the compiler ran, it would call INT18, which would then dive into the BIOS with loco registers crashing the machine, or just not working. The trick turned out to be simple. I used a TSR to fake things out. The TSR would load at an address that could be known at run time. This TSR hooked into the MS-DOS interrupt vector which was used to set interrupt vectors and into the interrupt vector that was called when a progam terminated. I also hooked into the INT 18h interrupt vector to vector calls to it through my code. When a normal program would run, it would get the system's INT 18h call, and all the video would work. This let me run my editors, terminal emulators and games without rebooting. When Turbo C++ would run, it would install its own INT 18 handler to do its thing. My program intercepted this call and just saved the target for later use rather than writing it into low memory to give it effect. Later, when an INT 18 call happened, I'd check the calling address. If it was at or below my TSR or in the ROM BIOS, then I know it was something in the system making the call and I called the original INT 18 handler. If it was above my address, I knew it was boreland's compiler and would call the address it had tried to install earlier. When the turbo C++ terminated, I'd reset things to the initial state.

I did this trick almost 18 years ago now, yet it still strikes me as a clever way around a problem. The program took me an afternoon to write and debug, yet provided me with years of useful service until I retired the Rainbow...

Building a bootable ARM SD

A few weeks ago, I wrote up how to make a bootable i386 image. People have been asking me to extend it for my ARM platform.

Here are the instructions for a hypothetical box that has a boot loader that can load an image from a FAT partition of a SD card, and FreeBSD runs off of a ufs partition on the SD card. This will be for a big endian ARM target, based on the AVILA kernel. I'm also assuming that this platform doesn't use FreeBSD's normal /boot/loader.

1. setenv MAKEOBJDIRPREFIX /blah
2. setenv TARGET arm
   
3. setenv TARGET_ARCH arm
4. setenv TARGET_CPU xscale
   
5. make buildworld
6. make buildkernel KERNCONF=AVILA
   
7. fdisk -I da0
8. fdisk -f disk-partitions da1
   
9. newfs -t msdos /dev/da0s1
   
10. bsdlabel -w da0s2 auto
11. newfs /dev/da0s2a
12. mount /dev/da0s2a /mnt
13. make installworld DESTDIR=/mnt
14. make installkernel DESTDIR=/mnt KERNCONF=AVILA INSTALL_NODEBUG=t
    
15. make distrib-dirs DESTDIR=/mnt
16. make distribution DESTDIR=/mnt
17. echo /dev/da0s2a / ufs rw 1 1 > /mnt/etc/fstab
18. echo ifconfig_DEFAULT=DHCP > /mnt/etc/rc.conf
19. echo hostname=demo >> /mnt/etc/rc.conf
20. cp /mnt/boot/kernel/kernel /tmp
    
21. umount /mnt
22. mount -t msdos /dev/da0s1 /mnt
23. cp /tmp/kernel /mnt
24. umount /mnt
    

In step 8, one needs to have a special file. Here's what I use, but one needs to automate it a little more. Here's a hand-tuned one that I used for a 1GB SD card I had laying around.

# partition table
g c940 h32 s63
p 1 6 63 201537
p 2 165 201600 1780128

Clearly, one needs to tune the above for their specific part. NanoBSD tries to automate this process, but that is beyond the scope of this posting.

Update: fixed step 17.

Article about my past life

Wired has a great article about the high precision hobbyist timing community. These folks are less fanatical about timing than the professionals, but not by much. You know you are a true time geek when you understand the joke in Tom's quote from the article about his trip to a tall mountain near his home "It was the best extra 22 nanoseconds I've ever spent with the kids." Check it out here.

Cisco and Observations of a new Mac user.

Last week I started at Cisco. The entire week I tried to figure out how to drink from a fire hose without getting hurt. This week, I'll be learning where to go for help with fire hose related injuries. :-).

Seriously, I've joined a very dynamic and energetic group here at Cisco. The job so far has been a blast, and I'm really enjoying the work. We're at the early stages of many things in my group, so it would be a little premature to talk about them right now. since Cisco hires a lot of people, they have managed to optimize the typical new-hire HR madness enough that I was able to get something accomplished last week.

And I even found time to tweak the new MacBook Pro that showed up on my doorstep. This is the first employer provided laptop that I've had in at least 15 years. The last one was a Tadpole Sparcstation II class machine... I knew that Apple had done a great job on the GUI, but when I was able to setup printers and get onto my Samba shares without doing more than a few key clicks to accept automatically discovered items, I was hooked. On my Windows box that I keep around for the rest of the family, these two activities took me a couple of days of hacking to get barely working some of the time. OS X Leopard just works, and I have all the unix tools I know and love.

A quick thumbnail to cross debugging core files

This is just a quick thumbnail about how to take a core file from a FreeBSD/arm box and debug it on a FreeBSD/i386 box you cross compiled the FreeBSD/arm image on.

For normal gdb, one would just type:

• gdb prog prog.core

and be done with it. However, without careful setup, this won't work when cross debugging. If you do the above, you'll get a number of warnings or errors from gdb when it tries to load the i386 libraries with the arm binary.

There is a simple trick, however, that gets around these problems. It can be used both when debugging cross architecture and when debugging different a core from one release on a newer/older release of FreeBSD. This example, I use 'gdb-arm' which is the debugger I use for cross debugging.

• # gdb-arm
• (gdb) set solib-absolute-prefix /path/to/built/image
• (gdb) file prog
• (gdb) core-file prog.core
• (gdb)

By setting solib-absolute-prefix, you are instructing gdb to use that path as a prefix for all files it has to look up. So rather than /lib/libc.so.7, it will look in /path/to/built/image/lib/libc.so.7. Now that you have all the libraries that you need for debugging, you'll be able to track down the problems in no time.

This is just the barest of sketches for what to do. There are many other ways to make debugging easier in a cross debugging environment, and gdb can do more than just look at core files. But those topics will have to wait for another day. The curious are invited to read through the gdb info files. They are very complete, and well indexed by google or other search engines.

SDHC purchased

Today, while I was ordering a new keyboard for my laptop (a 1-year-old is very hard on them), I ordered a 4GB SDHC card. When it arrives, I plan on expanding FreeBSD's support for SDHC. I had been waiting for the card to arrive to post this, but so far it has taken two weeks and no SDHC card. I bought it on ebay, but the buyer was located in Hong Kong, so my card is now stuck in customs. Let this be a warning to others.

The SDHC card implements the SD Card's 2.0 specification. It is unclear to me if the only change is in the commands sent to the card, or if additional hardware support is required to make things work.

I'll keep everybody posted.

Building bootable FreeBSD/i386 images

From time to time I hear people complain at how hard it is to build an image from the FreeBSD sources. This week, I'll explain how I built a bootable i386 image on a USB flash device and also make some observations about the results.

Recently, I needed to create a bootable i386 image. The easiest way was to build one on my amd64. Here's what I did. I setup my environment for the build (steps 1-3), built FreeBSD/i386 userland and kernel (steps 4 and 5), prepared the flash (steps 6-11), installed userland and the kernel (steps 12 and 13), added the extra files needed for boot (steps 14-18). The detailed commands follow:

1. setenv MAKEOBJDIRPREFIX /blah
2. setenv TARGET i386
3. setenv TARGET_ARCH i386
   
4. make buildworld
5. make buildkernel KERNCONF=GENERIC
   
6. fdisk -I da0
7. fdisk -B da0
8. bsdlabel -w da0s1 auto
9. bsdlabel -B da0s1
10. newfs /dev/da0s1a
11. mount /dev/da0s1a /mnt
12. make installworld DESTDIR=/mnt
13. make installkernel DESTDIR=/mnt KERNCONF=GENERIC INSTALL_NODEBUG=t
    
14. make distrib-dirs DESTDIR=/mnt
15. make distribution DESTDIR=/mnt
16. echo /dev/da0s1a / ufs rw 1 1 > /mnt/etc/fstab
17. echo ifconfig_DEFAULT=DHCP > /mnt/etc/rc.conf
18. echo hostname=demo >> /mnt/etc/rc.conf

I put all this on an SD card and inserted that into an usb adapater and booted the laptop with it. FreeBSD's base system isn't too big these days, only 205MB for a full system.

% df /mnt
Filesystem 1024-blocks Used Avail Capacity Mounted on
/dev/da0s1a 484822 205570 240468 46% /mnt

This can easily be trimmed, but with a 512MB SD card for $5 at Office Max, there's little point in trimming for the project I needed the flash for. I just needed something that I could boot to transition my FreeBSD/amd64 laptop to a FreeBSD/i386 laptop.

If one needed to reduce the amount of space used by an installation, then there's a number of options than can be used to reduce the footprint of the system. In my experience, it is best to build everything without these options, then use the WITHOUT_* options on the installworld step to keep the image size down. It is easy to cut FreeBSD's footprint in half with these options. In addition, the kernel is 28MB with all its modules, and this can easily be reduced below 5MB in most cases.

nanobsd can be used to automate this process, as can TinyBSD. I'll save for another column the techniques I have used in the past to reach 16MB.

Cross debugger

GDB Cross building on FreeBSD

I recently had to build a gdb that worked on our development host, but understood arm binaries and core files. It turns out to be fairly easy to do this in the FreeBSD tree. I did this on RELENG_6, but the same techniques will work for RELENG_7 or -CURRENT.

The Cookbook

1. % setenv TARGET_ARCH arm
2. % cd /usr/src/gnu/usr.bin/binutils
3. % make depend all
4. % cd ../gdb
5. % make depend all
6. % cd gdb
   
7. # make install

These steps assume one is using csh-like shell. Commands prefixed by '%' can be run with anybody with write privs to the source tree. Commands prefixed by '#' need to be executed by root. These instructions will install a gdb-${TARGET_ARCH} in /usr/bin that can be used to look at core files, or connect to a gdbserver on the target platform. As of this writing, the arm gdbserver glue hasn't been written or committed to FreeBSD's CVS tree. The author will happily review patches and commit them if they are good enough.

Running the debugger

You can run the debugger just like you would other binaries.

1. gdb-arm binary binary.core

to get a where listing of where one went off the tracks in binary 'binary' or look at a forced core dump.

That's it! Pretty simple, all in all, to create the cross debugger.

The cross-gdb port

As has been pointed out in feedback, one can also use the devel/gdb-cross port to make this even easier. I've not used this method. Maybe I'll investigate and report in another posting.

FreeBSD/arm on a new SoC

Today I started work on porting FreeBSD/arm to a new SoC. I'm taking very detailed notes on this process, as well as attempting to learn from my atmel port how best to organize the port. I'll keep everybody posted and once I get far enough into it, I'll summarize the notes in a series of articles here. Hopefully this will help speed adoption of FreeBSD/arm on new hardware.

Speaking of new hardware, there's people working on a FreeBSD port to the NEO1973. Andrew Turner's email on the latest snapshot can be found here.

Merging *BSD's usbdevs

Once upon a time, all the BSD's got their USB stack from NetBSD. With it came a usbdevs file. This file has grown and mutated on the different BSDs for some time. The time has come to merge them all back together. This sounds simple in theory, but in practice it is a lot more complicated. Different BSDs use different names for some vendors and some devices. Sometimes one is more correct than the other. Other times, they are merely both wrong. Still other times the names are the same, but the descriptions of the device are different. There's about ~2k lines in each of the files, and the merged file is closer to ~3k lines long.

I can do basically whatever I want to the FreeBSD usbdevs file to make this happen. The tricky part is justifying all the changes to a skeptical audience for the other projects. The other tricky part in merging is making sure that nothing breaks. Of course the final tricky part is getting the leg work done and the files committed quickly enough that they do not become stale. That's going to likely be the hardest part of all.

USB update

I've finished much of the work on improving the in-tree USB stack, apart from adding device IDs. This is just the initial round. Now that all the cruft has been removed from the in-tree USB stack, we'll be able to support it better in RELENG_7. I plan on merging most of the client drivers into RELENG_6 after a couple of weeks, depending on how testing goes. So far, so good. I've added about 30 new devices. In fact, I've added so many that I've had no time to update the manual pages. If there are any doc people that can help synchronize the man pages to what we now support, that would be great.

After removing the obfuscating macros from the code, it is clear to me that adding some primitive locking to the USB stack might be possible now. It isn't going to happen for 7.0, however.

ooo builds!

After the shared library version bump, the xorg 7.2 transition and the new gcc compiler in the base, I started to rebuild everything. After some snafu deleted all my +CONTENTS files, I've finally managed to rebuild everything (I think). The last major thing was ooo. After my laptop would shut down for being too hot a dozen times, and having to play whack-a-mole to remove all the extra bogus junk that's accumulated in /tmp, I've finally been able to do a build.

The biggest lesson learned, apart from how fragile java can be on amd64, is that ooo itself takes 8G of disk to build. That's /usr/ports/editors/openoffice.org-2/work only, not counting the dozens of packages it depends on. I thought I was being insanely pessimistic when I installed /tmp with 10G and put my ports tree there. Now I see that I'm right up on the edge. I'll have to repartition my disk, or get a bigger one.

Now, if only kino worked on FreeBSD/amd64 and current. It isn't even compiling for me these days :-(.

Minor USB cleanup

I did a quick pass through the tree tonight eliminating all the devinfo stuff. This code had been implemented in uhub a while ago, but the client drivers were never updated to remove the bloat. We set the device name to usbd_get_devinfo in uhub, but then in the client drivers, we computed it again, and printed it, so the data would appear twice.

So far the only wrinkle is that kldunload/kldload doesn't set the description, but there are bigger problems there.

I hope to investigate a few of the bigger usb problems before 7.0 branches and get them fixed.

CardBus and USB minor improvements

I've started working on some minor improvements to CardBus and USB. I'd like to see each of them using filters. CardBus and PC Card necessarily share interrupts between the bridge and the client devices. There's only one interrupt pin. Having to schedule an ithread just to check on the status of the card (for eject events) or power (for the interrupt driven power-up sequence) imposes a large overhead on the system. In addition, it may interfere with CardBus drivers that use filter. The current ISR isn't written to be a filter (what used to be known as a fast interrupt), so some work is needed to migrate it to such an interrupt.

For USB the problem is different. There's no interrupt sharing inside of USB, per se, since all the work goes through the pci<->usb host controller. However, on many systems the designers cheaped out and forced usb and the network to share interrupts. Since usb is still Giant locked, this causes a lot of extra Giant contention. The idea hear is to introduce a filter that turns off the interrupt bits and requests that the ISR run. The ISR would be exactly like we have today, with the addition of code to turn the interrupt sources back on. The theory here is that network inspired interrupts don't have to take out Giant at all since usb can schedule its ithread only when there's real work to do. If the ithread isn't scheduled, then Giant isn't taken out.

Since the freeze is coming up soon, I need to get this done in the next two weeks.

Found cheap SDIO 802.11b card

At CompUSA today I found a cheap 802.11b SDIO card. This card is labeled as a Kodak Camera wireless card, but has a Marvell 8686 lurking under the hood. I've seen this card before, but it was $100. Today it was $20, which was cheap enough to take a flier on. This card should come in handy when I get around to implementing an SDIO stack for the SD/MMC stack that is in FreeBSD right now. I don't know if there's any information available for this card, and I've seen indications that there might not be, but at the very least I have a card that I can parse the CIS from and do other things in preparation. OpenBSD has a partially reverse engineered driver for a PCI cousin of this card that might have some clues, but I doubt I'll find anything useful for it.

But playing with this card will have to wait until I can get the simple memory cards working with the sdh driver, or the sdmmc stuff. I need to get the ZyDas ZD1211 driver going first.

I also found a TRENDnet wireless card based on the RTL8187B chipset. Linux driver, a couple of different ones, are available. But it uses some non-standard 802.11 code, so decoding it might be interesting. It is way down in the queue at the moment, since I have no documentation for it and no known working driver. At least with the ZD1211, I have two known working drivers from which to crib.

SD/MMC

There's another SD/MMC stack for FreeBSD that's been posted. It was written by Akihiko GOTANDA-san using the SD simplified specifications. It works well for the Ricoh based parts, but fails to attach to my TI based parts. It doesn't have the hacks necessary to put the TI parts into standards conforming mode either, but even with those added there are some problems. This driver was recently announced in the mobile@ mailing list.

This joins my SD/MMC stack that I wrote for the Atmel AT91RM9200 port. Andrea Bittau also has written a 'sdh' driver that plugs into my stack. It works with Ricoh parts, but not TI parts either, but it gets closer than Gotanda-san's driver. It was announced a while ago in the mobile@ mailing list.

And there was much interest in my SD/MMC card talk at BSDcan! I'm glad to see more interest here.

ZyDas usb stick update

After my last entry, I noticed that Linux, OpenBSD and hps' new stack all have ZyDas drivers, at least for the ZD1211 and ZD1211B. hps' zyd driver appears to be a relatively recent port to his new stack.

I have the OpenBSD one builing on FreeBSD's current USB stack. I still need it for other reasons, and a MFC of HPS' stack will never happen, so if we want to support it in 6.x, we'll need one for the old stack.

I get to the part where the driver tires to setup the MAC with the RF specific register values when I get a panic. I've not had the time to debug it further, but hope to find that time soon.

My experiences with BSDcan showed that I needed both USB and CardBus wireless working...

Ativa Wireless G Network Adapter

I stopped by Office Depot today to buy a wireless usb card. I wound up getting the rather generic sounding Ativa Wireless G USB network card. This turns out to be a rebadged Belkin F5D7050 v4000 wireless card. This has the ZyDas ZD1211 chipset in it. This is a fairly popular chipset to judge by ebay (where it is just about the only chipset advertised).

There's a Linux driver. Well, there are two. One that was basically binary only that was written by ZyDas, and another that was a rewrite by the community. I'll have to see if there's one for FreeBSD :-)

eHome Cardbus card

Looks like my run of luck with CardBus cards has reached an end.

I came home with an eHome card, but it is a Marvel part for which little documentation can be found on the web. Of course, if you have info on the 88W8310 part used in this CardBus card, please let me know. This is H/W REV A1, in case they have multiple versions of the EH101.

BSDcan 2007

I had an absolute blast at BSDcan 2007.

There's more buzz this year about embedding FreeBSD than there has been in prior years. I believe that we're building momentum in this space.

There were a number of cool talks about developments in FreeBSD, which you can see at the bsdcan web site.

My talk on SD/MMC cards was well recieved. The slides are available now. This contains an overview of the various flavors of SD/MMC card.

My talk on the Atmel AT91RM9200 slides are also available, as are my 2000 talk on NEWCARD as a paper or as MagicPoint slides, and the follow on paper on ISA vs PCI interrupt dispatch in the PC Card code I gave in 2002.

I've had a chance to also hack on the cardbus resource allocation code a bit while here. There's still something screwy going on with it, but I'm having trouble locating the oddness.

I had hoped to collect some of the older SD/MMC cards while at this conference. However, nobody had any cards that I was able to take with me. I did have a few of them that I was able to try out on my Atmel AT91RM9200 board and fix minor timing issues with the stack (the older ones are a little slower and needed slightly longer timeouts).

The best part is seeing everybody again from past years. It is good to see the community growing. It is also good to see many people form companies that have been designing in FreeBSD into their products for years. These companies have also started to fund the chances necessary for them to run FreeBSD in different embedded platforms (both i386 based, as well as powerpc and mips based hardware was talked about).

There's also been some buzz about creating a FreeBSD community summit to complement the FreeBSD developers summit. A place where developers, engineers from companies using the products and users could get together and discuss how to drive FreeBSD development as well as ways to facilitate integration of changes that companies have developed for FreeBSD that they desire to give back to the FreeBSD project.

It was also good to see developers from the sister BSD projects. Renewing the personal ties to the people in these sister projects is good for the BSD ecosystem, and we need to do more of them.

That's all for now. I hope to post links to videos from my talks shortly (or find some way to put them on you tube, if they are short enough).

see everybody next year!

All in all, a very exciting BSDcan.

FreeBSD SD/MMC stack talk

My talk for BSDCan 2007 on the FreeBSD SD/MMC stack is going nicely. The basics are done, and now I'm into the hard part of the talk: actually documenting the dataflow and interfaces between layers. This is the part of the talk that will be the call to arms: more work needs to be done to add support for all the latest SD/MMC goodies.

I'll post a pointer here once I've finished the talk to my slides with notes. Maybe people here can find any silly errors in them before the question and answer session of my talk brings them up. I'll also report on the status of the Standard SD Host Adapter driver that's been written, but not yet committed to current yet. The talk also gives a lot of background for these devices, and has a number of pointers to publicly available documents that thankfully nearly totally document all aspects of SD/MMC. The exceptions being mainly the DRM features of SD and the secure MMC extensions.

FreeBSD wireless picture frame

Tonight I finished a present for my wife. A wireless picture frame based on FreeBSD.

The frame is running on an old Sony VAIO PCG-505TS that I had from many years ago. Its battery is nearly shot, good only now for powering the BIOS battery. However, I put in a solid state disk and a subset of FreeBSD. It now displays pictures from a directory on our home network in rotation.

The hardest part turned out to be how to securely mount the laptop into the picture frame.

I'll describe the software that I used in the coming days. For now, just this little teaser.

Catching up

Things at my day job are finally starting to slow down a little, so I've been trying to catch up with a bunch of changes that I made at a breakneck pace over the past several months. These should be trickling into the FreeBSD tree over the next few days and weeks.

I've setup three different Atmel boards for testing this latest batch of changes. The first board is one that my company makes. The second is a Kwikbyte KB9202A (modified to have a 32kb iic eeprom, since I burned out the first one). The third is a new eval board that I'm going to try to get running FreeBSD/arm. It has uBoot on it, so will be a departure from what I've done in the past.

I've also setup my Gateworks boards finally, so I can get FreeBSD/arm on there for testing purposes.

I also have new hardware from Atmel that should be setup soon. As well as a cute little Atmel AT91Rm9200 based smart terminal that needs to find some time to get on my dance card.

I've managed to get the necessary infrastructure into place. I have a usb hub and a bunch of serial to usb dongles to create a makeshift terminal server. I'll be transitioning this to one of the atmel boards so I can at least put it to good use.

Anyway, now that it is looking like I'll have some more time I should be writing these more often as well.

FreeBSD ExpressCard Support

I was browsing the support for ExpressCard in Linux. I couldn't find any explicit support for it at all, not matter where I looked. I found usb support and pci express support, as well as hot plug versions of the latter. This puzzled me a bit, since I'd read that Linux supported ExpressCard cards. Maybe no special software support is required at all...

So, I thought I'd get a couple of ExpressCards and take them for a test spin. It turns out that FreeBSD versions back to 4.x that can boot on modern machines with ExpressCard slots will support one important class of cards (assuming the drivers are in the version of FreeBSD that you try).

The ExpressCard standard specifies two types of cards, from an electrical point of view (not to be confused with the two width form-factors offered: 34mm and 54mm). The first kind is card that contains a USB 2.0 device in it. In this configuration, the host bridge connects it up to power and a standard usb hub. The second kind of card is a single channel PCI Express card, used for applications where greater bandwidth is required between the CPU and the hardware on the card.

The USB versions just work. I plugged in a Delkin Devices eFilm ExpressCard that I was recently able to purchase due to the generosity of a someone who sent me money to buy toys with. This device supports SD, SDHC, MMC, MS, MSpro and xD cards. Since I also wanted to enhance the SD/MMC stack to support SDHC cards, I thought this would be a good choice (since the price was right) for me to test things out on.

The PCIe versions will need PCIe hot-plug support added to pcib (or a subclass). I've had trouble finding cards that are PCIe for sure, so if you know of one, please let me know. I'm guessing that the eSATA cards are a good choice for that, but confirmation would be great.

So the good news is: FreeBSD has supported at least some of the ExpressCard cards since before the ExpressCard standard was published.

Followup Atmel AT91 family work

After receiving an evaluation board from Atmel with an AT91SAM9260 onboard, I've started updating the Atmel support for FreeBSD in the p4 repository.

This chip is basically the same as the AT91RM9200, but with a few twists. First, it uses a ARM926EJS core rather than a ARM910T core. This means that it has an arm9e core in it (half way between an arm9 and an arm10 core). Fortunately, the NetBSD support for these CPUs was easy to port, and has been nearly a drop in (I'm sure I botched something it went so smoothly).

Next, there are three classes of devices on the AT91SAM926x parts that need addressing. The first class of parts are those that have FreeBSD drivers that utilize the AT91RM9200 errata to produce proper function. Many of these bugs in silicon have been fixed, so the kernel needs to cope in some manner. The MCI device is the most prominent here, as it needed to have byte swapping done in and outbound to be interoperable with other systems. There were structural changes that severely affected performance that will need to be ifdef'd for the old part. There are a number of other minor tweaks that may be necessary.

Next, we have those devices that were present in the RM9200, but have been augmented for the SAM9 series. These include enhanced support for SDIO cards in the MCI device, as well as a number of minor additions to the TWI, SPI and USART devices (or so my first reading would suggest... some are so minor that they could just be clearer documentation).

Finally, we have those devices that aren't present at all on the RM9200, but are in the SAM9 devices. There's an LCD controller and a cool image processing whatsit that fall into this category. The EMAC from the RM9200 has been removed, replaced by a newer 10/100 part that has more buffers for transmit, so the bottlenecks seen on the RM9200's network performance can be corrected. This will take a little time, but since the SD card boot should just work, or nearly should, the pressure to have this falls from 'critical' down to 'necessary for basic support.'

The eval board I received also has some neat audio hardware attached to the SSC bus (think McASP-like bus, if you are familiar with TI's DSPs). Back of the envelope calculations suggest that I could use this device as my 'audio server'. It is small enough to slip into the rack of audio/video gear I have upstairs and fast enough to play skipless audio. This is a ways off, but time will tell.

The devices are located at different addresses than the RM9200. Initially, I'll augment the hint structure we have to cope. It should be nearly complete as it is. Eventually, however, it might be nice to have 'super kernels' that can run on any member of the Atmel family of processors that FreeBSD supports. To do that, hints will need to have some way of selecting which subset to use.

The boot loader that I carefully optimized for the RM9200 is likely too big for the SAM9. There's 16kB of SRAM on the RM9200, 12kB of which is useful for the first stage of a loader. The SAM9 has only 4kB, but a much augmented boot ROM code that may be able to help. The bootspi boot loader is about 9k with all the bells and whistles, and could be chopped to about 5k w/o much effort. I'm unsure if I should do anything here, or just have the redboot that's on the board load the kernel (or a special boot2 that runs in SDRAM rather than SRAM).

I'm sure I'm missing some other issues that will come up during the port. But so far it has been an uneventful few hours. Time, alas, conspires against me. I find I have only a few hours here or there to work on it. Although I've been working on it for weeks, I have only put 3 hours of actual grunt time into the port...

General cleanups in the newbus area

From time to time I notice things, or others do, in newbus that need to be cleaned up. These things aren't all that hard to do, usually, but finding all the places that need to do them can be a daunting task.

So I'll just report a couple that I've noticed in the past couple days, along with one from Hans Petter Selasky.

Callers of device_get_children do not check the return value. It can only fail when memory is low and the allocation fails for the children. However, it should still be checked. Going through the tree and doing sane things when it does would take some doing. This is the problem that Hans found, and many places in his usb tree he just made sure that nothing bad happened, but there might be other things necessary in some cases.

Next, not busses free up their ivars. Some just need to have a simple free(ivar) call, while others need more complicated freeing. One pass would be to audit all busses to make sure they do the right thing. This is complicated by the fact that almost all network drivers are now bus drivers too because they add mii bus to the mix.

Perhaps an idea might be to combine these two issues. device_delete_child, which would delete all the children automatically. Busses could register a bus_child_about_to_be_deleted callback which could cleanup the ivars. The default should be to just free device_get_ivar of the child deleted. Coding this up shouldn't take long, but modifying busses to use this, as well as making sure it is sufficiently general might be a bit if a challenge.

I've also done some work in p4 to create an obio concept. In this concept, all the busses in things like SoC that need help with enumeration would derive from obio (or maybe just use obio routines) which should help cut down on the large amounts of duplicated code currently present in the system. OK, small amounts, but each done almost right, but with niggling little (or large) hacks that a more regular system would solve.

I'm sure other people can suggest improvements for other problems they have noticed. Leave comments if you've seen something specific in FreeBSD and I'll update this article, or do a followup, as necessary.

Digital Camcorder to DVD on FreeBSD

A while ago, my stepson started playing competitive hockey. One of the team mothers suggested that the coach was looking for someone to tape the games and provide video. I had just purchased a camera to film the birth of my son, and there was a tripod knocking around, so I started taping the games.

I used a standard definition sony DV camera, that recorded everything in DV25 format. I then tried to transfer it to FreeBSD and create DVDs. Here's the outcome of the journey.

Ideally, there'd be one program to do this all, but there's not.

To capture the video, I used fwcontrol in the base system. I had to compile in firewire in my kernel (although the module also works). I was able to grab the video by starting fwcontrol -R, then hitting play on the camcorder. This created 10GB files.

Next, I loaded kino (from multimedia/kino) to do the editing. This is a very nice program. It handles the dv files. One word of caution. For me, sometimes it played the frames out of order, which gave me the characteristic lines on the video. mplayer and ffplay also did this. Ignore it. The final DVD will be good. Once you've edited the video, use kino's exporting function to create the mpeg files. kino will create a dvdauthor file as well. Save them both for the next step.

dvdstyler is an excellent front end to dvdauthor. The multimedia/dvdstyler port automatically installs everything it depends on. Use it to create the DVD. Most of it is fairly strait forward to use. You can even use it to edit and create menus. I needed to use the gimp to create a custom background, but those are just .jpg files. dvdstyler will either burn the DVD for you, or will let you do it. Doing it by hand is possible, but chances are excellent you'll just burn up a lot of time and wind up with an inferior set of menus...

I know this is a very high level overview. I'm blogging it so that people can find it all located in one place.