Warner's Random Hacking Blog

Canon Rebel Xt and 32GB CF

2011-05-29T22:57:00.001-06:00

Just thought I'd mention that I've been using a 32GB CF card with no special formatting in the Canon Rebel Xt. It has been working great for years.

How to run something other than login on a tty

2011-02-02T12:49:00.002-07:00

Sometimes you don't want to run login on the console or other terminal. The reasons for this vary. Some appliances want to boot to a shell prompt on the console for debugging purposes (this console typically isn't exposed, so no security problems there). Sometimes you want to use init's restart feature to keep critical daemons alive. I've developed several control and measurement applications that used this feature. Sometimes your want to offer a menu driven interface to your users instead of a cli-based one.

Recently, I had to add support for a menu console to FreeNAS. I couldn't find any immediately available documentation on how-to do this, I had to dive into some little-configured recesses of FreeBSD to make this happen. Thankfully, no code changes to FreeBSD were required to make this work. I thought I'd do a quick how-to here to cover the basics.

We'll start with /etc/ttys. This is the file that has entries like the following:

ttyv0 "/usr/libexec/getty Pc" cons25 on secure

which tells init to run getty with the parameter Pc on /dev/ttyv0 with TERM set to cons25. Getty is a program, for those that don't know, that sets up the tty device for interactive use so that normal interaction works as you'd expect. While one can run programs without getty, especially on a 'device' that doesn't exist, I'll ignore that path for this post. This is about creating an interactive program that runs on a tty device.

The 'Pc' here is the key to understanding what getty is doing. Pc refers to the gettytab entry 'Pc' which looks something like:

P|Pc|Pc console:\
:ht:np:sp#115200:

which, according to the gettytab(5) man page means "Use 115200 baud, no parity, hard tabs". This is great if you want to run login to get an account and password, but what if you want to run a program other than login instead? Maybe one that doesn't know about tty sessions, stdin/stdout redirection etc?

The answer turns out to be fairly straight forward. You just tell tty to use a different entry. This scales well for a small number of programs, but not so well if you have dozens since you can't pass parameters to the final program. For my case, I just needed a menu for FreeNAS. I added the following entry:

#
# FreeNAS menu system entry
#
FreeNAS|freenas|FreeNAS Menu:\
:ht:np:sp#15200:lo=/etc/netcli.sh:al=root:

to /etc/gettytab. This tells getty to run /etc/netcli.sh as root. Since netcli.sh was mucking with the network it had to run as root, but there's no reason it couldn't run as a different user for safety. Once I had this entry in gettytab, I changed the above /etc/ttys line to look like:

ttyv0 "/usr/libexec/getty freenas" cons25 on secure

and sent init a hup with a "kill -1 1" command (the -1 is very important, otherwise you reboot your system). Once init reparsed /etc/ttys, netcli.sh started running.

Normally, that would be the end of it. However, in this case netcli really is a pyhton script. Why did I have to wrap it in a shell script that looks like:

#!/bin/sh
# Helper script to set the path for netcli menu
export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
exec /etc/netcli

When you run out of getty, you have a very restrictive shell. The python script called a bunch of other programs, so it needed to have a good path. Also, since the shell didn't need to stick around, I tossed the typical 'exec' at the end of it to save a little bit of memory on this embedded system.

And there you have it. All the steps to create a program that runs automatically at boot. If you have any other cool tricks, please feel free to comment here...

FreeBSD/mips for Cavium Octeon

2011-01-17T22:51:00.003-07:00

I've recently improved the ABI selection in FreeBSD/mips. Now it is all handled via MACHINE_ARCH or TARGET_ARCH. mipsel and mipseb are the o32 32-bit versions. mips64eb and mips64el are for n64 64-bit versions.

I've knocked together a script for building an image for the Cavium Octeon eval boards that have CF that can boot using uboot from SDK 1.9.0 and 2.0.0 (and likely earlier versions, but I've not tested them). It creates two partitions: a FAT partition for the kernel and BSD partition for the rest of FreeBSD. It also takes care of building a big endian ufs system on a little endian system. You can find the script here. I've also create an image as well you can grab here. The script contains instructions for how to create the CF image.

Enjoy

New FreeNAS beta snapshot released (r5648)

2010-12-02T11:36:00.004-07:00

We've released the next beta snapshot of FreeNAS.

You can read the release notes for all the details.

Here's a highlight of the issues that we've fixed since the last beta (r5606):

Upgrades from prior FreeNAS 8 beta releases are now supported.
zfs creation failures have been fixed. The gui would indicate it succeeded, but you couldn't share it.
ufs creation failures have been fixed.
booting off USB or SCSI cdrom is now supported.
Many disk initialization errors that showed up as odd failures have been corrected.
lagg has been introduced.
Errors in the network screen have been corrected.
The storage wizard no longer says 'of X' when creating the storage unit. This eliminates the confusing 1 of 3 -> 2 of 2 dialog heading.
Improved compatibility with IE and Safari.
The installer has been streamlined.

Also, I was pleasantly surprised today when I logged in and found 19 comments from my last blog entry. I'm not used to so many, so hadn't been checking. I'll be checking more here. I'll also write a blog entry to address many of the issues raised in them.

You can download the latest from sourceforge.

New FreeNAS Beta (r5606)

2010-11-19T21:37:00.003-07:00

Just after the release of the first beta, a couple of serious problems came to life. As such, we've respun the release to fix these issues.

gmirror error displayed bogusly
the extremely long write times for disks
The lack of feedback during the installation (although with the significantly faster imaging, this is much less of an issue).
Default ownership and permission for Windows shares has been fixed
Shares are browseable by default now
The extra and redundant steps in the installer have been eliminated
The inconsistent menus in the installer have been made regular
A ZFS issue on reboot has been corrected
Multi-line config fields have been fixed
Problems with snmp have been resolve
proftpd's config file is now generated properly the second time
minor nits fixed in many places

You can find the release notes or the release. Stay tuned for more updates.

FreeNAS 8 Beta released

2010-11-18T23:40:00.003-07:00

iXsystems is pleased to announced FreeNAS 8.0 Beta. FreeNAS 8.0 has undergone a complete rewrite. We've redesigned the GUI to be easier to use and extend. We've upgraded many technologies in the system for improved hardware support, faster I/O, better modularity, and easier upgrades. We trust that you'll find the system easier to use and, in time, much more feature rich than the current FreeNAS offering.

The base system has migrated from FreeBSD 7.x and the m0m0wall build system to FreeBSD 8.1-RELEASE and NanoBSD. The system startup has migrated from the older php scripts to the standard FreeBSD rc.d boot system. We've pushed many of the bug fixes and system improvements back into FreeBSD.

We've rewritten the GUI using Python and Django. We've completely removed the old php system. In addition to Django, we're using Dojango and Dojo to implement AJAX features. The new system is much more modular than the old system. We will use this modularity in a future version for easy integration of custom features into your FreeNAS box.

The installer has been rewritten using pc-sysinstall, the future FreeBSD installation technology. The scripts have a similar feel to the old PHP scripts for users of the current system. The ISO now is only an installer. You can no longer run in live mode from a CDROM.

The installation types have changed; there's no longer an embedded or full install, nor can the image be installed on a data disk. You must now install FreeNAS onto a dedicated device. FreeNAS supports USB flash, CompactFlash, hard drives, ssd or any other mass storage device supported by FreeBSD.

FreeNAS 8.0 features ZFS version 14.

FreeNAS 8.0 beta has retained the core functionality of a storage appliance. The media center features of the box have not been reimplemented in the core FreeNAS package. A media center add-on package will provide this functionality in the future. We've focused on creating a robust, easy-to-use, and extensible system. We're creating the base to allow other types of packages to be added, such as printer support, scanner support, or home automation.

To help prioritize what current features are turned into packages in future FreeNAS releases, please visit http://support.freenas.org/ to provide feedback. Please add feature requests tickets. If a feature you would like to see in FreeNAS already has a ticket please just subscribe to it add a small comment, even if it's a "++." It will help us better judge and meet community needs.

You can download FreeNAS from Source Forge and read all the release notes on our wiki.

Using devfs.conf as a porting aide

2010-10-28T23:37:00.002-06:00

I've recently been given the task of moving some software from FreeBSD 4.x to FreeBSD 6.x on a consulting basis. This move was very simple and easy, except for one area: serial device names.

In the great tty-rewrite to make it MP safe that was done, there were a number of name changes that were unavoidable when making the tty layer more uniform between drivers. Unfortunately, sio was one of the drivers that had its names changed. This company used many multi-port expansion cards, so finding all the places where /dev/cuaaX was hard coded was difficult.

To validate the rest of the software, I created the following /etc/devfs.conf file to allow the old names to work while I hunted down all the places they were used.

# 4.x compat names on 6.x
link cuad0 cuaa0
link cuad0.init cuaia0
link cuad0.lock cuala0
link ttyd0.init ttyid0
link ttyd0.lock ttyld0

These blocks were repeated for each of the serial devices in the system (I wrote a quick script to generate them for all 12 devices). One thing to remember is that the 11th device is named "/dev/cuada" not "/dev/cuad10", so you need to print the unit number as hex.

You could easily extend this to FreeBSD 7 the names changed again when FreeBSD moved from sio to uart.

# 6.x names on 7.x system
link cuau0 cuad0
link cuau0.init cuad0.init
link cuau0.lock cuad0.lock
link ttyu0 ttyd0
link ttyu0.init ttyd0.init
link ttyu0.lock ttyd0.lock

again, it is a simple matter to look over the unit numbers necessary.

Note, this will only work for devices that are statically created at boot time. If you have a number of usb serial devices on a newer FreeBSD system that are replacing an older sio based FreeBSD 4.x system, you may need to use devfs.rules to accomplish this transition, which is beyond the scope of this article.

New FreeNAS alpha

2010-10-01T10:33:00.002-06:00

After a few unforeseen delays, we have a new FreeNAS snapshot available over at source forge. As an experiment, we are distributing these snapshots using xz as well as bzip2 compressed images. It makes a big difference.

What's new?

First and foremost, we have a completely new GUI look and feel. We've imported dojango into the GUI to take advantage of Dojo JavaScript Toolkit. The flow of the interface is much nicer, it looks better, and we've added additional help to make it easier to use. We think you'll like this new GUI. We've made dozens of improvements over the past few weeks to the GUI. We hope you like the new location for enabling shares.

We've also added support for Apple's AFP protocol to the new GUI. You can now use the GUI to export CIFS, AFP and NFS shares (which we'll be calling Windows, Apple and Unix shares in the GUI). We've added back-end support for FTP as well. The major protocols are now all supported, although some media options are still unimplemented.

This image has a universal booting feature. The first image was difficult to use on some VM systems since they were exclusively SCSI disks (appearing as da0, rather than the ad0 that was hard-coded into the fstab). We've added support for using UFS filesystem labels to nanobsd, so we're now able to produce an image that you can put on any kind of disk that FreeBSD supports and have it boot, since it finds the partitions for /, etc by looking for a label on the filesystem rather than insisting on ad0s1a.

At the present time, we support only adding NEW volumes to the system. If you add a volume to the system, we will always initialize it. This means that you shouldn't (yet) attach old volumes to FreeNAS 8. We will try to correct this problem for the forthcoming FreeNAS 8 beta. While we've made every effort to make things reliable, we've not put this software through a complete QA cycle, so it should be used for testing purposes only.

PC-BSD install without a bootable DVD

2010-09-29T15:06:00.006-06:00

Recently, I tried to get PC-BSD onto an older system. This system didn't support booting off of USB, nor did it support swapping in a DVD player for the CD player that was shipped with the system. The computer seemed to be fast enough to support PC-BSD. Since all I really wanted was something that could play some games that my 4-year-old wanted, I wasn't willing to spend a ton of money on a new computer just yet. Better to wait a year or two until we really needed something better.

PC-BSD only installing off a memory stick or off a DVD these days. I opted to go the DVD route because this computer didn't support booting off USB media as far as I could tell (I tried it on two machines: an old HP eVectra 933MHz box, and a slightly newer Dell Optiplex GX270).

The summary of this hack is 'Copy all the files from /boot on the ISO image onto a hard disk that has been made bootable and use a USB expansion box to store the real DVD'.

First, I needed to create a bootable hard disk. I put the hard disk that was going to be installed into the system info a USB expansion box and plugged it into my FreeBSD system. This gave me a disk as 'da0'. This disk had an old Windows installation on it. All commands are run as root. You are advised to triple check your typing, as transposed letters and such might have adverse effects.

Relabeling the Disk

Since I had an old windows install on the disk, I just needed to delete the old windows partition and create a FreeBSD one.

gpart delete -i 1 da0
gpart add -t \!165 da0
gpart create -s bsd da0s1
gpart add -t \!7 da0s1

Did the trick. Your milage may vary at this stage. I've also used "gpart delete -i X da0" and "gpart destroy da0" to totally wipe a GPT scheme off the disk in the past. If you do that, you'll also need "gpart create -s mbr da0" before proceeding to step 2 above. There's lots of other ways to skin this cat too involving dd, but since gpart has been added to the system, its so easy to use I prefer it for quick tasks like this so I don't have to remember "do I need to blank the front of the disk, or the end of the disk..."

Making the disk bootable

Next we need to make the disk bootable. This is a lot easier than it used to be in days of yore:

gpart bootcode -b /boot/boot0 da0
gpart bootcode -b /boot/boot da0s1

We have to make it bootable twice due to the multi-stage boot-loader that FreeBSD has (other systems are likely similar). "boot0" just prompts for which partition to boot (doing the first one by default). "boot" will then load /boot/loader to finish out the booting process and hand control over to FreeBSD's kernel.

Copying The Files

The pc-bsd installer for the DVD just loads a ram disk, and then runs off that. All the early stages of the boot loader need are contained in /boot. This includes the kernel, the third stage boot loader and various config files and scripts. So the next steps are to copy these files from the cd onto the bootable hard disk we just made. This assumes that the mdconfig command below prints "md0" for unit 0. If it prints anything else, you'll need to adjust accordingly. Also, if you are reading this in the future, you may need to change the PCBSD iso name.

newfs /dev/da0s1a
mount /dev/da0s1a /mnt
mdconfig -f PCBSD8.1-x86-DVD.iso
mount -t cd9660 /dev/md0 /cdrom
mkdir /mnt/boot
cd /cdrom/boot
tar cf - . | (cd /mnt/boot; tar xvf -)
cd /
umount /mnt
umount /cdrom
mdconfig -d -u 0

Moving the disk

At this point, I had to unplug the USB expander from the first system, remove the hard drive and install it into my target box. I also grabbed my UBS DVD player and connected it to the box with a copy of the DVD I burned from the above file. I selected hard disk as the boot media in the bios and let the system boot. It booted off the hard disk, but then started looking for the proper cd device. Luckily for me, it found the DVD on cd1 instead of cd0 (which is what was built-in to the Dell I was booting off of). It then became a 'normal' PC-BSD installation, which is described in detail elsewhere.

Hopefully, people find these instructions useful. Since I only had one of these systems, I didn't create any kind of script to make it easier. Sorry.

[[ Updated to fix problem noted in the comments ]]

FreeNAS 8 -- Update

2010-09-21T23:06:00.004-06:00

If you are following the commit messages to FreeNAS, you'll already know that FreeNAS 8 is shaping up nicely. If not, allow me to summarize some of the progress here.

In the past two weeks we've managed to break the gui completely, and put it back together. In the process, we've moved from a fairly static system to a more dynamic DOJO driven system that's much easier to use. It is less cluttered, while still providing escape hatches to the more complicated stuff. In addtion, we've written an installer (really, used the installer from PC-BSD, now part of FreeBSD-current). We've upgraded all the ports to FreeBSD 8.1-RELEASE (basically about 6 weeks newer ports). We'll need to do security updates before beta on some of these as well, so we've added the ability to patch the ports tree (which should also mean we'll retain much of the flexibility that the legacy FreeNAS system has wrt upgrades).

All of this is building to a new alpha snapshot. We're nearing the feature complete phase and want to get at least one more alpha out the door before we go to beta, most likely early next month. Watch this space for updates.

If you've been trying to follow the process at a source code basis, you may have run into a problem. While the build system does a good job at applying new patches to FreeBSD's src and ports tree, it doesn't do a very good job when the patch has been updated. Its unclear to me what the best way to manage this process is, since it can be hard to manage a new ports tree, new src and ports patches, as well as preserve the '-b' functionality of nanobsd which allows for fast builds after an initial build.

So, if you encounter problems updating your tree from an earlier snapshot, there's an easy work around. If you define 'force_update=1' in your environment, then you'll force the do_build.sh script to update your src and ports trees (it is needed if you have a June 24th ports tree from before the cut over). This will force csup to run, which 'fixes' all the files that have been patched. I've enhanced do_build.sh so that it will force all the patches to be reapplied when you do this. It is best to avoid the -n and -b flags to do_build.sh when doing this, since you really want a fresh build after the latest series of changes.

Later this week, we'll have another alpha snapshot ready. The last bit of GUI work should be done tomorrow (we need to add back the ability to control sharing/exporting of volumes which was lost in the great GUI reshuffle). We'll also need to test it to make sure the basics work and then we'll put binaries up to share. The installer may even be ready too, so that might also include ISO images you can boot and use to install the images into a CF.

Finally, we've reworked Olivier Cochard-Labbé's to nanobsd for device label support to be more flexible. As a result, they are now in both FreeNAS, as well as in FreeBSD. The practical upshot is that now a single image can boot off CF (where the root device is /dev/ad0s1a), a VirtualBox container (where it might be /dev/ada0s1a or /dev/ad0s1a) or a USB Stick/VMWare container (where it would be /dev/da0s1a). This should solve some difficulty people reported using VMWare in the first alpha, as well as provide a nice, flexible system to have a single image for all media types going forward. Briefly, we can label things within NanoBSD so they appear as /dev/ufs/FreeNASs1a so that the root filesystem can be found on any underlying media. If you're interested in the details of this, check out glabel and the -L option to tunefs and newfs.

As always, we welcome feedback. We'll have something in place soon to help collect and organize the feedback from the beta tests. I'll announce it after we go live.

My BSDTalk Interview

2010-09-19T15:16:00.002-06:00

I just gave Will Backman over at BSDTalk an interview on FreeNAS 8, NanoBSD, ZFS and embedding FreeBSD. You can check it out at the BSDtalk blog. Hope you all enjoy.

Another FreeNAS Alpha snapshot is slated for early this week. Stay tuned. It will include a revamped user interface, local user support, FreeBSD 8.1-RELEASE ports and a few other goodies.

FreeNAS 8 -- First alpha snapshots

2010-09-06T18:11:00.006-06:00

My blog has been somewhat quiet lately. I've been busy with other things lately.

The biggest of the other things is FreeNAS. iXsystems stepped up to the plate late last year when FreeNAS was going to become a Linux based platform.

The iXsystems engineering team has moderized FreeNAS in a number of ways. We wanted a platform that was more extensible than the current m0m0wall-based framework allowed. We wanted to create a platform that could be expandable by modules (possibly not even written by us). We wanted to make it easier to upgrade the base FreeBSD release, as well as leverage more base FreeBSD technology that has been integrated into the system since FreeNAS was originally developed.

We've migrated the build to be NanoBSD based. This allows us to leverage the embedded work that has gone into NanoBSD. It also allows us to push some of the features that are important to FreeNAS back into the base FreeBSD distribution. NanoBSD gives us the flexibility that we need. Since we're using the FreeBSD package system to add ports and packages, users will be able to add their own packages (we'll likely expand the basics to use the PBI's that PC-BSD produces for ease of installation). We're using the normal rc.d system, so upgrading is easier as well.

On the GUI side we've moved to using django. This is a python-based front-end to sqlite3 which is extensible and easy to use. While we've kept things rather plain so far, we plan on jazzing things up a bit now that we have the basics working fairly well. Since the GUI is based on django, we'll be able to allow packages to hook into the GUI to present a united interface to the user.

The GUI calls into a middleware layer that manages the services on the box. This middleware is responsible for generating updated config files, as well as starting, stopping and restarting daemons on the system. The same code generates the files at boot as well. Having all the configuration data in one database makes it easier to upgrade from release to release, since you don't have to merge your changes into the config files: the system takes care of that details.

If you'd like to read about the nuts and bolts about trying out the latest snapshot, you can check out my forum post on the topic over at the FreeNAS forums.

What is all the TBEMD stuff

2010-08-28T12:52:00.003-06:00

I've been getting questions about what TBEMD means in my commits to FreeBSD head.

First, I'll tell you the name. TBE stands for TARGET_BIG_ENDIAN. For the MIPS and ARM ports, it is an environment variable that you have to set to build these targets for big endian (otherwise they default to little endian). MD is Must Die. A few years ago, there was an action movie whose title was more memorable than the file "Romeo Must Die." So I just stole the name for this branch.

So why must TARGET_BIG_ENDIAN die? It breaks things. For all the other platforms that FreeBSD supports, you just set TARGET and/or TARGET_ARCH to do cross builds. You know everything you need to know from the MACHINE and MACHINE_ARCH in the resulting image. In addition, the build system segregates things so you can build all the targets in one tree..But for mips and arm, you don't know what endian a binary is, the obj tree will collide if you try to build both little endian and big endian binaries at the same time. There's also other, more subtle issues. For example, TARGET_BIG_ENDIAN isn't set on big endian mips, so a native biuldworld tries to build little endian binaries (oops).

So, mips will be moving to mipsel (for little endian) and mipseb (for big endian). Arm will move to arm (for little endian) and armeb (for big endian). You could fill an entire email archive with all the possible other names and why they are better or worse than these names. History, however, trumps all those arguments: these are the names used elsewhere and we're just following convention.

Once tbemd is completely merged, you'll be able to build both endians of MIPS in the same object tree, for example, and all the other issues I've discovered. It also helps with new architectures as we move into powerpc64 (already merged) and mips64 (to be done after tbemd is collapsed). It will also mean we can have different packages for the different endians now.

The dangers of forgetting svn add

2010-08-14T12:44:00.002-06:00

I spent a lot of time before the 8.1 release adding code to transparently map COMPAT_IA32 to COMPAT_FREEBSD32. I tested all kinds of different scenarios to make sure that it works, that it got error messages from old config programs, etc.

Today I got a complaint that COMPAT_IA32 doesn't work in 8.1-RELEASE. Linux module was failing to load properly. I thought it was really odd and that the folks complaining must be doing something wrong.

It turns out that I did something wrong. When I originally merged the work, I modified the options file, but didn't add the new options-compat file to stable/8 tree, so it didn't get merged into the releng/8.1 tree used to create 8.1. The result is that COMPAT_IA32 just fails silently in 8.1-RELEASE.

All that hard work ruined because I forgot to do a svn add... How disappointing.

silly vpn hack

2010-05-30T11:08:00.003-06:00

I finally got frustrated by the inability of /etc/resolv.conf to do what I'd say in pseudo code:

if domain==example.com; then nameserver=1.2.3.4
else nameserver=4.3.2.1

since I need that when I'm on my VPN to example.com.

So, round one of the hack is to tell dhclient to use my local nameserver, and do it with named.

So, in /etc/dhclient.conf, I have:

interface "wlan0" {
prepend domain-name-servers 127.0.0.1;
}

and I have the following snippets in named.conf:

...
options {
...
forwarders {
4.3.2.1;
};
...
};
zone "example.com" IN {
type forward;
forwarders {
1.2.3.4;
};
};

The only problem I have yet to figure out is how to dynamically update the 4.3.2.1 in /etc/namedb/named.conf as I roam from network to network. There doesn't appear to be a hook in dhclient to say 'run this command when my IP address changes' or 'run this command when I get a new lease' or anything useful like that....

All this is with FreeBSD-current, but these techniques should work back to at least FreeBSD 6.x.

Finally, does anybody know a good way to quote code (as opposed to text) in blogger.com? By default, it seems, all leading whitespace is eaten, which make this post look ugly.

video on FreeBSD with skype

2010-05-01T00:20:00.002-06:00

Hans Peter Selasky has been very busy adding support for web cams and other usb video devices. We tried a number of different cameras that we had laying around iX Systems the other day. All of them were recognized as valid video cameras. Some of them worked with pwcview and skype. Some of them had problems with the palettes, or would only work with some resolutions.

I was able to make a Skype call on my FreeBSD to another developer. Unfortunately, the distfile necessary for this isn't freely available. You have to download it from Skype. And it is no longer available from Skype. I got lucky and down loaded it a while ago, thinking I'd play with skype.

If you haven't had a chance to check it out, install the multimedia/webcamd. There's lots of devices that it works with.

Now, the interesting part. Hans Peter Selasky has implemented a user land driver for all of this. He implements a device driver in two parts. One piece in the kernel, which setups the shims to talk to webcamd, and the actual driver in webcamd. He uses the Linux drivers in webcamd. This separates the GPL drivers from Linux from the BSD kernel. So there's no issues with licenses. This is the same technique that's done in the Linux world to implement proprietary drivers and keep them separate from the Linux kernel. This opens the door for a wide array of devices. It is useful to think of this as something similar what Bill Paul did with NDIS and Windows drivers (except in two different address spaces rather than one).

Booting instructions for FreeBSD on Cavium Octeon

2010-01-26T08:30:00.003-07:00

Here's a quick note on how to net boot the Cavium EBT3000 board running uboot. The Cavium kernel is still a work in progress as I restore all the fixes I made to an earlier version of this code that I was unable to release.

You'll need to break into the boot sequence for this board. Usually that's just hitting ^C before uboot starts to load the kernel.

Once you have a uboot prompt, similar to below, you'll be ready to start to fix the environment for netbooting.

U-Boot 1.1.1 (Development build) (Build time: Jan 26 2007 - 14:06:35)

EBT3000 board revision major:4, minor:0, serial #: xxxxxxxxx
OCTEON CN38XX-NSP revision: 1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)
DRAM: 2048 MB
Flash: 8 MB
IPD backpressure workaround verified, took 14 loops
Clearing DRAM........ done
BIST check passed.
Net: octeth0, octeth1, octeth2, octeth3
Bus 0 (CF Card): OK

ide 0: Model: SanDisk SDCFB-1024 Firm: Rev 0.00 Ser#: X0308 20050726142909
Type: Removable Hard Disk
Capacity: 977.4 MB = 0.9 GB (2001888 x 512)

Octeon ebt3800#

It couldn't be easier to setup for netbooting. You'll need to setup an tftp server. I had to install the FreeBSD port from net/freebsd-tftpd to get a working tftpd, since there appears to be some issues with the stock FreeBSD tftpd. Once I had that, I set different addresses like so:

set ipaddr 10.0.0.30
set netmask 255.255.255.0
set serverip 10.0.0.10
set gatewayip 10.0.0.1
set bootdelay 0
saveenv

This sets the board's address to 10.0.0.30/24. 10.0.0.1 is the router to the other networks. The tftp command will now use tftp from host 10.0.0.10 to get the named file. saveenv makes these setting permanent. I usually reboot at this point to make sure that the settings took.

Next, you'll need to know how to load a FreeBSD kernel. This is also fairly straight-forward, although finding the right addresses took a bit of guesswork. You are tftping the ELF kernel to a chunk of memory. The bootoctlinux command then takes that image and copies it to the right place and starts executing it. So you have to be sure whereever you load it doesn't clash with uboot's memory use or with where the kernel will ultimately reside.

tftp a800000 kernel.octean1-32
bootoctlinux a800000 numcores=1

As with any u-boot platform, it can often be useful to define macro commands. I did the following:

set fbsd 'tftp a800000 kernel.octeon1-32; bootoctlinux a800000 numcores=1'

so that I don't have to keep typing it over and over. I can just type

run fbsd

to run my next test.

The Cavium port is in a state of flux right now. The latest status as I am preparing this to be published is that there's an interrupt problem that's causing the serial port to be non-responsive after going to multi-user (eg, the login: prompt) and also the network stops responding. Also, I have to comment out the enableintr() call in mips_vector_init around line 365 of sys/mips/mips/machdep.c. I'm trying to hammer these issues out, and I think they are related. Earlier versions of the code worked with earlier versions of FreeBSD, so I'm sure this is just a shift in interfaces that needs to be accounted for.

Here's the latest slightly trimmed log from a boot:

U-Boot 1.1.1 (Development build) (Build time: Jan 26 2007 - 14:06:35)

EBT3000 board revision major:4, minor:0, serial #: xxxxxxxxx
OCTEON CN38XX-NSP revision: 1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)
DRAM: 2048 MB
Flash: 8 MB
IPD backpressure workaround verified, took 32 loops
Clearing DRAM........ done
BIST check passed.
Net: octeth0, octeth1, octeth2, octeth3
Bus 0 (CF Card): OK

ide 0: Model: 512MB CKS Firm: 2N3-0925 Ser#: 2435E7C34A52930007
Type: Removable Hard Disk
Capacity: 491.6 MB = 0.4 GB (1006992 x 512)
Octeon ebt3000# print
bootdelay=0
baudrate=115200
download_baudrate=115200
ls=fatls ide 0
nuke_env=protect off BFBFE000 BFBFffff; erase BFBFE000 BFBFffff
autoload=n
ipaddr=10.0.0.31
gatewayip=10.0.0.1
netmask=255.255.255.0
serverip=10.0.0.16
fbsd=tftp a800000 kernel.octeon1-32; bootoctlinux a800000 numcores=1
fbsd_cf=fatload ide 0 a800000 kernel.imp; bootoctlinux a800000 numcores=1
coremask_override=0xffff
numcores=16
stdin=serial
stdout=serial
stderr=serial

Environment size: 1274/8188 bytes
Octeon ebt3000# run fbsd
octeth0: Down 10Mbs Half duplex, (port 16)
Using octeth0 device
TFTP from server 10.0.0.16; our IP address is 10.0.0.31
Filename 'kernel.octeon1-32'.
Load address: 0xa800000
Loading: octeth0: Up 100Mbs Half duplex, (port 16)
########################
done
Bytes transferred = 3370489 (336df9 hex)
argv[2]: numcores=1
ELF file is 32 bit
Skipping non LOAD program header (type 0x6)
Skipping non LOAD program header (type 0x3)
Skipping non LOAD program header (type 0x70000000)
Allocated memory for ELF segment: addr: 0x1000000, size 0x295a60
Loading .text @ 0x81000000 (2064384 bytes)
Loading .MIPS.stubs @ 0x811f8000 (16 bytes)
Loading .rodata @ 0x811fa000 (33248 bytes)
Loading .reginfo @ 0x812021e0 (24 bytes)
Loading .rodata.str1.4 @ 0x812021f8 (189300 bytes)
Loading set_sysctl_set @ 0x8123056c (3496 bytes)
Loading set_sysinit_set @ 0x81231314 (1684 bytes)
Loading set_sysuninit_set @ 0x812319a8 (952 bytes)
Loading .interp @ 0x81231d60 (13 bytes)
Loading .dynsym @ 0x81231d70 (78320 bytes)
Loading .dynstr @ 0x81244f60 (72163 bytes)
Loading .hash @ 0x81256944 (35984 bytes)
Loading set_kdb_dbbe_set @ 0x8125f5d4 (8 bytes)
Loading set_modmetadata_set @ 0x8125f5dc (288 bytes)
Loading set_cons_set @ 0x8125f6fc (8 bytes)
Loading .data @ 0x8125f704 (116476 bytes)
Loading set_pcpu @ 0x8127be00 (3136 bytes)
Loading .got @ 0x8127ca40 (5296 bytes)
Loading .rld_map @ 0x8127def0 (4 bytes)
Loading .sdata @ 0x8127def4 (12 bytes)
Clearing .bss @ 0x8127df00 (97120 bytes)
## Loading Linux kernel with entry point: 0x81000000 ...
Bootloader: Done loading app on coremask: 0x1
Boot Descriptor Ver: 6 -> 1/2 CPU clock: 500MHz
Dram: 2048 MB Board Type: 2 Revision: 4/0
Octeon Chip: 0 Rev 0/0 Mac Address 00.0F.B7.10.1B.32 (14)
octeon_dram == 80000000
reduced to ram: 2048 MBReal memory bytes is 7ffff000
phys_avail[0] = 1296000 phys_avail[1] = fffffff
realmem_bytes is now at 5ffff000
Next block of memory goes from 20000000 to 7ffff000

Total DRAM Size 0x80000000
Bank 0 = 0x 1296000 -> 0x FFFFFFF
Bank 1 = 0x20000000 -> 0x7FFFF000

physmem: 0x6ed69Cache info:
picache_stride = 4096
picache_loopcount = 8
pdcache_stride = 128
pdcache_loopcount = 64
cpu0: Cavium processor v1.0
MMU: Standard TLB, 32 entries
L1 i-cache: 4 ways of 64 sets, 128 bytes per line
L1 d-cache: 64 ways of 1 sets, 128 bytes per line
Config1=0xbe3303da
Config3=0x80
Physical memory chunk(s):
0x1296000 - 0xfffefff, 248942592 bytes (60777 pages)
0x20000000 - 0x7fffefff, 1610608640 bytes (393215 pages)
Maxmem is 0x7ffff000
KDB: debugger backends: ddb
KDB: current backend: ddb
hz=100 cyl_per_hz:250000 cyl_per_usec:250 freq:250000000 cyl_per_hz:2500000 cyl_per_sec:250000000
Copyright (c) 1992-2010 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-CURRENT #141 r202986:202997M: Mon Jan 25 20:07:07 MST 2010
imp@paco-paco.bsdimp.com:/tmp/imp/be-obj/mips/pe/imp/svn/head/sys/OCTEON1-32 mips
real memory = 1859555328 (1815972K bytes)
Physical memory chunk(s):
0x013ab000 - 0x0fffefff, 247808000 bytes (60500 pages)
0x20000000 - 0x7d830fff, 1568870400 bytes (383025 pages)
avail memory = 1813180416 (1729MB)
mem:
nfslock: pseudo-device
null:
nexus0:
Compact flash found in bootbus region 3 (8 bit).
rgmii0: on nexus0
rgmx0: Ethernet address: 00:0f:b7:10:1b:32
rgmx0: type (null), full duplex
rgmx1: Ethernet address: 00:0f:b7:10:1b:33
rgmx1: type (null), full duplex
rgmx2: Ethernet address: 00:0f:b7:10:1b:34
rgmx2: type (null), full duplex
rgmx3: Ethernet address: 00:0f:b7:10:1b:35
rgmx3: type (null), full duplex
rgmii0: [FILTER]
cf0: on nexus0
clock0: on nexus0
clock0: [FILTER]
obio0 at mem 0x1 flags 0x1 on nexus0
uart1: on obio0
uart1: [FILTER]
uart1: fast interrupt
uart1: console (115200,n,8,1)
uart0: on obio0
uart0: [FILTER]
uart0: fast interrupt
uart0: console (115200,n,8,1)
Device configuration finished.
Reducing kern.maxvnodes 117045 -> 100000
Timecounter "MIPS32" frequency 250000000 Hz quality 800
Timecounters tick every 10.000 msec
GEOM: cf0: partition 2 does not start on a track boundary.
GEOM: cf0: partition 2 does not end on a track boundary.
GEOM: cf0: partition 1 does not start on a track boundary.
GEOM: cf0: partition 1 does not end on a track boundary.
GEOM: cf0s2: geometry does not match label (64h,32s != 16h,63s).
Trying to mount root from ufs:cf0s2a
WARNING: / was not properly dismounted
warning: no time-of-day clock registered, system time will not be set accurately
start_init: trying /sbin/init
Setting hostuuid: 1c99760e-1dd2-11b2-bd6a-00156dc12838.
Setting hostid: 0xb0f278c1.
No suitable dump device was found.
Entropy harvesting:.
Starting file system checks:
t_delta 16.010fac783006c1a6 too long
/dev/cf0s2a: 11100 files, 256973 used, 150946 free (426 frags, 18815 blocks, 0.1% fragmentation)
Mounting local file systems:.
t_delta 15.fece5003dd62224a too short
Setting hostname: ebt3000.
SIOCSTIFFLAGS UP/Running
Starting Network: lo0 rgmx0 rgmx1 rgmx2 rgmx3.
lo0: flags=8049 metric 0 mtu 16384
options=3
inet 127.0.0.1 netmask 0xff000000
rgmx0: flags=8843 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:32
inet 10.0.0.31 netmask 0xffffff00 broadcast 10.0.0.255
media: Ethernet autoselect
rgmx1: flags=8802 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:33
media: Ethernet autoselect
rgmx2: flags=8802 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:34
media: Ethernet autoselect
rgmx3: flags=8802 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:35
media: Ethernet autoselect
Starting devd.
Creating and/or trimming log files.
Starting syslogd.
/etc/rc: WARNING: Dump device does not exist. Savecore not run.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
Clearing /tmp (X related).
Updating motd:.
Starting cron.
Starting inetd.
Starting background file system checks in 60 seconds.

Tue Jan 26 02:07:16 UTC 2010

FreeBSD/mips (ebt3000) (ttyu0)

login:

DLINK DIR-615 REV C1 Redux

2010-01-15T14:00:00.002-07:00

After hitting a dry hole with my DIR-615 REV A1, I thought I'd try the DIR-615 REV C1 that I have. It has an Atherose AR9130 in it. This is very similar to the AR7130 that the RouterStation I have running, and I thought it would be a simple matter to use uboot to netboot this code.

Again, I was mistaken. There's no way to break into the boot sequence for this device. The best I can hope for is to create a compressed image that's < 1MB and blow it into the FLASH partition for the kernel under linux. That would work out OK, if I had known good image to start from. Alas, I don't. And there doesn't appear to be an accessible JTAG header that I can use to write a new u-boot. Linux seems to protect u-boot from being overwritten as well.

But the biggest issue of all is that the FLASH is only 4MB in size. It would take some doing to get everything shrunk down that small. The DIR-615 REV A1 at least had 8MB of FLASH, which is doable. Maybe I'll perfect the kernel for the DIR-615 REV A1 on the Marvell eval board I have. Looking at the Linux sources, the deltas between these kernels are small and inconsequential for bootstrapping purposes. Hmmm, the Linux kernel just has support for the DNS-323, now that I look closely, but the required bits of hacking after I have a known good kernel doesn't seem huge. Plus, if I could get one good kernel on the box, then I could put a better uboot on there which would support the network for netbooting...

If I ever make good images for these boards, I'll be sure to post pointers to them here.

DLINK DIR-615 REV A redux

2010-01-15T08:02:00.002-07:00

Yesterday I burned a couple of hours playing with the DIR-615 REV A1 that I have had for ages. FreeBSD runs on the Marvell Orion SoC that's inside it now and I thought I'd netboot the box and try to put together an image that can replace my firewall since this little box uses less power and my firewall doesn't need to do *THAT* much.

Unfortunately, it appears as if the network was purposely crippled on this box. Not disabled entirely, but crippled to not work. Sometimes you can get packets out, but generally there's a huge packet loss rate that makes pinging impossible, let alone netbooting.

Talking to other folks that have worked in this area, it appears that this was a deliberate attempt to discourage others from running alternative operating systems on the box.

Since the box will still boot to Linux, there's hope that one can bootstrap the FreeBSD installation from there. When I get some time again, I'll be pursuing that avenue. Maybe I can find the JTAG header for this board and reflash uboot that way, since debugging a new kernel can be difficult to get right in one shot on hardware the kernel has never run on before...

I had planned on publishing instructions for netbooting the thing, but alas, it appears that isn't possible.

Post-mortum on projects/mips branch

2010-01-13T08:00:00.000-07:00

Greetings to one and all. As you have read elsewhere, I recently merged all the changes from the projects/mips branch onto head. In other reports, I've made cryptic reference to the branch being damaged. I thought I'd go through all the problems we encountered running this development effort on the projects/mips branch.

First, we created the branch in the normal way:

svn cp svn+ssh://svn.freebsd.org/base/head svn+ssh://svn.freebsd.org/base/projects/mips

and then started committing to it. So far so good.

Then it came time to merge in changes from head. Here is where we made our first mistake. We merged just the kernel changes, and not the entire tree. The next time we merged the entire tree. The effect of this sequence meant that the changes between the creation and the first merge in the rest of the tree (outside of src/sys) were lost. I don't know if this is the result of a buggy svn client, or if they are a fundamental flaw in svn. So when it came time to collapse the results back into head, we couldn't just do a reverse integration, or even a more conventional merge.

Next, we pulled in pieces of the projects/mips tree into head, either by hand or with the svn merge command. Doing both was a mistake, I think. Although svn coped with later merged from head into projects/mips fairly well, occasionally we'd have issues to sort out.

So, when it came time to merge everything back into head, we were left with a number of difficult choices. We opted to copy the new files/directories in the repo (to preserve the history) and merge patches by hand with svn log entries. The latter went really well. There were no problems with it apart from the odd fuzz and .rej file to sort out (no different than a normal merge). The former, however, causes a lot of problems.

First, it created svn:mergeinfo entries on the files I copied over. This conflicts with the project's "merge everything from sys" edict. This happened because I did the copy in the server rather than the client (which was an attempt to preserve history). It was a simple matter to delete these entries when it was brought to my attention.

Next, somehow we created the files on the branch without the required svn:keywords, svn:eol-style and svn:mime-type properties. This caused problems when we went to commit changes to these files. The precommit complained, at this point, that the files lacked svn:keywords, and we had to add $FreeBSD$ by hand (since the copy didn't complaint that they were lacking). Also, I didn't discover these problems on my own. Other sharp-eyed individuals saw the problems and reported them to me.

Finally, since I did the copies on the server, there was no way to batch them. When I copied a directory, I got the whole directory. When I copied a file, I got the file. Each one generated a commit message.

I'm unsure what other damage was lurking in the projects/mips tree, but the lack of properties, the inability to easily use merge info and the missing commits lead me to the conclusion that it was easier to abandon the branch and create a new branch if needed in the future.

The mips team plans to create more branches, that are more for special purposes, that will be shorter lived in the future. We have no further plans to have one monolithic mips branch that acts as a staging ground for commits into head.

Hack to allow automatic wired/wireless failover with lagg on FreeBSD

2010-01-12T15:19:00.007-07:00

For years I've had a love/hate relationship with my wireless card. On the one hand, it allows me to roam. On the other I like being able to plug into the wired Ethernet when I'm sitting at my desk since I get better throughput and lower latency. So for years, I just loaded and unloaded kernel modules to make dhclient (and/or wpa_supplicant) do its thing. This changed the IP address of my box, which messed up NFS mounts until I went back to the old way.

Recently, Josh Paetzel came up with a way to use lagg to do automatic failover. Inspired by the lagg man page, he created a number of rc.conf lines that would do this automatically. Xin Li and I refined the trick a bit. To completely integrate this into FreeBSD, I suppose that the rc.d scripts would need to be enhanced to automatically generate these lines when appropriate. However, the hack is so easy to do now, I'm going to share it with you here. Xin Li is writing up a longer description of how to set this up for the handbook. Let's say you have a ath0 wireless interface, and a re0 wired interface, just add the following to your /etc/rc.conf file:

ifconfig_re0="up"

ifconfig_ath0=`ifconfig re0 ether`

ifconfig_ath0="ether ${ifconfig_ath0##*ether }"

wlans_ath0="wlan0"

ifconfig_wlan0="WPA"

cloned_interfaces="lagg0"

ifconfig_lagg0="laggproto failover laggport re0 laggport wlan0 DHCP"

This bit of code brings up re0, programs ath0 with the ethernet MAC address of re0 (so your MAC doesn't change when you switch over), creates a wlan0 interface from ath0, causes lagg0 to be created and configured to do the fail over.

Hope this tip is useful...

Revision at Wed Jan 13 16:00:00 UTC 2010:

Note: "ifconfig re0 ether" doesn't work in versions prior to 8.0. The following should fix it, but I don't have a system to test it on:

ifconfig_ath0=`ifconfig re0 ether`

ifconfig_ath0="${ifconfig_ath0##*ether}"

ifconfig_ath0="ether ${ifconfig_ath0%%inet*}"

This sequence avoids using grep, awk and/or sed to accomplish this (and thus touch /usr too early for systems where / and /usr are on different partitions).

Also corrected the bwi0 reference in the feedback...

Revision at Mon Jan 18 23:00:00 UTC 2010:
Corrected () vs {} confusion in the above...

FreeBSD/mips updated

2010-01-12T14:04:00.002-07:00

The base/projects/mips branch has been merged into base/head. The merge is complete and the sanity tests have passed. The code has booted on both a Ubiquiti RouterStation (big endian) as well as in gxemul (little endian).

The branch lived for one year, minus a day, and accumulated much work:

A new port to the Atheros AR71xx series of processors. This port supports the RouterStation and RouterStation PRO boards from Ubiquiti. Other boards should work with minimal tweaking. This port should be considered as nearing production quality, and has been used extensively by the developers. The primary author of this port is Oleksandr Tymoshenko (gonzo@freebsd.org).

A new port to the sibyte BCM1250 SoC on the BCM91250 evaluation board (aka SWARM). This port is reported to be stable, but this hardware is a little old and not widely available. The primary author of this port is Neel Natu (neel@freebsd.org). Only one core is presently supported.

A port, donated by Cavium, to their Octeon and Octeon plus series of SoC (CN3xxx and CN5xxx). This code is preliminary, supporting only a single core right now. It has been lightly tested on the CN3860 evaluation board only in 32-bit mode. Warner Losh (imp@freebsd.org) has been driving the efforts to get this code into the tree.

A port, donated by RMI, to their XLR series of SoCs. This port is single core only as well. The code reaches multi-user but should be considered beta quality for the moment. Randal Stewart (rrs@freebsd.org) has been driving the efforts to integrate this into the tree.

Preliminary support for building a mips64 kernel from this source base. More work is needed here, but at least two kernels successfully build in 64-bit mode (OCTEON1 and MALTA64).

Very early support for N32 and N64 ABIs

Support for booting compressed kernels has been added (gonzo@).

Improved support for debugging

Improved busdma and bus_space support

Many bug fixes

More types of MIPS cores are recognized

Expanded cache handling for newer processors

Beginning of a port to the alchemy au1XXX cpus is present but experimental.

Work on SMP is underway to support multicore processors like the sibyte, Octeon and XLR processors.

I'm sure there are minor items I've forgotten. If so, please forgive any omission on my part...

The branch had been updated incorrectly several times over the past year, and the damage was too much to repair. We've retired the branch and will do further mips development in "head" for the time being. If you have a checked out tree, the suggested way to update the projects/mips tree you have is to do a "svn switch svn://svn.freebsd.org/base/head" in that tree.

I'd like to thank everybody that has contributed time, code or hardware to make FreeBSD/mips better.

We are still investigating how feasible merging all this work into stable/8 will be, as it represents a huge leap forward in code stability and quality.

As development proceeds, I'll keep posting updates. In addition, I hope to have some mini "how-to" wiki pages done for people that want to try it out.

Some TiVo hacking

2009-11-05T13:32:00.004-07:00

Recently, I started hacking a TiVo HR10-250 that I've had for a million years unhacked.

Back when I first got it, there were dozens of threads of hundreds of messages in about 10 different forums you needed to read in order to hack the goofy thing. Everybody did their own thing, it was manual and a big pain.

Then came zipper (you can find all the details out at this link). It makes it very easy to hack the TiVos in general. However, there was just one problem. The ISO authoring tools it used were windows-based. Since I don't generally run on a Windows machine, this was a problem for me. So I tried to do the steps from the batch file by hand, and it didn't work out so well.

So I rewrote the batch file in a unix script running on FreeBSD. FreeBSD's iso extraction feature of tar made this process hassle free. I can build a new ISO image from an old one using only one port (mkisofs) and no privs are needed at any step. I thought about trying to teach tar how to create an iso image, but since mkisofs does such a good job, and there are *SO* many knobs and dials that need to be implemented to do that right, I just let mkisofs do its thing. I just can't get over how easy FreeBSD's tar made it to break apart the old bootable cd that does the hacking to add the zipper files. Kudos!

I've uploaded the script to the TiVo Community Forum thread about zipper if you are interested in looking into it.

T-Mobile and iPhone 3.1.2 firmware update issue

2009-11-04T04:48:00.003-07:00

Recently, I used PwnageTool on my mac to update my 2G iPhone from 2.2.1 to 3.1.2. I've been using T-Mobile since day one with my iPhone without hassle. Well, except for the upgrade hassles that I have from time to time. Once I unlocked the SIM, I was able to use the SIM from my T-Mobile DASH in the phone.

When I upgraded this time, however, there was a problem. My data connections stopped working. Since 3.1.2 was otherwise a nice upgrade for me, I went looking for the problem. It turns out that the bundled carrier files that describe what you can and can't do with the iPhone basically said no data.

Carrier files that are correct for T-Mobile, and the data plan I have, can be found iPhone-Dev.ca under the IPCC section of the web page. Instructions for loading the IPCC file were there as well. For T-Mobile, you'll need to call T-Mobile to find the right APN to use. For my plan it turned out to be wap-apn. Your milage may vary, but worst case would involve trying all 4 of the files.

While not a programming hack, this should come in handy for anybody that's in a similar situation. Alas, I have no experience with the 3G or 3GS phones, and problems with the upgrades for those phones seem to be more widely covered (and blogged about).

Broadcom BCM43xx support (bwi) committed

2009-05-14T21:15:00.002-06:00

The BCM43xx driver from DragonFlyBSD has been ported to FreeBSD. Andrew Thomas and others have done the port. I've pushed it into the tree, and plan on fixing the one or two quirks with my card.

The driver is far from perfect. 802.11a isn't support. Some older BCM4306 cards don't support channels 1, 2 or 3. It is rumored not to work at all for some people.

The driver was written by Sepherosa Ziehau for DragonFlyBSD. He used the specs available at http://bcm-specs.sipsolutions.net/Specification to write the driver. The driver is written against the v3 firmware. The v4 firmware is needed for support of newer cards, and many people have made noise about porting to that, but nobody has emerged yet to do that work.

The driver will be in FreeBSD 8.0. A back port of the driver to 7.x is difficult because it relies on all the new VAP work that Sam Leffler has done that is not in 7.x (although backports exist for 7.x, they won't be committed to the FreeBSD tree because it breaks binary compatibility).

You need the net/bwi-firmware-kmod port installed for the firmware. Although the binary .o file that the firmware is extracted from is apparently easy to distribute, distributing derived works of that .o file is not. So, there's a port to download, extract the firmware and make a kld for the driver to access.

A man page will be committed soon.

Belkin F6D4230 "Enhanced Wireless Router"

2009-05-05T16:41:00.003-06:00

Just got the Belkin Enhance Wireless Router F6D4230-4. Looks like it has a ralink RT3052 or RT3050 under the hood. This is a MIPS24KEc based SoC, with a full Linux GPL tarball available from Belkin's web site.

I haven't had a chance to open the box yet to see what else is inside, and this is inferred entirely from the Linux config file that was in the GPL compliance package.

Not sure what I'm going to do with this box...

ExpressCard

2009-04-23T10:22:00.004-06:00

Now that I've managed to knock down the number of bugs in NEWCARD to a more reasonable level, I've started looking at ExpressCard again.

As you may know, FreeBSD supports ExpressCard right now for usb devices. They all work, and there's no additional work needed here, except maybe to make more drivers. In fact, I recently purchased an ExpressCard to CardBus adapter (one that lets me plug in an expressCard into a CardBus slot) and it worked with all the usb-based ExpressCards that I have laying around with no changes.

The problem with ExpressCard for PCIe-based devices is one of resource allocation. If the BIOS allocates the resource, then the ExpressCard works without hot-plug. If the BIOS doesn't, then we can see the device with pciconf -l, as well as probe the device. However, since there's no resources setup for the PCI bridge, attempts for the driver to allocate them fail. There's also some bus numbering issues as well.

I've started to attack the resource problem. I'll keep people posted

Update my old CardStatus page...

2009-04-21T16:10:00.003-06:00

If anybody still cares about 16-bit PC Cards and 32-bit CardBus cards still, I've updated the latest testing I've done.

You can find it here. I think it does more to show what a big pack-rat I am, but maybe someone will find it useful. It shows I'm down to one or two ed cards that aren't working for me yet...

3Com 3C1 works

2009-04-13T12:33:00.002-06:00

Years ago, I bought a 3Com 3C1 CF card. This card was an early attempt by 3Com to minimize the power usage of the traditional 3C589 line of cards. These cards were important for the early palm-sized PDAs that were on the market. Through a number of different connections, I was able to get documentation on the 3c1. At the time I got it, I barely understood network drivers or even how the hardware worked. The documentation I got from 3com had pages for the flow charts for transmit and ISR, but the flowcharts weren't actually included on those pages. My connections couldn't get a better copy of the documentation, always saying it was coming soon.... The descriptions of the commands had some vague hints, but I never could make them connect. This was around 1999 or so.

Over the years, I kept trying to make this card work on FreeBSD. I considered it unfinished business from my past and I kept trying to find time to work on it. I never could find the time, or if I had the time my brain was burned out from the deliverables I had to produce for work. Over the years I've acquired all the cards that I could find in the 3c5xx line and made them work, but never got the 3c1 working.

Recently I stumbled across the 3c1 documentation and had a few minutes to read it. It was a nice change of pace from all the other stuff I'd been working on lately. After reading through it, I realized that there were a few key items that I'd not properly implemented before. There was a cryptic new command to turn on and off the TX PLL. So I had 30 minutes the other night and started reading the document again.

In the interim between when I started looking at the 3c1 documentation and now I did a lot of work in the high precision time and frequency world. This world is all about PLLs and controlling the oscillators/clocks to an insanely high degree of precision. So this clicked in my brain "Hmm, TX PLL sounds like it controls the clock used to do the transmission." So everywhere we enabled TX in the driver, I added a call to turn on TX PLL. This got me started: dhclient worked! I was able to transmit packets. This was the first. However, it stopped working after a while. So I searched through the 3c1 docs. There was nothing about PLL enable and disable, except for the documentation of the command. It appears there's an automatic timeout in the hardware which was turning off the PLL. So, an additional enabling of the PLL in start did the trick. I only enabled the PLL when we're not actively transmitting (which means we were idle for some unknown period of time). I could refine it further if I knew what the timeout of the PLL (which isn't documented), but there appears to be no ill effects doing it all the time the TX unit has been idle.

Ah, one more piece of unfinished business from the past is now complete. I can move on to other more interesting things...

More PC Card enhancements

2009-04-02T08:52:00.002-06:00

Every few months/years I take all the cards that I have in my collection and start to run through them testing to see what's working in FreeBSD and what's broken. Over the past year or two I've neglected this task as I've been involved in other things. After my last trip to Akihabara, I've been playing with this stuff, so I thought I'd pull out the cards again. However, for the moment, I'm restricting things to the cards supported by the ed driver.

I've already blogged about the AX88x90 enhancements. The only additional enhancement is that I've eliminated some code that duplicates reading the NIC from what is just memory. I'll be committing that soon.

I've finally implemented the workaround for fullduplex disabling SQE in the DL10019 parts. This gives slightly better performance on the parts that are affected, but I'm having trouble measuring it. I'm not sure that it is worth the hassle.

I've also made a number of cards work by tweaking the MII initialization. I think I have a good way to kick the PHYs that are on these cards to get them working (this has helped to get three cards going).

I'm approaching 100% of the ne-2000ish cards working. I'll do a final cleanup of my patches and try to get them into the tree. It is time to move on to the CardBus cards. Most of them appear to work, once certain resource allocation issues are addressed. At the very least I need to test them all to see how they work. I may skip this and move directly and address the resource issues there.

AX88x90 updates

2009-03-30T11:28:00.002-06:00

After focusing so much time on that old Toshiba card, I thought I'd see if I could get something more modern working as well. I've had one card in my bag knocking around in my bag based on the AX88790 chip. This is the corega FEther II-PXD. I saw it on sale in Akihabara when I was there a few weeks ago, and this card came from Akihabara a few years ago from a friend in Japan. So I thought I'd try it out.

Well, at first it didn't work too well. The MII code was finding gobs of PHYs that were phantoms. This in turned caused bad things to happen when dhclient(8) was run.

There turned out to be three root causes. First, the mii bit-bang code was wrong for reads, yielding bad register reads. Second, the AX88790 has an odd quirk for talking to its internal PHY, so more code was needed to cope with that. Third, the reset code for the NE2000 was also causing problems, so I rewrote it to be AX88x90 reset code. Finally, the internal phy for the AX88790 has a few quirks in silicon that need software workarounds.

Once I'd corrected these problems, the card started working again. Since most of this data is readily available on the net (the asix web site has datasheets for these parts), I'll skip over all the blow-by-blow details. I hopefully had enough comments in the code for people to follow along, as well as the datasheet should make what I've done easy to see.

Now, the arduous task of testing all the AX88x90 cards in my collections... The target card now works correctly, and I believe all of the ones in my collection will work too. Maybe I'll get to the point were all my ed-based PC Cards in my collection will work (except for the Mitsubushi B8895 which I've given up on, and the ones I don't have proper dongles for).

Toshiba LANCT00A PC Card

2009-03-27T23:09:00.004-06:00

When I was in Akihabara last, I got a lot of PC Cards from the junk vendors for 100 Yen. One of these cards was the Toshiba LANCT00A. This card didn't probe when I first got it. FreeBSD didn't support it. This was a challenge.

As long-time reads of this blog may know, I sometimes go out of my way to hack on really oddball hardware that somehow I get it into my head needs to work. Never mind that I have a dozen other projects cooking that is more relevant to FreeBSD and its user base. Never mind that some of this hardware was obsolete when I started hacking on FreeBSD's PC Card layer. The cool thing is that I can find just enough documentation to be interesting on these cards.

This Toshiba card was like that. Google searches for information about it turned up a half dozen posts to different FreeBSD and Linux mailing lists asking about it, plus some speculation about what's in the card. One brave soul even took his card apart and reported it had a NS DP83902 under the hood. However, nobody that I can find ever got this card working under FreeBSD or Linux. Checks of the current Linux kernel was no help. Checks of OpenBSD and NetBSD also yielded no results. There was a windows driver available (actually an NDIS driver of unknown version).

So I was printing out random values for this card. At one time the FreeBSD PC Card probe routine for ed tried to probe WD80x3 cards as well as NE2000 cards. I removed this about 4 years ago after failing to find any cards that needed this, used it, etc. However, this card is almost a WD80x3 clone. I was printing the board type in the attach routine, trying to track down why attach was failing. I saw that it was 20 decimal or 0x14. A quick scan of the if_edreg.h file showed this was one of the Toshiba cards. The comments said it was a PCETC board. I immediately thought "Hmmm, isn't this a Toshiba card?" and then wondered if the 'C' in LANCT00A meant it was the same silicon revision as the 'C' in the PCETC. So I hacked up a quick little test into the if_ed_pccard probe routine.

It recognized it the card when I did that! Success. Well, almost. There was a snag. The memory allocated for this card was at 0x88000000. This didn't pass through the sanity test in the driver designed for ISA cards, so it didn't work. A little more hacking and I was getting a sane MAC address and the card was attaching. It dhcped its address w/o incident.

However, all is not right in mudville. As soon as I started using TCP, I started getting sbdrop panics. It looks like the packets that the WD80x3 code is pumping out aren't quite rightly aligned, or something (I have never had a sbdrop panic, so I'll have to investigate what that means when I have some time). This isn't too terribly surprising, since I haven't tested the ISA cards in years. Maybe something broke for the memory copyout path, but not for the PIO copyout path. I'll have to track that down another night.

[[ UPDATE: It looks like it was the copyin path. There was a bug in it that caused 2x too many bytes to be read into memory, causing the corruption. So now it works well enough for me to update this post and commit the change. ]]

Now, what does this gain FreeBSD? Not much, in the grander scheme of things. I'd be surprised if any of my readers actually had one of these cards and started using it as a result of my work. However, it was a relaxing hour or two of my life. Given my current job stress level, this has great value to me. Also, I think this is one of the first PC Cards to work with both I/O and Memory windows mapped at the same time from the CFE entry. While there are some other cards that manually allocate memory ranges, I don't think any of them have it in their CIS.

Postscript

Well, while I was doing research for this blog entry, I stumbled accross http://linux.toshiba-dme.co.jp/linux/eng/download.htm which has a tos_cs for download. There's a lot of voodoo magic in that driver that I'll have to investigate. This driver is 1000 lines long, and was for Linux 2.0.35. It was never integrated into Linux, so I'd stand by my statement that Linux doesn't support it. Just for kicks I tried building it under 2.6.29, and it didn't compile anymore.

My changes to the ed driver in FreeBSD are more like 20-25 lines of code. There's magic kick of a magic offset in shared memory that isn't present in FreeBSD's ed driver today. I'll keep that in mind in case I have the card wedge on me under load or something.

Dumpster diving for PC Cards at Akihabara Junk Shops

2009-03-25T14:52:00.003-06:00

When I was in Japan two weeks ago, I took a trip to Akihabara and found a few junk shops. Well, these places are just second hand stores for used computer hardware and software. There is quite the range of shops. Some are nice, neat, well lit and have plenty of room to move about. Others aren't quite so nice and are very cramped. The further you get away from the street level and from the main streets, the less polished these stores are. The harder it is for a non-Japanese speaker like myself finds it to do business. This was my first trip to Akihabara that I didn't have Japanese speakers to help me out, so I didn't get too far away from the beaten path. I managed to find about 20 PC Cards at these shops, and spent only 2500 yen for them.

First, the cards that worked. The CyQ've ELA-010 and NEC PC-9801N-J12 both worked with the ed driver without changes to the drivers. The TDK LAC-CD02x worked with the fe driver, again without changes. In a way, these were the disappointing cards in the lot. While I didn't have examples of them in my collection, I also didn't expand FreeBSD's support for PC Cards at all.

I was able to modify the fe driver to support two additional cards that I found. One was the Hitachi HT-4840-11. I had to update the fe driver to select an alternative CFE. What's a CFE? It is the Configuration Function Entry in the CIS. These nodes in the CIS describe the resources that the card will use, as well as control different ways the hardware interacts with the system. The PC Card infrastructure usually selects the proper CFE based on the available resources. However, there are cases like this where some CFEs don't work for whatever reason and alternatives need to be selected. That's what happened here. The HT-4840-11 CFE entries less than 10 appear to be designed to be used in a PC-9801 and PC-9821 laptop. They don't work in my modern HP laptop at all. Entries 10 and above all work fine.

The second fe card I was able to make work was the CONTEC CNETPC. Its first entry had a similar problem to the first few of the Hitachi HT-4840-11, but the rest were good. A quick modification to my earlier modification for the Hitachi and I was supporting both. Well, with one caveat. I can't get this card to get any link with any of the hubs that I have. I don't know if this is because the dongles got mixed up over time and I didn't have a good dongle, or if other parts of the card are 'blown out' or otherwise messed up. I had several cards like this.

Next up was a Toshiba JT Ethernet card. It worked out of the box with the ed driver. However, there was an issue. The MAC address showed up as 00:20:16:ff:ff:ff which seemed odd. Especially since 002016016b31 was labelled on the card. Digging into this deeper showed that the card has the bogus-looking address in the CIS. None of the other standard ways to get a MAC produced anything other than 0's. There's no aux entries in the CIS that could be mined for this data either. Since that's the case, I'm almost convinced that this is working the best it can given the circumstances. Also, there was no good dongle for this card...

There were a number of other boring cards. These were cards that I was able to add to the drivers more easily. The NextComK.K NC5310B Ver1.0 seems to work with the fe driver, but I don't know if the driver I have works or not. Also the MACNICA ME1-JEIDA worked with the ed driver, but the dongle didn't seem to be the right one.

Now, on to the odd-balls.

The NEC PC-9801N-J02R is a card made especially for PC-9801 and PC-9821 laptops. I've never been able to make this work with the smc driver that's in the tree. The card has a sonic chip in it, and it should just work. However, it is extremely limited to where it can be mapped in the I/O space (only at 0x880!), and none of the laptops I've ever had worked with this address.

The TDK LAC-CD011 I found turns out to be rare. At one time they were expensive, but no longer. Sadly, I didn't get dongles for this card. There's sources on the web that document it not working with the fe driver, but that it may have the MB86960A inside, which we might be able to make work. Plus, the NetBSD mbe driver supports the CD010, so I may fiddle with the fe driver to see if I can make this work...

The Toshiba LANCT00A doesn't work out of the box at all. The only information about this card on the net today is that once there was a web page that Toshiba said it was a custom-built card. Chances are good that there will never be a driver for this card.

I'll take a look at the other cards another time. They are even weirder, I fear...

AsiaBSDCon 2009

2009-03-19T11:40:00.003-06:00

I just presented a paper at AsiaBSDCon 2009 on the FreeBSD/mips status. There's a lot going on with the FreeBSD/mips project these days. You can find out most of the details from my paper, which is available in letter or a4 format.

In summary, there's a number of new ports underway. There's Cavium Octeon, Atheros 7xxx/9xxxx, RMI Au1xxxx and a couple others.

AsiaBSDCon is always fun. There's many Japanese developers that don't get a chance to go to other conferences who I only get to see in Japan. This conference allows me to see them, as well as talk to them to see what they are using BSD in general, and FreeBSD specifically. The Japanese have often lead the way in many areas, and seeing their current projects is very interesting.

Also, I could cut and paste large parts of my paper here, but I'm not sure of the value of doing that. Can people that have strong opinions leave feedback on this issue.

Melman Lives!

2009-01-29T19:03:00.003-07:00

My son got a toy Melman at McDonalds in a happy meal a while ago. He love the thing. He pounds it on EVERYTHING. When you pound it, Melman says one of three phrases. Recently, however, Melman's voice started to give out. So I thought I'd replace the batteries. Should be easy, right, since those toys are just held together with a couple of screws.

Turns out it wasn't quite so easy. The screws are triangular. This means that a regular screw driver wouldn't work. I tried forcing it with some precision screw drivers I had, but they were all just the wrong size. A few google searches later and I found a few places that have them. The cheapest one sells bits for about $3.00. Currently, there's a catalog link here, but who knows how long it will last. I also found amermeida who sells through amazon that sells a screw driver for the triangle recessed screws for $2.00, but with $5.00 for shipping. I wound up ordering there.

Once the screw driver arrived, I removed the screws. There were L738H cells in place that were drained. I didn't have any of these cells on hand, and it was a cold night, so I raided my watch battery drawer in my shop. I had Energizer 392's. These were about 1-2mm thinner than the L738, but were close enough. I was able to bend the battery holder a bit and they slid right in. I snapped Melman back together, and I had a happy 2-year-old again.

Maybe I'll be able to repair other toys with this screw driver. I hope so, since I don't think the original toy itself cost $7.00. There's a couple of cars that he has which are held together with the same type of screw.

More on FreeBSD/mips another time...

Alchemy Au1550 status update

2009-01-27T00:04:00.002-07:00

I'm in the very early stages of bringing FreeBSD/mips on the Alchemy Au1550. Plat'Home was kind enough to put an OpenMicroServer into my hands for the port. After months of sitting idle, I've started putting the board to use again. I test booted my first kernel tonight. It is still early days, but I thought I'd mention that I'd committed the very preliminary code to the project/mips tree in svn.

As an aside, the FreeBSD/mips main development has moved from the so-called "mips2" tree in perforce to the projects/mips tree in svn. This move allows us to more easily share the work in progress mips ports that people have been asking for in the past. We'll see how well this works out and see if theory matches reality or not.

Quick Mac OS X hack: Time Machine

2009-01-23T11:59:00.004-07:00

I have two macs in my house. One has a 750GB drive on it for backups, plus its normal disk. It had been setup for Time Machine before and was working well. I also have a MacBook Pro that needs to be backed up. I didn't want to swap the disk back and forth between the two, since that would significantly erode the value of Time Machine's automatic backups. I didn't see any place where this was well documented, so I thought a quick note would be helpful to people. Most of the references I've seen have talked about different hacks to make this happen, or that it is only on a specialized Apple product that I didn't have.

I'm running Mac OS X 10.5.6 on both machines.

First, I went to the store and bought a 750GB drive. I plugged it into the mac mini that I have sitting on my desk. I configured it for Time Machine on the Mac Mini. I did a couple of time machine backups.

Next, I exported 750GB disk using the 'Sharing.' I clicked on 'File Sharing' tab to enable it. I clicked on the '+' button under the "Shared Folders" menus. I selected the volume that I had setup for Time Machine. I made sure that I was sharing via "AFP" in the "Options..." dialog.

Once I had things shared this way, I went to the other mac. I went to "Finder" and selected "Go->Connet to Server.." tab. Here I explicitly typed "afp://ipaddr/" and connected. Once I was connected this way, I proceeded to enable Time Machine. The remote disk appeared on the list of choices, so I just selected it and started it up.

I discovered that I have a 100Mbps switch, rather than a GigE switch. This made the initial backup very slow, on the order of 10GB/hr, which means I'm looking at a 10 hour backup time. Maybe it makes sense to head off to staples/best buy to get a GigE switch to plug them both into.

I've not tried to do this wirelessly, so I don't know how painful that will be. I've also not tried to connect this to the Samba server that I have running FreeBSD. I can normally see that volume on the macs without needing to do the manual step.

UPDATE: A GigE switch does help quite a bit. It was more like 6GB/hr with the 100Mbps switch, but with GigE I'm getting about 22GB/hr... So I have about 3 hours left in the backup now... 22GB/hr is about 6MB/s which seems way too slow (the hard disks are limited to about 15MB/s on each side, and the network to 125MB/s, so why the fall off in speed).

Minor at91 hacking

2009-01-22T20:25:00.003-07:00

The recent changes to the SD/MMC stack broke AT91. It seems like there was an assumption that was bogusly baked into the driver. We had assumed that the only data transfer commands would be 512 bytes.

The problem is that the newer stack does commands that are 16 and 64 bytes long. These aren't really needed for basic support of the stack, but to get things like SDHC and high speed transfers working, they are needed.

I've fixed this, and now things work again. I've also taken this opportunity to do some minor cleanups to the at91 mmc/sd driver.

Audio Ripping on FreeBSD

2008-12-16T08:42:00.002-07:00

For years I've been batching up my new CDs to rip all at the same time. Sometimes, this means doing a couple a month. Other times, like this year, it means catching up with 12 months of backlog. And while my ripping habits likely are uninteresting to readers, what I've learned over the years may be helpful.

Years ago I installed the 'abcde' port, and update it from time to time. Having it in-place, and configured, makes all the difference in the world. I use cdparanoia as my reading method since it is the slow but reliable choice.

There are many silly process things that make things go faster. First, make sure that you have all the CDs in the cases that you have stacked up. A few days before I start to rip, I go through the stack to make sure that none are on walk-about. Next, grab a bottle of windex, and a lint-free cloth. Wash each of the CDs, assembly line style. Sometimes I only do this for the discs that fail, but often times it helps things go so much faster that it can be worth the up front time.

Once I have the discs prepped, I pick a time that I'm not using my laptop too much to do the ripping. For me, this is often my work day, since I look at it from time to time for email and irc messages only. My laptop DVD player isn't the best in the world, but that's OK. The ones that rip quickly, I don't have to do anything about. The ones that are scratched badly, I put aside for later. The easy ones are boring, since abcde makes them boring, so I'll not describe the process much further. Each disc takes about 10 minutes to rip and transcode them.

The scratched discs are more of a challange. I've used a lot of players over the years. They vary widely in their quality. The cheapo player I have in my laptop copes with most of the discs. If I see too many + signs in a disc, I'll set it aside for later. Here, too many is 'more + than spaces in the output' when I look over at the screen several times. Or if there's any 'V' indications on any of the tracks. These usually take longer to do anyway, and get in the way of breezing through the easy CDs. Each year, I have maybe 5 discs that are stubborn, out of maybe 30 that I do. Long term averages are that 1 in 10 newly purchased 'used' disc show this. Discs that have been knocking around in the car for months are more likely to suffer from this as well. Really stubborn ones are more like 1 in 100, but more on that in a bit.

I have a special CD-ROM (not DVD) that I use for the scratched discs. Of all the drives I've tested/used over the years, this one has the best success rate by far of any of them. This one happens to be a 52x/20x/20x IDE drive branded by HP I got at Target, of all places, several years ago. I can take a disc that has 20-30 V's on it with my cheapo laptop setup, and sometimes get a good read with this player. Often, the discs just read quickly, and that's the end of it.

Sometimes, however, the disc is badly damaged and is that it is very slow. I was able to rescue a Moxie Froovis CD and a Phil Collin's greatest hits CD with this drive. But it took a week each of reading. When the drive is connected to a server, it doesn't matter so much that it is doing this. The CPU time isn't all that great during the week, so it doesn't interfere with the normal server operations. I look in on them once an hour at first, then once a day until the songs are done. Since I rarely have more than one or two of these a year, the extra time isn't that big a deal.

At one point, I had a dozen laptops with CD Roms that I could use for ripping. I thought this would be great, since I could do so many in parallel. It turns out that the CD Rom drives in these were even cheaper than the one I'm using today. I spent so much time running between the laptops that I didn't get any work done. And the CPUs in these boxes were all, at best, squirrel powered, so they couldn't handle the encoding as well as my 2GHz HP Pavilion laptop does now. At the time, I just did it serially on the server I had (which was a 666MHz Intel), and changed it every hour or two. I could still run it remotely, so I wasn't running back to the server room except to change a disc.

Today, with iTunes and other music places online, the art of coaxing bits off of audio CDs seems a bit passe. However, it is still something I find I'm doing a lot. Used CDs cost almost nothing, compared to $1/song for iTunes. Plus, format shifting is so much easier with audio CDs than with iTunes. Also note: I'm not talking about sharing my collection over the internet, just making the archival copies, and format shifted copies, as allowed by law and court decision. I still have most of the original CDs, and those I no longer have had been destroyed by my two-year-old when he was younger... But that's a story for a different time...

DLink DIR-615 Rev C1

2008-12-14T02:00:00.000-07:00

Today I saw that Office Depot had a DIR-615 on sale for $40.00. It was a new hardware revision: C1. I took a chance that it was a new non-Ubimax based router. I'm glad I did, at least so far.

The router is based around the Atheros AR9130 CPU. This is MIPS based, so there's a good chance that I'll be able to put FreeBSD onto it. We'll see how it goes.

I'll report more when I get a chance to examine the RAM and Flash parts, as well as map out the serial port to see what is reported there. Stay tuned.

More CardBus fixes

2008-12-13T16:39:00.002-07:00

After I committed my previous set of CardBus fixes, reports came in about interrupt storms, first with 16-bit cards, and later with 32-bit cards. These have been corrected in my latest fixes. We always act the CSTS interrupt when we see it. In addition, I've changed the acking of the 16-bit ExCA register to only happen when there's a "R2" or 16-bit card in the slot. Otherwise we now skip it. Hopefilly, this will help the shared interrupt case as well by eliminating some PCI bus cycles...

The CSTS bit is interesting. It is the least well documented bit in the CardBus standard. It is unclear when it fires from reading the standard, but seems to be related to the card finishing its reset sequence. How this differs from the power-up interrupt, I'm not sure. I think we could further optimize the bring up of CardBus cards with it.

In addition, I noticed there's a READY bit in the ExCA CSC register. We're currently doing busy waits to bring up the cards, including a couple of millisecond long DELAYs. I'll have to look into the prospect of using that interrupt to get around these issues there.

Finally, I'm looking in earnest at the Alchemy Au1550 CPU for the OpenMicroServer port I'm doing. I've noticed that it uses a 16-bit PC Card interface. Maybe I'll finally need to rewrite the old PCIC driver from OLDCARD to make use of it (snagging the exca routines to make that easier). The OpenMicroServer has a CF card attached to it.

Cardbus Fixes

2008-12-04T23:47:00.002-07:00

I just checked into the tree some CardBus fixes. The biggest change was to the power-up sequence, as well as transitioning to using filters for the card change events. These two changes are somewhat intertwined, unfortunately, since the latter exposed some holes in the former. In a nutshell, we now register a filter for the card status change events. This means we can mark the card as bad right away before any additional interaction can happen to the card. We defer doing anything about the badness for a little bit (basically until the machine is idle enough for the cardbus kernel thread to run). The interlock mechanism is also much ligher weight, having moved from mutexes and CVs to simple msleeps. Fast interrupts require some care to get right, so I hope I've gotten it all right. The move is dictated by what you can and cannot do in a fast interrupt handler.

Before making these changes, my atheros card would often reset for no reason. After these changes, I've not had it reset until I started using a kernel without the changes. I'm not sure why this would make such a big difference, and I hate mysteries. They indicate that there's something I don't understand, which I also hate. I'd be interested to see if I still see this when the fully integrated code is committed and looped back (I have several trees, and I most frequently run an unmodified kernel from svn, but sometimes also run one of these trees). There's a small chance that one of my other local changes could be the cause, but given what they are it seems doubtful. Still, a good datapoint if it is.

In addition, these fixes add a retry option for the BadVcc errors that we'd see sometimes. These are annoying on some machines, and often times the best way to get around them was to reload the driver. They don't happen at all on TI based chipsets that I've seen (at least more recent ones). Instead, they were confined to the Ricoh chipsets. Since I switched my laptop a couple of years ago, I haven't seen them. These changes post-date the change, but have been tested lightly on another Ricoh laptop that I have. They've been heavily tested on the TI laptop. They are based only on the description of the problem in the NetBSD PR, since the implementations are so different for the different BSDs in this area.

Finally, I did some comment tweaking and shuffling. I also managed a style change or two. These should have absolutely no effect on the running code, but hopefully help the reader of the code understand it better. Past experience has shown that these types of commits are most likely to provoke comment, even though the other two parts of my commits are actually quite a bit more important.

Oh, why the flurry of commits? I was low on disk space, so I thought I'd go clean up. 11MB free just isn't enough. So I was looking around and discovered I still had a CVS tree and was going to blow it away. I did a final update just to see what changes I had. I'm glad I did, since I found these, and a few others. I still have more local changes there than I realized. Sometimes I guess it is good to run out of disk space and go on a cleaning spree? It also makes for smaller backups when you don't have top copy 5GB of /usr/obj and kernel build trees...

Moving towards board files

2008-12-04T14:56:00.003-07:00

I've spent a little bit of time implementing the start of board files for the arm port. The initial push has been for the at91 subport only, and many improvements could be made to this. I've written up my initial thoughts on this on the FreeBSD wiki FreeBSD Arm Boards. It could use much improvement, I'm sure.

One idea that hasn't been reflected there yet, was shown to me by Sam Laffler who suggested using linker sets to allow boards to 'probe', 'init' and other standardized functions. This is an interesting idea and I plan on working on adding it to the above links when Sam has results to share.

I'd also like to expand the above wiki page to be a 'best practices' guide for all architectures where there's great diversity of boards/cpus/etc (eg, not the homogeneous env that x86 provides).

I'm also soliciting comments on the above boards in addition to the above. Send them to me, or post them here.

Cassio Fiva's screen dead

2008-12-02T12:40:00.002-07:00

I was very fortunate enough to get a Casio Fiva MCP-205 given to me in 2000. I've used this laptop for years. I built Mozilla on it once, and melted the frame around the LCD! This resulted in a new warning in the Mozilla build instructions. Since Casio is a Japanese company, and I got this as the result of my contacts in Japan at the time, my news of melting the Fiva spread like wild fire there. Recently, when I was in Tokyo for AsiaBSDCon people were asking me about it.

I've been using this laptop to do background builds, as a mobile terminal, firewire debugging platform, etc since I got it. Recently, i had put the laptop on the shelf. I'd not used it in the past 6 months or so for a variety of reasons. Last night I thought I'd give it a spin again, since I need a nice mobile platform. Sadly, while it was in storage the screen was cracked. I cannot get any text on the screen at all. I don't have the VGA cable so I could keep this laptop around as a server, since there's no console access. I cannot find an LCD to replace it, but I imagine that it would be expensive. More expensive than a modern netbook, which would be better for the FreeBSD project anyway...

It is sad to see this laptop give up the ghost. Every time I used it, I was reminded of the good times in Japan. I'll hang onto the laptop for a little while on the off chance I can turn up a replacement LCD, but I fear it is dead.

MeetBSD

2008-11-14T22:12:00.003-07:00

I'm in California this weekend for MeetBSD, celebrating FreeBSD's 15th birthday!

Come on out and join the fun.

Marvell ARM support in svn

2008-10-13T13:17:00.006-06:00

Rafal Jaworowski just committed support for a number of Marvell Processors. There are a number of commits going into the tree. He's just completed the first of these:
Introduce low-level support for new Marvell core CPUs: 88FR131, 88FR571.

From an earlier email message, we know that Rafal is working on support for Marvell 88F5182, 88F5281, 88F6281, and MV78100 SoCs.

The following peripherals will likely be supported:

EHCI USB 2.0
Ethernet
GPIO
Interrupt controller
L1, L2 cache
Timers, watchdog, RTC
TWSI (I2C)
UART

This is a fairly mature port that is going into the tree. It works with NFS or USB mounted root filesystems. The port is self hosted. Kernel and world builds succeed on the box. The box also boots to multiuser mode. Rafal has created a web page with all this information on it on the FreeBSD wiki at http://www.wiki.freebsd.org/FreeBSDMarvell

This is way cool news! Good Job Rafal.

Note: Much of this information was taken from a posting Rafal made to arm@freebsd.org.

gxemul update

2008-10-04T01:02:00.004-06:00

Oleksandr Tymoshenko posted a patch to gxemul that allows FreeBSD/mips to boot on it.

I've committed it to the gxemul port. So now if you have gxemul 0.4.6.5_1 or later, you can run FreeBSD/mips, the MALTA kernel. I'll post a howto and a pointer to a image in a few days.

SD/MMC work

2008-09-30T19:50:00.003-06:00

Recently, I've pushed many bug fixes from Alexander Motin to the SD/MMC stack. He's submitted a driver for the SD Assocaition's standard SD Host Controller, and fixed many of these minor issues as part of doing that. I've tested these improvements on my AT91RM9200 system that boots off an SD card.

There should be more support for MMC cards, as well as SDHC cards forthcoming. In addition, once I get a chance to review the sdhc driver, it too may be ready to head into the tree. I know that if I had such a working driver, I'd be able to work more on SDHC and MMC card support. The current AT91RM9200 box I have makes it difficult...

Stay tuned for more updates.

FreeBSD/mips updated

2008-09-19T14:14:00.003-06:00

Here's an update in status to the FreeBSD/mips tree. David O'Brien has been working through issues one at a time to get things building, primarily getting gcc support merged in. After learning from David that more patches were needed than were in my blog, I spent some time collapsing changes from p4 into the main tree. I've updated my patches, and confirmed they work.

First, you can find the patches from my FreeBSD drop box. These patches are 42k right now. This is from my tree that I pulled at revision 183198, taken at 3:30pm MDT September 19th, 2008, for the nit-pickers out there. At this level, I'm able to apply the patch, do a buildworld and all the kernels except SENTRY5 build. To build things:

% svn update
% fetch http://people.freebsd.org/~imp/MipsSvn.diff
% patch -p0 < MipsSvn.diff
% setenv TARGET mips
% setenv MAKEOBJDIRPREFIX /tmp/$LOGNAME/obj
% make buildworld
% make buildkernel KERNCONF={ADM5120,IDT,MALTA,QEMU}

I've just done the build testing at this point. Stay tuned in the coming days for how to setup a qemu or gxemul image and how to run this in emulated. I hope to get instructions for how to deal with real hardware a little bit after that. Right now the ADM5120 kernels are a little too big for the hardware that I have and it will take some time to sort out.

Hopefully we'll have these last 42k of patches into the tree in the coming weeks. I'll keep everybody posted. There are some additional fixes in p4 as well on top of this which I'm still sorting out, but those should be going in as well.

FreeBSD embedded (mips, powerpc, arm) update

2008-09-03T23:21:00.007-06:00

The last week or so has show a spike up in FreeBSD embedded platform work. Fixes for mips and powerpc have gone into the tree. Support for a new ARM platform is getting ready to be committed. FreeBSD/mips is almost self-hosting in the svn tree (it has been in the p4 tree for some time). FreeBSD/powerpc and FreeBSD/arm continue to mature.

Thanks to a flurry of activity over the weekend by David O'Brien, the number of patches needed for FreeBSD/mips has been reduced to a set for libpam and for binutils. David cleaned up the diffs for gcc and pushed them into the tree in a manner that allows for easier upstream committing.

In addition, David has cleaned up a number of dangling nits in the build. This is the result of a slight mismatch between p4 and the code in FreeBSD's svn tree. Oleksandr Tymoshenko has looped back most of these changes into p4, and is working on resolving the conflicts that were generated. Once this is sorted out, Oleksandr or myself will generate a new set of diffs and go for the final push into the tree.

I'm looking forward to having a good base for FreeBSD/mips in the tree. I've been busy with a number of other projects and haven't given it the time that I really want to give.

On the PowerPC side of the ledger, Marcell Moolenaar and Rafal Jaworowski have been polishing uboot boot loader support in the FreeBSD/powerpc subversion tree. In addition, some bug fixes to and code polishing of the low level Book E PowerPC support have been committed. Many of these improvements were obtained from Semihalf consulting.

On deck to be committed soon is support for Marvell's Orion NAS chips. This code has been in perforce for a while now and is nearly ready for the FreeBSD tree. I'll keep everybody posted when that happens. I hope to have my DLINK DIR-615 Rev A1 working by then with this code, but time has been tight and there's already too many different projects afoot in my life.

Sandbox pictures

2008-08-29T23:49:00.002-06:00

I've had several requests for pictures of my son playing in the sandbox. I'll go one better and show pictures of my son and I playing in the sandbox. My wife took them just after I filled it with sand.

Now, all I have to do is make a lid for the box to keep the local cats and racoons out.

Sandbox

2008-08-28T00:30:00.004-06:00

I debated writing about this. After all, it isn't about hacking code. Instead, it is about hacking together a sandbox. It is about making stupid design choices and then over engineering "solutions" to those problems. It is a cautionary tale about doing the design right from the start and sometimes "free" lumber costs a lot in time and effort.

So, I wanted to make a 4'x4' (just a little over 1m x 1m) sandbox for my 2 year old. He loves sand in the parks and I wanted him to have a place to play in the back yard. My wife suggested we just get one of those turtles, but I wouldn't hear of it. After all, I had a bunch of 2x4's in the garage. Why pay good money for a plastic turtle when I could just build a simple box out of the 2x4's.

I wanted a 4" depth of sand, so I figured I'd have to stack the 2x4's. Rather than just buy a couple of 2x8's and bolt them together, I had to use 2x4's. Of course, I had planned on stacking them, but what I hadn't planned well was how to do it. I wound up with small 2x2's as wooden cleats. So I get the box put together. It was fairly straight forward to do this. I connected the two halves of the box with short 2x2's. I proudly showed it to my wife who immediately said "those sharp edges sticking out look dangerous." She was talking about the 2x2's. Once I saw it from her point of view, I realized she was right.

Now, rather than spend $8.00 on 2 2x6's or 2x8's for the box, I spent $9.86 on 12 6" ceramic coated screws to hold the box together. I then drilled holes and recessed the heads (why? I don't know, just seemed like a good idea even though they were on the bottom). I drove the screws in and removed the 2x2 cleats. The box looked great! And I even managed to add the trip onto another trip to a near by store, so I didn't blow a ton of time and gas getting them. That was luck, pure and simple, since usually for things like this I'm making another trip...

I then went to put it where we'd agreed and realized I had no bottom. I had to do something about a bottom. I thought I'd just use some of the yard fabric I had left over from all my landscaping chores. Turns out that I left all that at my old house. So instead I took and old tarp with a tear in it and cut a 4'x4' section and laid it on the ground and put the sandbox on top of it. I then filled it with the play sand I'd been lugging around in my car for a week while I finished the box. Time will tell if I secured things in place well enough or not. I was planning on going to the store to get a bunch of rebar to drive into the ground to keep the box from moving, but I realized just how crazy this was getting and I just made a couple of wooden wedges to hold the thing in place. We'll see how that works out, but they are likely good enough, and were scrap anyway...

In the end, I spent 4 hours building this box. While a turtle would have saved me a bunch of time, and would likely have cost about the same as what I spent on wood, nails and screws (all but the last one had already been purchased, but I did pay for them). I'd have been done in 30 minutes. Of course, I wouldn't have gotten to use the power tools, and I really like the idea of building things for my son. I put the sand in tomorrow and we'll see how well he likes the box.

So, am I crazy? Did I go overboard on trying to save a nickle only to cost myself a ton of time? Maybe. Did I have fun putting it together? You bet! Was it worth it? Yes...

Low hanging fruit

2008-08-27T00:47:00.002-06:00

Tonight was low hanging fruit night. I went through all the PRs that had "pccard" or "cardbus" and committed the easiest of the fixes. When I did this with USB last year there were maybe 50 of them I committed over the course of a few weeks. For PC Card and CardBus, there were maybe 5 or 6 that I was able to commit. And a similar number of stale ones. Unlike USB, there's only a few PC Card and CardBus bugs in the PR database, at least that are flagged as such.

I find this surprising, since any CardBus card that uses I/O ports hasn't worked in a while. And CardBus card whose CIS is stored in "Expansion ROM" won't work. There's a bunch of problems with some of the newer "ed" cards as well for PHY handling. At least I've recently solved some of the power issues, or I'd be feeling pretty low about the state of affairs. Fortunately, most people want a CardBus slot for a wireless card these days, and atheros chipsets are memory mapped.

The ExpressCard stuff is stalled at the moment while I ponder a difficult problem with resource allocation. There's likely a simple solution to this problem that I'm not seeing, but it likely requires thinking a little outside the box to make it totally effective.

Summer tree cleaning

2008-08-26T01:02:00.002-06:00

As you may know, I'm a user of the p4 side repository to do large code development. With the switch to svn, I wasn't sure I'd still need to do that, so I let my trees there go for a while. It is clear to me that I'll need to use p4 a little while longer. So, I cleaned up my newcard tree.

Why do I need to cleanup at all? That's a good question. You'd think that you'd only have code cooking in the tree that was destined to be committed, so no cleanup is needed. Merging conflicts, sure, but no cleanup. Well, it turns out that sometimes perforce doesn't do quite the right thing when there's a loop back of new files into FreeBSD. If I create a file in perforce, and then later merge it into FreeBSD, then the $FreeBSD$ tag is messed up. Over time, you get lots of these files in the perforce tree, and you have to basically just copy the FreeBSD version over them.

The other reason to clean up is that code can cook for too long. I have several projects in my newcard tree that just didn't work out. I didn't get them into the tree quickly enough and the problems they were designed to clean up have been solved in other ways. Or they fixed problems in the MD code, and now that FreeBSD has grown more machines more code needs to be changed. Sometimes you change the new MD code. Other times, you just junk the old stuff since the improvements were so marginal that it isn't worth carrying those stones in your kit bag anymore.

Sometimes projects are only half-baked. I had a number of interesting ideas that turned out to not be such a good idea. That turned out to not be a big enough savings, or that some other key developer in the project didn't like. It happens. Some ideas are great, while others seem to be grating on others.

So all of these are reasons to cleanup the tree. I've managed to reduce my diffs from 200k down to 100k. Most of the 100k that's left could easily be described as half-baked. The biggest half-baked thing is a port of the NetBSD improvements to mii. The second biggest are a collection of hacks and bug fixes to the pccard/cardbus layer. Those always take a lot of time to sort out.

It is one of the more pedestrian things that I need to do when hacking. But if the litter piles up too high, it becomes increasingly difficult to see what should go into the tree, and what shouldn't so nothing does and many of my cool hacks don't benefit people, and I don't benefit from the wider exposure of the code. Nobody tests my private tree but me... Everybody notices when you put something even a little stinky into the main FreeBSD tree.

I debated about not blogging about it. However, one reason people have told me they read my blog is that they learn how to be better engineers from some of the things I do. They usually don't tell me if they are emulating things I do because they think they are good, or if they are avoiding the things I do because they think they are bad. Maybe a little of both... Documenting the mundane from time to time may bring better balance. Or it may put my readers to sleep... It is hard to say without feedback...

T-Mobile Dash experiment

2008-08-25T01:42:00.008-06:00

I have two cell phones. One of this is my iPhone, made by Apple and running Apple's version of FreeBSD. The other is the T-Mobile Dash. I usually use my iPhone, but this weekend I was using the Dash to see how well it worked. It was no better or worse than the iPhone in the one area that I'd hoped it would be better at: call quality.

In the process of trying it out, I discovered the uipaq driver in FreeBSD. I wanted to use the Dash as a tethered phone. It is the whole reason I got it. Since the iPhone doesn't officially support tethering, I thought I'd give the Dash a try.

The uipaq driver was added to FreeBSD 7.0, ported from NetBSD which ported it from OpenBSD. Its roots go way back to the old Compaq iPaqs that were made years ago. This same basic interface has gone through only minor evolution in the interceding years. Unfortunately, I wasn't able to make it work with my phone due to some minor issues else where.

In the process of trying, I discovered there was no man page, so I snagged one from NetBSD and hacked it to account for the differences with FreeBSD. In searching how to setup the phone as a ppp link, I stumbled into a PR that greatly expanded the number of devices the driver recognizes and made it committable, I discovered that it wasn't built as a module, and I discovered that an old PDA I had in the garage would work with this driver, in addition to my HCT dash. All the easy problems I fixed...

So I moved the rock up the hill a little. I'll try another day to see if I can track down the issues that were else where in the kernel and see if I can punch entirely through to the other side. The lack of a tether solution has been the biggest disappointment with the iphone that I've had. While a number of interesting hackers are possible in that area, none of them are as easy and as simple as my old Sprint Samsung SPC-7600 was. The Dash also has bluetooth, which I'll give a spin after I can get it to connect with the wires...

None of this would be necessary if Apple didn't have such a tight control on what one can and cannot put on the phone. Well, control isn't the real issue. Rather, the issue is that they choose to control it in a way I don't like. If they controlled it in every way the same way I wanted to configure it, I doubt very many people would care... Since they don't, a lot of people are very interested in hacking it... but that's starting to get off topic...

Quick device_get_children cleanup

2008-08-23T12:28:00.002-06:00

Hans Petter Selasky submitted a bug against the rather poor error handling of device_get_children a while ago. When the usb4bsd stuff he's trying to get into the tree was posted for review, one of the issues was with device_get_children() as it related to device_delete_children().

This reminded me of original issue. I've gone through the tree and done an audit of the device_get_children calls. I think that the issues have been resolved. However, it looks like there's a lot of code in the tree looking for siblings and such. This suggests a close review of the code to pull out common routines into subr_bus might be in order. Each of these hand-rolled routines had slightly different bugs in the error handling...

Mid august mips status update

2008-08-23T01:00:00.006-06:00

Just wanted to jot out a quick status update on FreeBSD/mips.

A few weeks ago, diffs were sent out for review. The 6 patches are now down to three. gcc.diff, binutils.diff and pam.diff remain uncommitted.

pam.diff (now pam3.diff) has been receiving feedback from DES and others and is getting close. The first iteration was really far away from being commit quaility, but the last iteration may be the one that goes in.

gcc.diff has received feedback recently. Gonzo and I are in the process of digesting and understanding the feedback. We hope to produce a revised patch shortly, as there are a number of minor issues.

binutils.diff has had no comments. One person volunteered to look at it and commit it, but so far we're still waiting for feedback to make the patch better, or for it to be committed. If we don't hear anything one way or the other, I'll don my flame resistant suit and push it into the tree. There's really no issues with the patch that I can see. What is going to be a challenge is doing the vendor import of a few upstream files. Hopefully the volunteer will pull through and I won't have to learn how to do this in the new svn world. :-)

All the other diffs have been committed to the tree. It was gratifying to get positive feedback on them all, since I was worried more of them would have maintainer timeouts.

You can see the current set of patches in gonzo's patch directory. Grab only the latest pam patch there. The cddl.diff patch should be removed shortly, as I just committed it.

Had I been paying closer attention, I'd have written all this up for the FreeBSD status report.

Slimming down the NSLU kernel

2008-08-19T22:49:00.003-06:00

A few days ago, I wrote about the NSLU kernel config file being committed to FreeBSD's svn tree. The default compressed kernel is about 1.6MB, but the size of the partition of the NSLU2's flash is 1.25MB, a gap of just under 300kB. With some hacking, I've been able to reduce the size of the kernel to fit.

The default kernel's text+data size is 3,425,844 bytes. Compressed, with the appropriate headers, this drops to 1,591,109 bytes. However, the goal is 1,310,720 bytes. We're short by at least 294,553 bytes of fitting into the kernel portion of the flash on the NSLU.

My first thought was to start cutting things out. I removed IPv6 (options INET6) since I didn't need it for my setup. This saved 210,797 bytes in the uncompressed kernel and 74,115 bytes in the compressed kernel. This one option shaved 5% (compressed) of the kernel size. A move in the right direction, but more was needed.

Next, I recalled from previous attempts that there's many inline functions that don't need to be inlined. There's a small performance gain from inlining these functions, but it can add a lot to the kernel. So I added three options to the kernel config:

options MUTEX_NOINLINE
options RWLOCK_NOINLINE
options SX_NOINLINE

This resulted in a size reduction of 432,984 bytes (uncompressed) and 87,289 bytes (compressed). This is also a significant savings, and meant that we'd saved 161,404 bytes in the compressed kernel so far. We were still ~133,000 bytes shy of the mark.

At this point I noticed that there were a lot of things in the kernel that weren't required for boot, and that had shown up as being larger. When doing the size reduction, often times I use the 'size' command to get the top 20 .o files to see if there's junk that can just be omitted from the kernel. In this case, I saw that usb, scsi and releated drivers consumed a lot of space. This could be loaded from a number of modules after boot if I was booting from a RAM disk. Since the plan was to put an initrd-like ram disk in the other partition of the flash, I went ahead and removed these pieces:

#device usb
#options USB_DEBUG
#device ohci
#device ehci
#device ugen
#device umass
#device scbus # SCSI bus (required for SCSI)
#device da # Direct Access (disks)

This trimming saved 278,837 bytes (uncompressed) and 133,520 bytes (compressed). This was huge, and brought the total trimmed up to 294,924 bytes. This is just barely over the line, so I thought I'd go for a little more.

Another big area of the kernel is the code brought in with 'options miibus' configuration line. This brings in all the phy drivers on the off chance you'll need it. In many embedded systems, you know which phy driver you have, and can wire in only the one or ones you need. Since I wasn't planning on using this box as a general purpose router, that needed to work with any Ethernet dongle that was plugged into the USB port, I went ahead and transitioned to this configuration. The NSLU has a RealTek Phy that needs the rlphy driver. This means that I changed the config file like so:

#device miibus # NB: required by npe
device mii
device rlphy

This option shaved a few more bytes off the total. 80,012 bytes (uncompressed) and 25,156 bytes (compressed).

At the end of all this, the compressed kernel had been reduced from 1,625,125 bytes down to 1,286,193 bytes, for a savings of 320,080 bytes. This was enough margin for me to feel comfortable trying to burn it into my NSLU. Unfortunately, I ran out of time tonight to try it, so the story of that experiment will wait for another post. Also, I don't know how to make blogspot do tables, or I'd have made some nice tables with all this data for this log.

There's likely more options that can be explored to squeeze even more bytes out of the kernel. bzip2 might be a good place to get maybe 10% more space back (at these sizes, an extra 100kB compressed, or 250kB uncompressed). The exploder for bzip2 is bigger and uses more memory.

Looking at the .o files that are laying around that pci.o is kinda big, suggesting that we need to break it up some more. In previous experiments, separating out the MSI and PCIe stuff helped quite a bit. I'm unsure what the best newbusly way to extract this functionality might be, but it could save us 20kB uncompressed (maybe 8k compressed, iirc).

There's a number of other places code is agresively inlined, these could be investigated. there's
a few things in the tree tagged as standard that likely could be made to be options. There's much room here to explore a number of different options to reduce the kernel size even further if the need presents itself.

The nfs server option is about 45kB. Soft updates run about the same. nfsv4 also eats a lot of memory (looks to be 55kB).

However, for all these options, it would be nice is the NAS could have RAID, FAT, ntfs, smbfs and ext2fs support as well. Continuing to trim would allow this more easily. Or at least eliminate the need for loadable modules.

This is to say, there's a lot of areas for further research and documentation. I didn't even look at using a /boot/loader either. Maybe that can be helpful... Maybe I'll turn that into a research paper for one of the conferences... Either the continued size reduction thing, or using /boot/loader.

Finally, I've uploaded the changed NSLU config file in case my verbal descriptions here were insufficient.

PCI Bridge hacks

2008-08-16T17:45:00.004-06:00

Recently, someone sent me a new ExpressCard. This isn't the first ExpressCard that I've had. However, this one is PCIe based. I plugged it in, and it didn't work. I had no idea where to begin tracking it down, so I've added some information to pcib device. Now, we report the all the bus numbers that are relevant. Turns out after a little more digging the memory allocation on the bridge is wrong. None was allocated by the BIOS and FreeBSD isn't picking up the slack. That's the next set of problems to solve.... After that, I'll look at the hotplug functionality...

FreeBSD/avr32 spotted in the wild

2008-08-09T21:52:00.004-06:00

A very preliminary port to the AVR32 has been spotted in the wild. Arnar Mar Sig posted a dmesg of FreeBSD/avr32. You can find his post through this link.

For those of you unfamiliar with the AVR32 architecture, it is an architecture that is used primarily in embedded applications. AVR may be initials familiar to many readers as microcontrollers. Atmel took its know-how and created a 32-bit version of this architecture basically from scratch.

You can find more information about the processor at atmel's web site, or at wikipedia.

The work here clearly is preliminary, but it is very cool none the less. FreeBSD embedded is on the move...

[[ typos corrected, I hope :-) ]]

The dog ate my homework.

2008-08-07T16:21:00.005-06:00

People have reacted favorably to a recent commit that I made where I said that my dog chewed up a card so I couldn't test it. I thought people would be interested in seeing this card.

First, the commit message:

Log:
Convert to new style PC Card front end driver. Add support for the
NEC PC-9801N-J02 and PC-9801N-J02R. I can't test the former because
it requires resources that conflict with my laptop. I can't test the
latter because my dog chewed up my -J02R card and it didn't survive
well enough for me to test.

You can see the whole commit here if you are into that as well.

Several people remarked this was the first time someone used "The dog ate my homework" excuse in the project. So I thought I'd produce pictures of the card for your enjoyment.

Here's a shot of the top. Notice how one of the teeth punctured the outer shell...

Here's a shot of the bottom. The damage doesn't show up well in the photo, but there's many teeth marks here.

Finally, the force of the chewing popped open the case. Here's a shot of the inside. Notice how the pins to the PCMCIA connector are severed. Otherwise, the card is in great shape...

Minor PC Card work

2008-08-06T01:36:00.002-06:00

I have a large collection of PC Cards. Well over 100, maybe over 200 cards. These cards are my test bed for the FreeBSD PC Card (both 16-bit and 32-bit) software that I maintain. Sometimes, I take cards out of the boxes at random at times and try them. Other times, I'll take all the cards that are affected by a certain driver change and test them. When I did that recently, I found a hole in the PC Card system.

This hole is very minor. There's no way for a driver to select the configuration entry that is used for the card. Usually, this doesn't matter one bit at all. Drivers work perfectly fine with the configuration that's automatically selected for cards. The PC Card spec has evolved over the years to the point where it is all automated. Certainly for 32-bit cards, but also for most 16-bit cards, even multifunction ones.

However, there's some older cards, some of them interesting to me, that need some special hand holding. These cards perform better with different entries, or activate both the ethernet and the modem on the card, instead of just the modem, when a different entry is selected. Trouble was, there was no way to do this in FreeBSD easily.

I just committed some changes to the PC Card system that will allow 16-bit PC Cards to select the configuration entry entry they will use. There are some restrictions. First, it can only be done in the probe routine for the driver. Second, the driver's probe routine must return 0. Big problems happen if the card is merely 'bidding' on a device and tries to set the right configuration. This may seem harsh, until you realize that if a probe routine knows enough about the card to try to set a configuration entry, then it isn't a generic driver anymore.

Also, as a nice side effect of this work, I've made things more robust for people that insert a card, only to discover that they didn't have the driver for it loaded. There used to be many sharp edges in this code that could cause resource leaks when just the right sequences of load/unload of drivers and removal and insertion of the cards happened. Now, it is all done in one routine, so things should be more robust. Please contact me if I somehow messed that up.

So this is one more ripple effect of the IF_NEEDSGIANT removal patches. I never would have looked at this stuff had I not been testing patches for John Baldwin. It is interesting the sorts of trails one finds oneself down when trying to help others out, as well as test some hardware that would otherwise be laying around.

NSLU2 support committed to FreeBSD/arm

2008-08-03T00:23:00.004-06:00

I just committed the basic Linksys NSLU2 support to the subversion tree. It boots and runs on an NFS root. We're currently lacking support for the leds, the buttons and the on-board flash. Here's some basic instructions if you want to try it on your NSLU2. They are adapted from Sam Leffler's README file for the AVILA, which is adapted from other sources, including Olivier Houchard's and my instructions that have appeared.

Build world
Setup NFS root
Configure diskless system
Build and Install Kernel
Put Kernel in tftp area
Boot from redboot

Build world

The FreeBSD handbook can help you find the sources, etc Once you have them, and they are from -CURRENT after 8am on 2008-08-03, you can proceed to build the world. Each section is cumulative from the previous section for simplicity sake.

setenv TARGET arm
setenv MAKEOBJDIRPREFIX /tmp/$USER/obj
setenv TARGET_BIG_ENDIAN t
setenv TARGET_CPU_TYPE xscale
make buildworld

Setup NFS root

Find a server that has some disk space. You'll need about 250MB of space for a full FreeBSD image.

setenv ROOT /data/freebsd/roots/slug
make installworld DESTDIR=$ROOT
mergemaster -m $SRC/etc -D $ROOT -i -A arm

Setup diskless system
diskless(8) gives a good overview in how to do this. It is left as an exercise for the read to come up with all the settings. Usually /etc/rc.conf and /etc/fstab must be configured. /etc/ttys is another one that usually needs some tlc. If there's enough demand, I'll write about this in the future.

Build and install the kernel

make KERNCONF=NSLU buildkernel
make KERNCONF=NSLU DESTDIR=$ROOT installkernel

Note: the NSLU kernel is configured with the root filesystem mounted via NFS over the npe0 wired interface.

You should now have an NFS-mountable root filesystem with a kernel. The final step is to setup network diskless booting from the board. It is assumed you have a DHCP server operating on your network and the server is configured to supply the necessary information in the DHCP lease. If you run the ISC DHCP server the following configuration information is an example of how to do this:

option root-opts code 130 = string; # NFS / mount options

host slug {
hardware ethernet 00:d0:12:02:47:68;
fixed-address 10.0.0.221;
next-server 10.0.0.251;
filename "kernel-slug.nfs";
option root-path "10.0.0.251:/data/freebsd/roots/slug";
option root-opts "nolockd";
}

Note: the root-opts item specifies the root filesystem should be mounted with the nolocked option; this just short-circuits file locking requests so you don't get complaints from programs that use the pidfile(3) routine (e.g. devd)

Install kernel in tftp area

Place the kernel in the TFTP area for booting from the prom monitor. If your TFTP server returns file from /tftpboot (default) then do something like:

cp $ROOT/boot/kernel/kernel /tftpboot/kernel-slug.nfs

Boot using Redboot

+(hit control C here)
RedBoot> ip -h 10.0.0.251 -l 10.0.0.1
IP: 10.0.0.1/255.255.255.0 Gateway: 0.0.0.0
Default server: 10.0.0.251
RedBoot> load -b 0x200000 kernel-avila.nfs
Using default protocol (TFTP)
Address offset = 0x40000000
Entry point: 0x00200100, address range: 0x00200000-0x004db2d4
RedBoot> go

dmesg
Here's a complete transcript of my doing this. The .gz file listed below really isn't the simple gz file, but rather kernel.gz.tramp from the kernel build. Also, I've hacked my NSLU to have a serial port, which make development like this easier. See the web for instructions on how to do a no-mod breakin.

+Ethernet eth0: MAC address 00:14:bf:68:47:ba
IP: 192.168.0.1/255.255.255.0, Gateway: 192.168.0.1
Default server: 0.0.0.0, DNS server IP: 0.0.0.0

RedBoot(tm) bootstrap and debug environment [ROMRAM]
Red Hat certified release, version 1.92 - built 15:16:07, Feb 3 2004

Platform: IXDP425 Development Platform (XScale)
Copyright (C) 2000, 2001, 2002, Red Hat, Inc.

RAM: 0x00000000-0x02000000, 0x000723a0-0x01ff3000 available
FLASH: 0x50000000 - 0x50800000, 64 blocks of 0x00020000 bytes each.
== Executing boot script in 2.000 seconds - enter ^C to abort
^C
RedBoot> ip -l 10.0.0.12 -h 10.0.0.8
IP: 10.0.0.12/255.255.255.0, Gateway: 192.168.0.1
Default server: 10.0.0.8, DNS server IP: 0.0.0.0
RedBoot> load -b 0x200000 kernel.nslu.gz
Using default protocol (TFTP)
Address offset = 0xf0000000
Entry point: 0x00200074, address range: 0x00200000-0x003830fc
RedBoot> go
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.0-CURRENT #12: Sun Aug 3 00:09:07 MDT 2008
imp@lighthouse:/tmp/imp/obj/arm/cache/svn/head/sys/NSLU
CPU: IXP425 266MHz rev 1 (ARMv5TE) (XScale core)
DC enabled IC enabled WB enabled LABT branch prediction enabled
32KB/32B 32-way Instruction cache
32KB/32B 32-way write-back-locking Data cache
real memory = 33554432 (32 MB)
avail memory = 25894912 (24 MB)
ixp0: on motherboard
pcib0: on ixp0
pci0: on pcib0
ohci0: irq 28 at device 1.0 on pci0
ohci0: [GIANT-LOCKED]
ohci0: [ITHREAD]
usb0: OHCI version 1.0
usb0: on ohci0
usb0: USB revision 1.0
usbd_get_string: getting lang failed, using 0
uhub0: on usb0
uhub0: 3 ports with 3 removable, self powered
ohci1: irq 27 at device 1.1 on pci0
ohci1: [GIANT-LOCKED]
ohci1: [ITHREAD]
usb1: OHCI version 1.0
usb1: on ohci1
usb1: USB revision 1.0
usbd_get_string: getting lang failed, using 0
uhub1: on usb1
uhub1: 2 ports with 2 removable, self powered
ehci0: irq 26 at device 1.2 on pci0
ehci0: [GIANT-LOCKED]
ehci0: [ITHREAD]
usb2: EHCI version 1.0
usb2: companion controllers, 3 ports each: usb0 usb1
usb2: on ehci0
usb2: USB revision 2.0
uhub2: on usb2
uhub2: 5 ports with 5 removable, self powered
ixpclk0: on ixp0
ixpiic0: on ixp0
iicbb0: on ixpiic0
iicbus0: on iicbb0 master-only
iicbus0: at addr 0
iic0: on iicbus0
ixpwdog0: on ixp0
uart0: on ixp0
uart0: [FILTER]
uart0: console (115200,n,8,1)
uart1: on ixp0
uart1: [FILTER]
ixpqmgr0: on ixp0
ixpqmgr0: [ITHREAD]
npe0: on ixp0
npe0: [ITHREAD]
npe0: remember to fix rx q setup
miibus0: on npe0
rlphy0: PHY 1 on miibus0
rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
npe0: Ethernet address: 00:14:bf:68:47:ba
npe1: on ixp0
npe1: [ITHREAD]
npe1: remember to fix rx q setup
npe1: Cannot find my PHY.
device_attach: npe1 attach returned 6
ixpclk0: [FILTER]
Timecounter "IXP425 Timer" frequency 66666600 Hz quality 1000
Timecounters tick every 10.000 msec
bootpc_init: wired to interface 'npe0'
Sending DHCP Discover packet from interface npe0 (00:14:bf:68:47:ba)
npe0: link state changed to UP
Received DHCP Offer packet on npe0 from 0.0.0.0 (accepted) (no root path)
Sending DHCP Request packet from interface npe0 (00:14:bf:68:47:ba)
Received DHCP Ack packet on npe0 from 0.0.0.0 (accepted) (got root path)
npe0 at 10.0.0.12 server 0.0.0.0 boot file kernel.slug
subnet mask 255.255.255.0 router 10.0.0.1 rootfs 10.0.0.8:/pe/slug rootopts nolockd
Adjusted interface npe0
Trying to mount root from nfs:
NFS ROOT: 10.0.0.8:/pe/slug
warning: no time-of-day clock registered, system time will not be set accurately
warning: no time-of-day clock registered, system time will not be set accurately
Interface npe0 IP-Address 10.0.0.12 Broadcast 10.0.0.255
Loading configuration files.
No suitable dump device was found.
Entropy harvesting: interrupts ethernet point_to_point kickstart.
Starting file system checks:
mount_nfs: can't update /var/db/mounttab for 10.0.0.8:/pe/slug
Setting hostuuid: 1c728616-1dd2-11b2-9fa6-0014bf6847ba.
Setting hostid: 0xb1c1273a.
Mounting local file systems:.
Setting hostname: slug.
net.inet6.ip6.auto_linklocal: 1 -> 0
DHCPREQUEST on npe0 to 255.255.255.255 port 67
ip length 328 disagrees with bytes received 332.
accepting packet with data after udp payload.
DHCPACK from 10.0.0.1
bound to 10.0.0.12 -- renewal in 900 seconds.
lo0: flags=8049 metric 0 mtu 16384
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff000000
npe0: flags=8843 metric 0 mtu 1500
ether 00:14:bf:68:47:ba
inet6 fe80::214:bfff:fe68:47ba%npe0 prefixlen 64 scopeid 0x1
inet 10.0.0.12 netmask 0xffffff00 broadcast 10.0.0.255
media: Ethernet autoselect (100baseTX )
status: active
Additional routing options:.
Starting devd.
Additional IP options:.
Mounting NFS file systems:.
Creating and/or trimming log files:.
Starting syslogd.
/etc/rc: WARNING: Dump device does not exist. Savecore not run.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
Initial arm initialization:.
Additional ABI support:.
Clearing /tmp (X related).
Starting local daemons:.
Updating motd.
Mounting late file systems:.
Starting cron.
Local package initialization:.
Starting inetd.
Starting background file system checks in 60 seconds.

Thu Jan 1 00:00:39 UTC 1970

FreeBSD/arm (slug) (ttyu0)

login: root
Jan 1 00:03:24 slug login: ROOT LOGIN (root) ON ttyu0
Last login: Thu Jan 1 00:42:28 on ttyu0
Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.

FreeBSD 8.0-CURRENT (NSLU) #12: Sun Aug 3 00:09:07 MDT 2008

Welcome to FreeBSD!

Before seeking technical support, please use the following resources:

o Security advisories and updated errata information for all releases are
at http://www.FreeBSD.org/releases/ - always consult the ERRATA section
for your release first as it's updated frequently.

o The Handbook and FAQ documents are at http://www.FreeBSD.org/ and,
along with the mailing lists, can be searched by going to
http://www.FreeBSD.org/search/. If the doc distribution has
been installed, they're also available formatted in /usr/share/doc.

If you still have a question or problem, please take the output of
`uname -a', along with any relevant error messages, and email it
as a question to the questions@FreeBSD.org mailing list. If you are
unfamiliar with FreeBSD's directory layout, please refer to the hier(7)
manual page. If you are not familiar with manual pages, type `man man'.

You may also use sysinstall(8) to re-enter the installation and
configuration utility. Edit /etc/motd to change this login announcement.

nslug# ntpdate harmony
3 Aug 07:04:18 ntpdate[582]: step time server 10.0.0.6 offset 1217746845.716155 sec
slug# date
Sun Aug 3 07:04:19 UTC 2008
slug# df
Filesystem 1K-blocks Used Avail Capacity Mounted on
10.0.0.8:/pe/slug 55871592 49741636 1660230 97% /
devfs 1 1 0 100% /dev
10.0.0.8:/pe 55871592 49741636 1660230 97% /pe

When worlds collide

2008-08-01T15:31:00.002-06:00

Recently, I had to check out a linux tree at work from cvs. I only knew the repository, but didn't know the module to check out. So I made my best guess and started building the kernel. After plowing through a couple of little things, I hit the wall with:

scripts/Makefile.build:17: /pe/imp/xxx/linux-2.6.xxx/net/core/Makefile: No such file or directory
gmake[2]: *** No rule to make target `/pe/imp/xxx/linux-2.6.xxx/net/core/Makefile'. Stop.

Guess what someone forgot to do when they were doing the cvs import :-)

MIPS Diffs

2008-07-29T17:03:00.002-06:00

Although FreeBSD/mips has been committed to the tree, a few stragglers haven't been pushed into the tree for various reasons. Oleksandr Tymoshenko and I have been honing them down and have just published a set for review.

If you'd like to try FreeBSD/mips out, you can grab these patches from Oleksandr's directory. There's four diffs included. Please send any feedback to arch@freebsd.org.

Minor wi(4) hacking

2008-07-26T17:21:00.003-06:00

I finally had some time, a need for wireless connectivity in the shop and found my stash of old Prism 2, 2.5 and 3 cards. I thought I'd try them out with the new vap stuff.

First, there's a new filter on the firmware revisions in the driver. Very old versions of the firmware basically don't work without a lot of coaxing and workarounds. They are no longer supported. In addition, symbol cards are fairly rare and there's no readily available documentation for them, so support for them was removed as well. This filtering is a good thing, since it will keep people from using cards that are known to be broken. However, I had to fix a bug in the interrupt registration to make this work out OK (and to also stop spontaneous panics sometimes on attach if there was a lot of interrupt activity on a shared interrupt). I basically moved the interrupt registration to the end of the attach function, rather than the beginning and solved both problems.

All of these cards that I had don't have new enough firmware to support WPA. This highlights another problem: wpa_supplicant doesn't filter out WPA APs when it is looking for things to attach to, or otherwise provide a meaningful error message that would tell the user the reason that WPA isn't working is that their card is too old/lame to support this new-fangled stuff....

I wound up solving my problem with an atheros card that just worked, once I found them...

Taking a break from FreeBSD core team

2008-07-22T01:41:00.003-06:00

As has been recently announced, the FreeBSD project has selected a new core team. The project elects a new team every two years. This time, I've decided to take a break from the core team. I have a new son and I want to spend more time with him and the rest of my family.

I've had a great time on the core team. I've been serving now for 8 years. I've really enjoyed being on the core team and leading the project through some difficult times. Since my son and family are taking a lot of my time these days, I have limited time to spend on the FreeBSD project. I've done my time in the administrative side of the project for a while, and I want to spend my limited time working to make FreeBSD a better platform for the embedded platforms. I'm taking a 2 year break. We'll see how things are in 2 years...

I wish the new core team luck, although they are good people and they won't need my luck to be successful.

Quick Guide to FreeBSD hostap for FreeBSD 8

2008-07-15T21:09:00.006-06:00

I have a need to setup an access point. Many of the discussions on the web include all kinds of extra goodies that one normally sets up with the access point. But I just needed an access point: no dhcp, bridging, etc. Here's how I did it.

Recently, some really cool stuff when into the kernel for wireless. This is the VAP stuff, or the virtual access point code. This allows freebsd to decouple the code for doing access points, and a bunch of other stuff, from the code to drive the wireless hardware. Great in concept, and quite useful with some of the advanced wireless hardware.

However, it did change the way things were configured. The handbook is now out of date. Until it can be updated, here's what I did, with some explanations about each step.

First, I needed to create an rc.conf setup for my needs. Here's the one I came up with.

devd_enable=YES
ifconfig_wlan0="inet 10.0.0.1/24"
wlans_ath0="wlan0"
create_args_wlan0="wlanmode hostap mode 11g"
hostapd_enable=YES

devd_enable turns on the devd(8) process. This process is responsible for dealing with the different device events in the system. This is the hook that we use for hot-plug insertions, which is how we configure the wireless devices.

wlans_ath0 is used to create the virtual interface wlan0 associated with ath0. This is how all the wireless stuff is setup now. You configure the basics with the wlan_IF=wlanX command, and set more specific stuff via the wlanX interface.

create_wlan0 gives the arguments with which to create the wlan interface. There are many that can only be created at wlanX creation time, and it happens that hostap is one of them. I'm not 100% sure that I need to bother with the 11g mode command or not here, but it seems to work.

ifconfig_wlan0 sets the device's IP address. In my example, I wanted the interface to have an IP address, but not be bridgable since I didn't have any other interfaces on the laptop I wanted to be the AP for the thing I'm doing.

hostapd_enable turns on the hostapd daemon. You won't get far without this, so we turn it on.

We've enabled a daemon, and now it is time to configure it. Here's the config file that I used for hostapd.conf:

interface=wlan0
hw_mode=g
macaddr_acl=0
auth_algs=1
ctrl_interface=/var/run/hostapd
ctrl_interface_group=wheel
ssid=bsdimp hacking blog

itnerface specifies which interface to use. Duh! However, this needs to be the wlan device, not the hardware device.

hw_mode sets the hardware into 802.11g. Not sure if this is needed, but it works for me.

macaddr_acl is wide open: it allows anybody to connect. As you may have noticed, this setup is very permissive. It has nothing locked down at all. It should be the first thing you do as an experiment before proceeding on to enabling WPA or WPA2 to get a reasonable level of security.

auth_algs sets which authorization algorithms can be used. See the hostapd man page for details.

ssid sets the name of the network you are serving.

The rest can be gleaned from the manual.

Chances are this post should be updated, so if you have any suggestions on how to do that, please leave then as comments.

Minor mips reorg and plans for the future

2008-07-06T10:49:00.006-06:00

In trying to bring the MIPS tree more into line with idealized FreeBSD practices, I've moved the SoC directories up one level in the hierarchy. Future changes will make it easier to define different boards, as well as stylizing these sorts of things across different CPU architectures. I need this for the Plat'Home OpenMicroServer work I've started on, so I'll use that to drive these changes.

For the curious, I've moved mips/mips32/adm5120 -> mips/adm5120, likewise with the idt, malta, and sentry5 directories. While all of these were, strictly speaking, mips32 CPUs, having them in their own subdirectory doesn't make a lot of sense. There's never going to be so many of them that having an extra layer of indirection is going to help. Also, mips32 is ambiguous. Does it mean the MIPS ISA known as MIPS32 and MIPS32r2? Does it mean that the ports are running in 32-bit mode? It wasn't clear where a 32-bit kernel for a mips64 processor would live. Also, the idt parts aren't completely mips32 compliant in a couple of minor details. Is that non-compliance enough to move them elsewhere?

The next set of changes will try to tease apart the boot loader environment from the SoC that it is running on. While many SoCs have their own bootloader, often times one may also run uboot on the same hardware. In addition, things like uboot are used in many different platforms, so duplicating that code will grow tiresome.

Once the boot loader stuff is better abstracted, the next set of problems will be how to create a system that can run on multiple different boards of the same SoC. At first, this will be statically compiled, but eventually it will be dynamic.

Another key component of this work, whose overall goal is to make it easier to add board support to FreeBSD embedded platforms, will be an overhaul of the hints system. While it can be made to work, there are some practical problems it solves poorly that are desirable to have solved better. For example, it has no good way to tie attributes or parameters to hardware. Today, one can only tie them to a driver instance, which itself can be tied to hardware.

Finally, the last planned component in this work is a simplification of creation of busses with attributes and fixed or semi-fixed children. You can't have variants of hints loaded today that say "If I'm running on a AT91RM9200, use this set, but if I'm runnning on an AT91SAM9620 use this other set of hints" to enumerate the busses. This makes it hard to achieve the dynamic selection of boot code to run needed to allow kernels to boot on a variety of boards.

Most of this is low-level code janitor stuff. However, it is code janitor stuff that's needed and will be important to FreeBSD's success in the future.

Adding a Serial Port to a D-LINK DIR-615

2008-06-27T22:12:00.007-06:00

A while ago, I added a serial port to the D-LINK DIR-615 (HW Rev A1). Today, I'll document how I did it.

First, you'll need a level converter circuit. There are many on the network. You can find a good write up on them and links from adding a serial port to a NSLU2. I always use the Parallax USB2SER that I bought years ago. It is very reliable, and I've had several provide years of good service to me. They convert 3.3V directly into a USB serial port, which vastly simplifies power and cabling. Since that's what I used for this project, all the instructions are geared towards it.

First, you'll need to disassemble the D-LINK DIR-615. There's only two screws so this is easy. Place the D-Link with the label with the serial number face up. There are four rubber feet on the bottom of this unit. Carefully pry up the two nearest to the Ethernet ports. Remove the screws found under them. The screws are clearly visible in this photo.

Carefully remove the lid. It should pry off just above the plastic bar that runs over the ethernet ports. You'll see a green PC board. On the right hand side of the unit, you'll find a header with 4 pins labeled CON5. They are labeled VCC, TX, RX, GND. This is the serial port. In my unit, it was hidden under the internal antenna, so the photos show it moved over a bit. Note also in figure two the board says "DIR635B1" on it...

I connected these pins to a ribbon cable and ran them out to a header that my USB2SER could fit on. I connected them so that they would match up to my USB2SER which had pins in the order GND, RX, TX, RES. The last one is unconnected. TX should go to RX, RX to TX and GND to GND. I didn't connect VCC, since it wasn't needed. USB2SER gets its power from the USB port. I reconnected everything and plugged the USB2SER into my computer. I threaded the ribbon cable out through the holes in the plastic, which is what you see in the first photo. Here's everything hooked up. I didn't include a photo of the USB2SER plugged into the serial port of my laptop for obvious reasons...

I see that it has a uboot bootloader:

% tip ucom0
U-Boot 1.1.1 (Jan 19 2007 - 11:08:07)
CAMEO uBoot Linux Loader version: 1.3.0.0

DRAM CS[0] base 0x00000000   size  32MB
DRAM Total size  32MB
before entry mvFlashInit
Flash: flashStructGet manu 0x89 id 0x17
INTEL 28F640J3A (64 Mbit)
Size:  8 MB,Bus Width: 1, device Width: 1.
Flash base: 0xff800000,Number of Sectors: 64 Type: REGULAR.
[8192kB@ff800000] Flash:  8 MB
Addresses 20M - 0M are saved for the U-Boot usage.
Mem malloc Initialization (20M - 16M): Done
*** Warning - bad CRC, using default environment


Soc: 88F5181 B1
CPU: ARM926 (Rev 0) running @ 500Mhz
SysClock = 166Mhz , TClock = 166Mhz


USB 0: host mode
PCI 0: PCI Express Root Complex Interface
PCI 1: Conventional PCI, speed = 33000000
Net:   egiga0 [PRIME]
Hit any key to stop autoboot:  0
Marvell>>

So that's how you add a serial port.

Old cards and the information gap

2008-06-23T01:20:00.004-06:00

This time I'll explore some of the issues relating to hacking on older hardware. I'll talk about expanding the work I did for the Olicom OC-2220 to the combo OC-2232 and the problems encountered.

In a previous edition, I talked about the Olicom OC-2220 card and getting it working. It works great, and in fact, I'm using it right now to make this blog entry. However, the next card I tried didn't fare so well. This is the Olicom OC-2232, which added a 33.6k modem to the same basic design as the OC-2220. It exposes the ethernet and modem such that I had to augment the PC Card system to access. I had to be able to specify the config entry to use to configure the resources for the card. However, when I started, FreeBSD's PC Card system lacked this feature.

I was able to augment FreeBSD's PC Card system to have a feature that's been requested infrequently. The ability for the driver to select the config entry to use. Right now, all of the FreeBSD PC Card drivers have been able to use the config entry that the system guesses is the right one. However, there are some combo cards that pre-date the PCMCIA multifunction standard that need to select a config entry.

So, after adding this, it was a simple matter to get the ex driver selecting the appropriate alternate config entry for the Olicom OC-2232 that I have. This card has the Intel 82595 and a 16550 compatible 33.6k modem on it. By default, FreeBSD will happen to pick the entry that corresponds to the modem only. However, with my hacks I have been able to make it select either the ethernet side only or both. FreeBSD needs a few changes to allow it to support both of these devices at the same time, but I'll write more about that at another time.

So the card comes up, and I'm able to attach to the ex driver. But there's a problem. dhclient doesn't finish. For the OC-2220, this returns immediately, so I know that the dongle cable is good, as is the port on the hub. The OC-2232 uses the same cables, as far as I can tell, but when I try to push packets through, I never get any interrupts, so dhclient never finishes. I've not instrumented the driver to see if adding a watchdog timeout would help make this work in the absence of interrupts or not.

This leads me think about some of these old cards that I have. It is impossible to get information on them. The best I've been able to find is binary only Windows and DOS drivers for them. These drivers don't work with the current version of Windows that I have on my laptop. This makes it hard to know if the problem I'm encountering is a hardware problem or a software problem. There's no support for this card in Linux (the base kernel doesn't support and 82595 based PC Cards, and it is nearly impossible to find a driver for the few cards that used this chip that wasn't incorporated into the kernel).

So assuming the card is good, there's no source available for it. I knew there was a similar card made by Silicom called the EtherSerial card. It has a real serial port in addition to the 82595 ethernet controller on it. Searching for it was difficult. Most likely there's a trick to forcing google to tell me "Where do I find the file silpmcica-1.02b.tar.gz?" with all the variant spellings and encodings (.tgz, tar.bz2, .tbz, .tbz2, etc), but I didn't see it. Altavista used to have this feature, and I'm sure that some sharp reader will tell me how to make google do it.

I was able to find, using the internet archive, the driver for the Silicom card. Well, kinda. I was able to discover the name of the file, and also discover that the internet archive didn't cache the files that people put on their FTP sites, so when I clicked on the driver link there, it tried to grab it from Silicom's current FTP server, which isn't setup for anonymous operation.

I was able to find the datasheets for this old part, however. I was lucky this part was popular enough that the various datasheet web sites have it available. There were three variants of the part, and only the FX variant appears to have been used in the PC Card devices.

So the resort of looking for data for this card was somewhat less successful than looking for data for the Crystal Semi based card that I had. There I was able to find everything I needed, but my needs were more meager. Now, I'm left wondering: do I have bad hardware? Do I need to prod some hidden register to turn on interrupts? Do I need to disassemble the drivers that I found to make further progress? These are all good questions, and I don't have the answers to any of them. All the questions that lead me to make progress involve a lot of time doing unfun tasks that really gain me no pleasure or knowledge.

Anyway, my trip down memory lane must end soon. I have other commitments in FreeBSD that I need to return to soon. This was a fun diversion (and hardware that's easy to carry), but I'm afraid this unsatisfying dead-end will likely be where this problem sits for a while as I have some processor porting work to do for people that have donated hardware to me. I've put that off long enough based on the fun I had one night with one PC Card. I'm afraid I'm going to have to put them back on the back burner unless new information surfaces. The other hardware is more relevant, and while it is fun to misbehave sometimes with something that may not be relevant for the fun of it, there's a different kind of fun to be had when doing things with more relevant hardware, and I think it is time I got back to that kind of fun in my spare time.

Good collection management software

2008-06-16T16:24:00.002-06:00

Regular readers of this column know that I have a large collection of PC Cards. Some would say 'too large' and I'm not sure they'd be wrong. I was out in the garage last night looking for a specific card. I couldn't find it. I know it is there, somewhere, but don't know exactly where. Maybe I'll find it tonight when I look in a place that I remembered driving into Boulder today.

This got me to thinking: how can I manage all these cards. I need to figure out some way of recording generic information about the card. Where it lives, what dongles it needs, what driver it uses, etc. It would also be nice to know when I last tested it, how well it worked, etc. Maybe some notes about the card, like "This card has a blue label on it, and needs to have the dongle wiggled just so: .... for it to work." Oh, and it would also be nice to generate reports so that I can maintain a list of cards that I've tested and when on a web page automatically. I did this years ago, but stopped when it became too big a pita to keep current.

In the past, I've not seen anything that's good at doing all these things. I thought I'd ask my readers if they knew of anything that solves this problem? I know that there's many tools that could be used to build a solution, but many of them require a fair amount of customization to make into a workable solution. A quick google search doesn't turn up anything that's more sophisticated than a generic database. Does anybody know of something that's fairly close?

Olicom OC-2220 works

2008-06-13T09:22:00.003-06:00

After applying John Bladwin's MPSAFE patches to the ex driver, the ex driver works with the Olicom OC-2220. If you are unfamiliar with this card, I'm not surprised. It was made in the mid 1990's. From a network perspective, it isn't that important. It is a slow 10BaseT card that doesn't seem to do full duplex.

However, from the PC Card perspective, there's some interesting things with ex. There's the Olicom OC-2232 that includes a modem. But it includes it in an interesting way that isn't quite standard. How do we pull in the serial port in this case? There's also some cards from Silicom that have a serial port bolted on as well, but in a different way. i've not seen good documentation for either of these cards, but at least there's a driver for the silicom available (or was once upon a time) for Linux. I sure hope that wasn't one of the things I lost in my disk crash...

So again, despite the age of the underlying chip, there's some interesting problems here. The trouble is that the problems are such that they can't easily be solved in an hour of hacking, so they may continue to linger....

Crystal Semiconductor Driver

2008-06-12T00:28:00.006-06:00

Since I've been maintaining the PC Card layer in FreeBSD, I've accumulated a lot of PC Card hardware. Well over 100 different PC Card r2 (16-bit) and CardBus (32-bit) cards. Recently, Robert Watson started pushing to have all the network drivers in the tree be MPSAFE. John Bladwin posted some patches to the if_cs driver to bring it up to those standards. Since I had a soft spot in my heart for this driver for reasons I don't entirely understand, I took the IBM EtherJet PC Card that I had with me on a recent buisiness trip with the vague notion of testing out John's patches.

I discovered that the driver didn't work. It didn't work before the patches, nor after them. The breakage was minor and I likely wouldn't have noticed it except for the fact that I didn't have the Ethernet cable plugged in when I was doing the early testing. Since the breakage annoyed me, I went ahead and fixed it. I also noticed that the reading of the EEPROM was very slow, and caused me to drop characters when I inserted the card. I hate that. I discovered a number of DELAY statements in the EEPROM reading which were the cause of the problem. I thought "this is nuts" to myself, and went off to locate the datasheet for this part. After reading through it, I discovered the EEPROM busy bit that I polled in place of the hard coded 3ms DELAY. I went from having about 1.2s of DELAY and other busy waiting when I inserted the card to having on the order of 800us, a speedup of about 1500x.

This brings up a number of philosophical questions. Why did I bother with such an outdated chip? Why speed up a one time operation? Why spend any time at all on this after a busy day of meetings and crisis management?

Let's answer the last question first. I did it because I was absolutely exhausted by my meetings. While productive, and they helped to drive the problems of the project I was working on towards resolution, they were draining. By hacking on this driver I had a very safe area to make improvements. Unless I broke the build, nobody would notice the work, yet I'd have the satisfaction of a job well done. Since it is in an out-of-the way part of the kernel, I had the room to fail without throngs beating down my door. I had complete freedom to tinker and tidy as I saw fit, and doing so was very relaxing and invigorating after a long, hard day. Maybe I'm just weird, but sometimes a little bit of hacking to make something better is just what my brain needs to relax and unwind.

The reason I was spending any time at all on this was that this part made a resurgence in recent years inside embedded systems. There are several embedded designs that use this part when they don't have the ethernet on the SoC. More recent parts have all this built into the SoC, so the popularity has waned once again, but I knew some ARM boards used it, so in the back of my mind I had a justification for spending time on it based on my FreeBSD embedded hat that I wear from time to time. I was shocked to get email from someone who had one of these boards and wanted to know how to write a front end for this driver for that board within a day of my committing the cleanups. That was a nice bonus. Time will tell if he follows through, of course, but I was very amused by it.

Finally, why spend time optimizing an operation that happens only once at startup? I hate losing characters when I type. Since I didn't want to re-write the syscons keyboard driver to have a fast interrupt handler to read the keystrokes and queue them for later processing (the only thing that will interrupt a DELAY is a fast interrupt handler), I thought this was the easier path to solving my problem. I spend a lot of time at work making sure that the right priority calls are made, and it felt good to be "bad" and work on something that clearly is the wrong priority call in terms of everything else that's going on in FreeBSD.

I guess it all boils down to "I had an itch, and it felt good to scratch it." rather than any rational thought process. Sure, I justified it after the fact in many different ways, but it felt fundamentally good to just make something better for the sake of making it better and not have to worry about big picture this, or deadline that, or you broke my whatsit complaints. I had a well defined technical problem, I had to research solutions and hack a bit of code. It felt good being able to find the data I needed, fix the code, test it and see it working and passing packets again. I wish all problems that I faced in my technical life were so easy and rewarding.

Oh yea, and it turned out that I found a bug in John's patches and he was able to commit them to the tree, eliminating one more Giant Locked driver. I'm sure Robert will be happy. I'm sure there are others that would have been just as happy to have the impending removal of Giant Locked network drivers sweep it into the dustbin of history. Maybe I have that soft spot in my heart for it because it was one of the drivers that the Japanese Mobile community hacked years ago and seeing it the tree reminds me of happy trips to Japan, drinking way too much sake, and speaking bad Japanese as a result? Who knows....

Plat'Home provides OpenMicroServer

2008-05-31T09:20:00.003-06:00

Plat'Home has generously shipped me an OpenMicroServer. Plat'Home has been doing Emebedded things for years in Japan. They used to have the coolest store to go visit in Akihabara, but that recently closed and now they are expanding their markets internationally through the web. I have fond memories of buying my first HappyHackingKeyboard there almost a decade ago...

So what is an OpenMicroServer? It is a ruggedized router with extra features to allow it to operate in a wider range of environments than just a router. It has DIO pins for control and monitoring. There's a console serial port, as well as an extra serial port. There's three multicolored LEDs as well. It operates over an extended temperature range. Inside there's a CF slot in case the built in flash is insufficient for your needs. Oh, and I almost overlooked the pair of USB 2.0 ports.

Under the hood there's an MIPS cpu inside the box, which is why they send it to me. I've started working on FreeBSD/mips support for it. There's a RMI Alchemy™ au1550 400MHz powering this box with 128MB RAM and 16MB Flash. It supports 3 ethernet ports. 1 10/100 port that also supports POE (to get power over the ethernet) and 2 that are 10/100/1000 gigE ports.

The only bad thing I can say about it is that it ships with Linux preinstalled... :-)

You can find more information about the specs here. Or general information about Plat'Home here.

Stay tuned for updates on how the port is going.

More mips support code committed

2008-04-26T23:32:00.003-06:00

This weekend, I've committed more mips support code. Everything except for the tool chain is now in the tree. The tool chain will be pushed in over the next week or so. I have to verify that some changes I remember having to make to the tool chain are compatible with the tool chain that's in the FreeBSD repository. When everything has been pushed into the tree, I'll enable it as part of the make universe target.

FreeBSD/mips is starting to get attention from other FreeBSD developers now that it is in the tree. Alan Cox has started to audit the Mips pmap, for example. This should help FreeBSD/mips transition to the next level of support.

Many changes are still needed for the mips port. We need to audit the basic libc functions to make sure that we haven't stubbed anything out that's going to bite us in the future. We need to look to see if there's anything that can be refactored from the 3 different embedded platforms in the tree today (powerpc, arm and mips). Better documentation and porting to new chips/boards would help us to make things more portable.

FreeBSD/mips entering the tree

2008-04-14T23:37:00.003-06:00

I've started committing FreeBSD/mips to the tree. It includes support for the ADM5120 SoC, a generic MIPS 4Kc in a Malta board, the IDT RC32434 (found on RB533) and preliminary support for the Broadcom Sentry 5 series of CPUS. It will likely be a few days before it is all in the tree and buildable. There's a number of issues that are cropping up as I'm pushing the code into the tree that weren't apparent until after the commits and merges back into p4. In addition, support for another platform is delayed pending approval to commit.

Look for more news, how-tos, etc in the days and weeks to come.

Tokyo AsiaBSDCon 2008 Dev Summit Day 1

2008-03-27T02:12:00.003-06:00

Hello from Tokyo.

Today we had a developer summit at AsiaBSDCon 2008. We talked about a number of items. We talked about the FreeBSD/mips: it is booting multiuser. We'll be committing to FreeBSD soon.

We talked about ways to making it easier to have parts of FreeBSD be optional, with the default being 'on'.

There's this ugly hack which allows one to test the suspend/resume paths of drivers. Check it outthis patch.

D-LINK DIR-615 B2 hardware

2008-03-20T23:52:00.004-06:00

If you go buy a D-LINK DIR-615, make sure that it is hardware revision A1. Hardware revision B2 has less flash (only 4MB) and RAM (16MB) and a completely different processor: Ubicom IP5090U cpu. This CPU uses a new instruction set (ubicom32) that isn't supported by any open source operating system that I could find. This is also the previous generation of the ubicom processor. The IP51xx is the latest generation of processor, and evidentially its only used on the DIR-655.

A good place to look for information about what processors and radios are in the different access points on sale now, including version information, can be found over at small net builder with information about the D-LINK DIR-615 B2.

D-LINK DIR-615

2008-03-18T10:03:00.002-06:00

The other day I saw that Target was selling the D-LINK DIR-615 draft N router for $50.00 each. Since it had been a while since I purchased a wireless router to hack, I bought one. It turns out that this was a wise choice.

This router chip has an 88F5180NB1 CPU. This is a Marvell Orion SoC. Support for this family of processors is coming into the tree shortly.

There's a FS28F640 flash chip on board. This is a 64Mb/8MB of memory. With compression, FreeBSD can fit into this size. There appears to be 32MB of RAM, which is also enough to run FreeBSD.

There's a 88E6061-LAJ1 PHY chip which should be relatively easy to program.

Finally, it has a mini-pci slot with a 802.11n card based on the 88W8361P-BEM1 chip. Maybe a driver for this chip will come along. I've not investigated the GPL tarball to see if there's a driver in source for this, or just a binary module.

There's a GPL tarball available at the D-Link web site.

Finally, there's two interesting connectors. One looks to be a JTAG connector, the other looks to be a serial port.

My next step is to look into what's coming out of the serial port and what's in the GPL tarball for hints while I'm waiting for the Orion support to come into the tree.

FreeBSD/arm on a Linksys NSLU2 (slug)

2008-03-16T00:33:00.003-06:00

Thanks to some encouragement from Gavin Atkinson, I have been able to boot the AVILA kernel on my Linksys NSLU2 (slug). I have it running of NFS root at 266MHz. Not super fast, but not too bad. I need to clean up a bunch of stuff still, but at least I have the basics working.

I'll post more in a few days, once I get a chance to clean up what I have and submit it to CVS. But this is very exciting...

Buffalo LinkStation PC-L4PWAP

2008-03-12T21:57:00.003-06:00

Today I'm writing about the Buffalo LinkStation PC-L4PWAP. It was on clearance at Circuit City, so I thought I'd take a chance and pick it up. At the very least, I'd have a new 'receiver' on my network to play tunes through at parties.

I cracked it open when I got it home. There's a Phillips PNX-1520E processor running at 300MHz inside. A quick web search turned up that it has a TM3260 CPU core with a lot of perripherals (I2C, PCI, IDE, etc). The TM3260 is a CPU from Phillips that is a VLWI processor. I had rather hoped when I saw pnx that it had one of their MIPS cores in it. But no such luck.

I didn't bother looking at the amount of memory or the amount of flash on the board once I discovered it wasn't MIPS based. I guess I'll have to content myself with listening to the music it is spooling off a uPNP server on my network... once I get one of them setup. I'll report back on how well it works.

Spot the Bug!

2008-01-10T22:27:00.000-07:00

I was looking over some code for someone today. They were having problems with preemption in the write routine and couldn't understand why, since they said to take Giant for all devsw operations. In digging into the problem, I discovered the following code. Maybe you can spot the bug:

#ifdef D_NEED_GIANT
#define FOO_GIANT D_NEEDGIANT
#else
#define FOO_GIANT 0
#endif

(FOO_GIANT is used in the devsw data structure to paper over differences between 4.x and 6.x in this driver).

I think I know who did this change, but without cvs access to the code I can't be sure. Which is good, because I think it was me, but it might have been the person I was helping out. The change was made almost 4 years ago. This code has been in there that long and it wasn't until an especially demanding application came along on a faster CPU was it a problem.

Just goes to show you can never be too careful, or test too much...

Random Acts of Kindness

2008-01-05T11:05:00.001-07:00

I went to my old house yesterday. Like many others, my old house hasn't sold in a while. There was a showing today, so I had to shovel the snow off the driveway. While I was there I discovered a most interesting package on my doorstep.

It was a nice, large calendar of African animals in the wild. I hadn't ordered the calendar, so I was very confused. It was from Cafe Press, so I thought it was a dividend from an item I'd designed and put up there. But after shoveling a little I realized that couldn't be it: I've just bought stuff from Cafe Press in the past. I hadn't designed anything for there. I'd thought about it, but never put my thoughts into actions.

After I finished shoveling, I took a closer look at the package. I found a card I'd overlooked before. Inside the card there was a nice note thanking me for the USB and other bits of FreeBSD work I'd done. It was anonymous, so I don't know who to thank for this bright spot in a busy day.

It is the little things like this that really make my involvement in the open source community enjoyable.

Thank you!

Building a big endian ufs image on a little endian system

2008-01-03T11:04:00.000-07:00

Let's say you want to create a ram disk for a kernel, or a ufs image for an emulator. To make it interesting, the image needs to be in big endian format and you just have an intel x86 machine which is little endian. How do you proceed? If you were building a little endian image, the answer is simple: just newfs an md partition, mount it and copy. But that won't work for a big endian image. Fortunately, the answer is straight forward:

create a tree of files you want in the image
install the makefs port
use makefs to create your image.

You'll need a /tmp file system of at least 2GB to follow these steps exactly. If you don't have one this big, then you'll need to use a different path below to use a different partition.

Create a tree of files

You'll need to create an image of what you want. Let's assume that it is a powerpc root disk for FreeBSD. For the sake of simplicity, I'm assuming that the kernel comes from elsewhere. If not, then you'll need to add the buildkernel/installkernel targets

setenv TARGET powerpc
setenv MAKEOBJDIRPREFIX /tmp/imp/obj
make buildworld
sudo make installworld DESTDIR=/tmp/imp/ppc-root
sudo make distrib-dirs DESTDIR=/tmp/imp/ppc-root
sudo make distribution DESTDIR=/tmp/imp/ppc-root
echo "hostname=ppc-qemu" > /tmp/imp/ppc-root/etc/rc.conf
echo ifconfig_DEFAULT=DHCP >> /tmp/imp/ppc-root/etc/rc.conf
echo /dev/ad0a / ufs rw 1 1 > /tmp/imp/ppc-root/etc/fstab

The above steps build the FreeBSD world, and create a directory that could be used as a root directory. It can be used via NFS, or placed on a flash or in a ram disk.

Installing the makefs port

NetBSD created a tool to create file systems. Colin Percival ported this to FreeBSD and make a FreeBSD port to boot. He's no longer maintaining the port, but I try to keep it compiling. Installing the port is done in the usual FreeBSD way:

cd /usr/ports/sysutils/makefs
make all
sudo make install
make clean

Making the image

Once you have makefs installed, creating the image is the next step. You'll need to figure out how big you wish to make the image. For me, I think that 500MB is a good size. The base system is about 250MB, and that leaves room left over for testing programs in the emulator.

makefs -B big -s 500m /tmp/imp/ppc-root-image /tmp/imp/ppc-root

The resulting file, /tmp/imp/ppc-root-image is going to be 500MB in size.

An old-school hack remembered

2007-12-29T13:11:00.001-07:00

Way back in the days of yore, when Turbo C++ had just came out, I was using a DEC Rainbow 100B+. For those not old enough to recall, this was a machine that came out in 1985 with software that came out in about 1990. Boreland had done a good job of not using all the IBM BIOS calls, so its compilers and such were useful on the Rainbow. The Rainbow had an 8088 processor in it, but wasn't IBM Compatible (being released at about the same time the IBM PC was). The Rainbow was better in a lot of ways, which is how I wound up with it, but worse in other ways...

However, Boreland hadn't done a perfect job. They had used an unused software interrupt (INT 18h) for memory management of the compiler. Early versions of the Turbo C++ used it in only a few places. Patches quickly appeared in the Rainbow community to allow one to run TurboC++ on the Rainbow. These patches were about a hundred lines of code and just patched the offending INT calls to use a different vector (one that was unused on the Rainbow).

I had seen these patches, and when my job was using C++, I thought it would be cool to be able to compile code for it at home. I went out and bought Turbo C++, only to discover it was a newer version of the compiler than the one the patches were for. They just didn't work. So I wrote the author of the patches, who seemed a little put out with me for asking so soon after the compiler was released. Feeling a little bad for not having given more to the online Rainbow community that had given me so much cool software, I resolved to create patches myself and post them.

To make a long story short, I failed. The code was too complicated in the new version for someone of my meager skills (at the time) to produce patches. The calls were all over the code, and the sequence of bytes that invoked INT 18h were in many places that turned out to be data.

Being young, and still struggling to make ends meet, I wasn't about to let this get in my way. I didn't want to have wasted that $80 on software I couldn't use and now couldn't return. Buying a new machine was out of the budget at the time. So I came up with a clever hack, which I'll describe now.

On the DEC Rainbow, INT 18 was used for video output. So when the compiler ran, it would call INT18, which would then dive into the BIOS with loco registers crashing the machine, or just not working. The trick turned out to be simple. I used a TSR to fake things out. The TSR would load at an address that could be known at run time. This TSR hooked into the MS-DOS interrupt vector which was used to set interrupt vectors and into the interrupt vector that was called when a progam terminated. I also hooked into the INT 18h interrupt vector to vector calls to it through my code. When a normal program would run, it would get the system's INT 18h call, and all the video would work. This let me run my editors, terminal emulators and games without rebooting. When Turbo C++ would run, it would install its own INT 18 handler to do its thing. My program intercepted this call and just saved the target for later use rather than writing it into low memory to give it effect. Later, when an INT 18 call happened, I'd check the calling address. If it was at or below my TSR or in the ROM BIOS, then I know it was something in the system making the call and I called the original INT 18 handler. If it was above my address, I knew it was boreland's compiler and would call the address it had tried to install earlier. When the turbo C++ terminated, I'd reset things to the initial state.

I did this trick almost 18 years ago now, yet it still strikes me as a clever way around a problem. The program took me an afternoon to write and debug, yet provided me with years of useful service until I retired the Rainbow...

Building a bootable ARM SD

2007-12-12T12:28:00.000-07:00

A few weeks ago, I wrote up how to make a bootable i386 image. People have been asking me to extend it for my ARM platform.

Here are the instructions for a hypothetical box that has a boot loader that can load an image from a FAT partition of a SD card, and FreeBSD runs off of a ufs partition on the SD card. This will be for a big endian ARM target, based on the AVILA kernel. I'm also assuming that this platform doesn't use FreeBSD's normal /boot/loader.

setenv MAKEOBJDIRPREFIX /blah
setenv TARGET arm
setenv TARGET_ARCH arm
setenv TARGET_CPU xscale
make buildworld
make buildkernel KERNCONF=AVILA
fdisk -I da0
fdisk -f disk-partitions da1
newfs -t msdos /dev/da0s1
bsdlabel -w da0s2 auto
newfs /dev/da0s2a
mount /dev/da0s2a /mnt
make installworld DESTDIR=/mnt
make installkernel DESTDIR=/mnt KERNCONF=AVILA INSTALL_NODEBUG=t
make distrib-dirs DESTDIR=/mnt
make distribution DESTDIR=/mnt
echo /dev/da0s2a / ufs rw 1 1 > /mnt/etc/fstab
echo ifconfig_DEFAULT=DHCP > /mnt/etc/rc.conf
echo hostname=demo >> /mnt/etc/rc.conf
cp /mnt/boot/kernel/kernel /tmp
umount /mnt
mount -t msdos /dev/da0s1 /mnt
cp /tmp/kernel /mnt
umount /mnt

In step 8, one needs to have a special file. Here's what I use, but one needs to automate it a little more. Here's a hand-tuned one that I used for a 1GB SD card I had laying around.

# partition table
g c940 h32 s63
p 1 6 63 201537
p 2 165 201600 1780128

Clearly, one needs to tune the above for their specific part. NanoBSD tries to automate this process, but that is beyond the scope of this posting.

Update: fixed step 17.

Article about my past life

2007-12-11T11:19:00.001-07:00

Wired has a great article about the high precision hobbyist timing community. These folks are less fanatical about timing than the professionals, but not by much. You know you are a true time geek when you understand the joke in Tom's quote from the article about his trip to a tall mountain near his home "It was the best extra 22 nanoseconds I've ever spent with the kids." Check it out here.

Cisco and Observations of a new Mac user.

2007-12-10T11:17:00.000-07:00

Last week I started at Cisco. The entire week I tried to figure out how to drink from a fire hose without getting hurt. This week, I'll be learning where to go for help with fire hose related injuries. :-).

Seriously, I've joined a very dynamic and energetic group here at Cisco. The job so far has been a blast, and I'm really enjoying the work. We're at the early stages of many things in my group, so it would be a little premature to talk about them right now. since Cisco hires a lot of people, they have managed to optimize the typical new-hire HR madness enough that I was able to get something accomplished last week.

And I even found time to tweak the new MacBook Pro that showed up on my doorstep. This is the first employer provided laptop that I've had in at least 15 years. The last one was a Tadpole Sparcstation II class machine... I knew that Apple had done a great job on the GUI, but when I was able to setup printers and get onto my Samba shares without doing more than a few key clicks to accept automatically discovered items, I was hooked. On my Windows box that I keep around for the rest of the family, these two activities took me a couple of days of hacking to get barely working some of the time. OS X Leopard just works, and I have all the unix tools I know and love.

A quick thumbnail to cross debugging core files

2007-11-28T00:19:00.000-07:00

This is just a quick thumbnail about how to take a core file from a FreeBSD/arm box and debug it on a FreeBSD/i386 box you cross compiled the FreeBSD/arm image on.

For normal gdb, one would just type:

gdb prog prog.core

and be done with it. However, without careful setup, this won't work when cross debugging. If you do the above, you'll get a number of warnings or errors from gdb when it tries to load the i386 libraries with the arm binary.

There is a simple trick, however, that gets around these problems. It can be used both when debugging cross architecture and when debugging different a core from one release on a newer/older release of FreeBSD. This example, I use 'gdb-arm' which is the debugger I use for cross debugging.

# gdb-arm
(gdb) set solib-absolute-prefix /path/to/built/image
(gdb) file prog
(gdb) core-file prog.core
(gdb)

By setting solib-absolute-prefix, you are instructing gdb to use that path as a prefix for all files it has to look up. So rather than /lib/libc.so.7, it will look in /path/to/built/image/lib/libc.so.7. Now that you have all the libraries that you need for debugging, you'll be able to track down the problems in no time.

This is just the barest of sketches for what to do. There are many other ways to make debugging easier in a cross debugging environment, and gdb can do more than just look at core files. But those topics will have to wait for another day. The curious are invited to read through the gdb info files. They are very complete, and well indexed by google or other search engines.

SDHC purchased

2007-10-18T18:42:00.000-06:00

Today, while I was ordering a new keyboard for my laptop (a 1-year-old is very hard on them), I ordered a 4GB SDHC card. When it arrives, I plan on expanding FreeBSD's support for SDHC. I had been waiting for the card to arrive to post this, but so far it has taken two weeks and no SDHC card. I bought it on ebay, but the buyer was located in Hong Kong, so my card is now stuck in customs. Let this be a warning to others.

The SDHC card implements the SD Card's 2.0 specification. It is unclear to me if the only change is in the commands sent to the card, or if additional hardware support is required to make things work.

I'll keep everybody posted.

Building bootable FreeBSD/i386 images

2007-10-16T23:46:00.000-06:00

From time to time I hear people complain at how hard it is to build an image from the FreeBSD sources. This week, I'll explain how I built a bootable i386 image on a USB flash device and also make some observations about the results.

Recently, I needed to create a bootable i386 image. The easiest way was to build one on my amd64. Here's what I did. I setup my environment for the build (steps 1-3), built FreeBSD/i386 userland and kernel (steps 4 and 5), prepared the flash (steps 6-11), installed userland and the kernel (steps 12 and 13), added the extra files needed for boot (steps 14-18). The detailed commands follow:

setenv MAKEOBJDIRPREFIX /blah
setenv TARGET i386
setenv TARGET_ARCH i386
make buildworld
make buildkernel KERNCONF=GENERIC
fdisk -I da0
fdisk -B da0
bsdlabel -w da0s1 auto
bsdlabel -B da0s1
newfs /dev/da0s1a
mount /dev/da0s1a /mnt
make installworld DESTDIR=/mnt
make installkernel DESTDIR=/mnt KERNCONF=GENERIC INSTALL_NODEBUG=t
make distrib-dirs DESTDIR=/mnt
make distribution DESTDIR=/mnt
echo /dev/da0s1a / ufs rw 1 1 > /mnt/etc/fstab
echo ifconfig_DEFAULT=DHCP > /mnt/etc/rc.conf
echo hostname=demo >> /mnt/etc/rc.conf

I put all this on an SD card and inserted that into an usb adapater and booted the laptop with it. FreeBSD's base system isn't too big these days, only 205MB for a full system.

% df /mnt
Filesystem 1024-blocks Used Avail Capacity Mounted on
/dev/da0s1a 484822 205570 240468 46% /mnt

This can easily be trimmed, but with a 512MB SD card for $5 at Office Max, there's little point in trimming for the project I needed the flash for. I just needed something that I could boot to transition my FreeBSD/amd64 laptop to a FreeBSD/i386 laptop.

If one needed to reduce the amount of space used by an installation, then there's a number of options than can be used to reduce the footprint of the system. In my experience, it is best to build everything without these options, then use the WITHOUT_* options on the installworld step to keep the image size down. It is easy to cut FreeBSD's footprint in half with these options. In addition, the kernel is 28MB with all its modules, and this can easily be reduced below 5MB in most cases.

nanobsd can be used to automate this process, as can TinyBSD. I'll save for another column the techniques I have used in the past to reach 16MB.

Cross debugger

2007-10-16T10:24:00.000-06:00

GDB Cross building on FreeBSD

I recently had to build a gdb that worked on our development host, but understood arm binaries and core files. It turns out to be fairly easy to do this in the FreeBSD tree. I did this on RELENG_6, but the same techniques will work for RELENG_7 or -CURRENT.

The Cookbook

% setenv TARGET_ARCH arm
% cd /usr/src/gnu/usr.bin/binutils
% make depend all
% cd ../gdb
% make depend all
% cd gdb
# make install

These steps assume one is using csh-like shell. Commands prefixed by '%' can be run with anybody with write privs to the source tree. Commands prefixed by '#' need to be executed by root. These instructions will install a gdb-${TARGET_ARCH} in /usr/bin that can be used to look at core files, or connect to a gdbserver on the target platform. As of this writing, the arm gdbserver glue hasn't been written or committed to FreeBSD's CVS tree. The author will happily review patches and commit them if they are good enough.

Running the debugger

You can run the debugger just like you would other binaries.

gdb-arm binary binary.core

to get a where listing of where one went off the tracks in binary 'binary' or look at a forced core dump.

That's it! Pretty simple, all in all, to create the cross debugger.

The cross-gdb port

As has been pointed out in feedback, one can also use the devel/gdb-cross port to make this even easier. I've not used this method. Maybe I'll investigate and report in another posting.

FreeBSD/arm on a new SoC

2007-09-29T03:29:00.000-06:00

Today I started work on porting FreeBSD/arm to a new SoC. I'm taking very detailed notes on this process, as well as attempting to learn from my atmel port how best to organize the port. I'll keep everybody posted and once I get far enough into it, I'll summarize the notes in a series of articles here. Hopefully this will help speed adoption of FreeBSD/arm on new hardware.

Speaking of new hardware, there's people working on a FreeBSD port to the NEO1973. Andrew Turner's email on the latest snapshot can be found here.

Merging *BSD's usbdevs

2007-07-07T16:54:00.000-06:00

Once upon a time, all the BSD's got their USB stack from NetBSD. With it came a usbdevs file. This file has grown and mutated on the different BSDs for some time. The time has come to merge them all back together. This sounds simple in theory, but in practice it is a lot more complicated. Different BSDs use different names for some vendors and some devices. Sometimes one is more correct than the other. Other times, they are merely both wrong. Still other times the names are the same, but the descriptions of the device are different. There's about ~2k lines in each of the files, and the merged file is closer to ~3k lines long.

I can do basically whatever I want to the FreeBSD usbdevs file to make this happen. The tricky part is justifying all the changes to a skeptical audience for the other projects. The other tricky part in merging is making sure that nothing breaks. Of course the final tricky part is getting the leg work done and the files committed quickly enough that they do not become stale. That's going to likely be the hardest part of all.

USB update

2007-06-23T21:48:00.000-06:00

I've finished much of the work on improving the in-tree USB stack, apart from adding device IDs. This is just the initial round. Now that all the cruft has been removed from the in-tree USB stack, we'll be able to support it better in RELENG_7. I plan on merging most of the client drivers into RELENG_6 after a couple of weeks, depending on how testing goes. So far, so good. I've added about 30 new devices. In fact, I've added so many that I've had no time to update the manual pages. If there are any doc people that can help synchronize the man pages to what we now support, that would be great.

After removing the obfuscating macros from the code, it is clear to me that adding some primitive locking to the USB stack might be possible now. It isn't going to happen for 7.0, however.

ooo builds!

2007-06-10T11:17:00.000-06:00

After the shared library version bump, the xorg 7.2 transition and the new gcc compiler in the base, I started to rebuild everything. After some snafu deleted all my +CONTENTS files, I've finally managed to rebuild everything (I think). The last major thing was ooo. After my laptop would shut down for being too hot a dozen times, and having to play whack-a-mole to remove all the extra bogus junk that's accumulated in /tmp, I've finally been able to do a build.

The biggest lesson learned, apart from how fragile java can be on amd64, is that ooo itself takes 8G of disk to build. That's /usr/ports/editors/openoffice.org-2/work only, not counting the dozens of packages it depends on. I thought I was being insanely pessimistic when I installed /tmp with 10G and put my ports tree there. Now I see that I'm right up on the edge. I'll have to repartition my disk, or get a bigger one.

Now, if only kino worked on FreeBSD/amd64 and current. It isn't even compiling for me these days :-(.

Minor USB cleanup

2007-06-09T00:55:00.000-06:00

I did a quick pass through the tree tonight eliminating all the devinfo stuff. This code had been implemented in uhub a while ago, but the client drivers were never updated to remove the bloat. We set the device name to usbd_get_devinfo in uhub, but then in the client drivers, we computed it again, and printed it, so the data would appear twice.

So far the only wrinkle is that kldunload/kldload doesn't set the description, but there are bigger problems there.

I hope to investigate a few of the bigger usb problems before 7.0 branches and get them fixed.

CardBus and USB minor improvements

2007-06-05T13:08:00.000-06:00

I've started working on some minor improvements to CardBus and USB. I'd like to see each of them using filters. CardBus and PC Card necessarily share interrupts between the bridge and the client devices. There's only one interrupt pin. Having to schedule an ithread just to check on the status of the card (for eject events) or power (for the interrupt driven power-up sequence) imposes a large overhead on the system. In addition, it may interfere with CardBus drivers that use filter. The current ISR isn't written to be a filter (what used to be known as a fast interrupt), so some work is needed to migrate it to such an interrupt.

For USB the problem is different. There's no interrupt sharing inside of USB, per se, since all the work goes through the pci<->usb host controller. However, on many systems the designers cheaped out and forced usb and the network to share interrupts. Since usb is still Giant locked, this causes a lot of extra Giant contention. The idea hear is to introduce a filter that turns off the interrupt bits and requests that the ISR run. The ISR would be exactly like we have today, with the addition of code to turn the interrupt sources back on. The theory here is that network inspired interrupts don't have to take out Giant at all since usb can schedule its ithread only when there's real work to do. If the ithread isn't scheduled, then Giant isn't taken out.

Since the freeze is coming up soon, I need to get this done in the next two weeks.

Warner's Random Hacking Blog

Canon Rebel Xt and 32GB CF

How to run something other than login on a tty

FreeBSD/mips for Cavium Octeon

New FreeNAS beta snapshot released (r5648)

New FreeNAS Beta (r5606)

FreeNAS 8 Beta released

Using devfs.conf as a porting aide

New FreeNAS alpha

PC-BSD install without a bootable DVD

FreeNAS 8 -- Update

My BSDTalk Interview

FreeNAS 8 -- First alpha snapshots

What is all the TBEMD stuff

The dangers of forgetting svn add

silly vpn hack

video on FreeBSD with skype

Booting instructions for FreeBSD on Cavium Octeon

DLINK DIR-615 REV C1 Redux

DLINK DIR-615 REV A redux

Post-mortum on projects/mips branch

Hack to allow automatic wired/wireless failover with lagg on FreeBSD

FreeBSD/mips updated

Some TiVo hacking

T-Mobile and iPhone 3.1.2 firmware update issue

Broadcom BCM43xx support (bwi) committed

Belkin F6D4230 "Enhanced Wireless Router"

ExpressCard

Update my old CardStatus page...

3Com 3C1 works

More PC Card enhancements

AX88x90 updates

Toshiba LANCT00A PC Card

Dumpster diving for PC Cards at Akihabara Junk Shops

AsiaBSDCon 2009

Melman Lives!

Alchemy Au1550 status update

More Video Creation hacks

Quick Mac OS X hack: Time Machine

Minor at91 hacking

Audio Ripping on FreeBSD

DLink DIR-615 Rev C1

More CardBus fixes

Cardbus Fixes

Moving towards board files

Cassio Fiva's screen dead

MeetBSD

Marvell ARM support in svn

gxemul update

SD/MMC work

FreeBSD/mips updated

FreeBSD embedded (mips, powerpc, arm) update

Sandbox pictures

Sandbox

Low hanging fruit

Summer tree cleaning

T-Mobile Dash experiment

Quick device_get_children cleanup

Mid august mips status update

Slimming down the NSLU kernel

PCI Bridge hacks

FreeBSD/avr32 spotted in the wild

The dog ate my homework.

Minor PC Card work

NSLU2 support committed to FreeBSD/arm

When worlds collide

MIPS Diffs

Minor wi(4) hacking

Taking a break from FreeBSD core team

Quick Guide to FreeBSD hostap for FreeBSD 8

Minor mips reorg and plans for the future

Adding a Serial Port to a D-LINK DIR-615

Old cards and the information gap

Good collection management software

Olicom OC-2220 works

Crystal Semiconductor Driver

Plat'Home provides OpenMicroServer

More mips support code committed

FreeBSD/mips entering the tree

Tokyo AsiaBSDCon 2008 Dev Summit Day 1