New FreeNAS beta snapshot released (r5648)

We've released the next beta snapshot of FreeNAS.

You can read the release notes for all the details.

Here's a highlight of the issues that we've fixed since the last beta (r5606):

• Upgrades from prior FreeNAS 8 beta releases are now supported.
• zfs creation failures have been fixed. The gui would indicate it succeeded, but you couldn't share it.
• ufs creation failures have been fixed.
• booting off USB or SCSI cdrom is now supported.
• Many disk initialization errors that showed up as odd failures have been corrected.
• lagg has been introduced.
• Errors in the network screen have been corrected.
• The storage wizard no longer says 'of X' when creating the storage unit. This eliminates the confusing 1 of 3 -> 2 of 2 dialog heading.
• Improved compatibility with IE and Safari.
• The installer has been streamlined.

Also, I was pleasantly surprised today when I logged in and found 19 comments from my last blog entry. I'm not used to so many, so hadn't been checking. I'll be checking more here. I'll also write a blog entry to address many of the issues raised in them.

You can download the latest from sourceforge.

New FreeNAS Beta (r5606)

Just after the release of the first beta, a couple of serious problems came to life. As such, we've respun the release to fix these issues.

• gmirror error displayed bogusly
• the extremely long write times for disks
• The lack of feedback during the installation (although with the significantly faster imaging, this is much less of an issue).
• Default ownership and permission for Windows shares has been fixed
• Shares are browseable by default now
• The extra and redundant steps in the installer have been eliminated
• The inconsistent menus in the installer have been made regular
• A ZFS issue on reboot has been corrected
• Multi-line config fields have been fixed
• Problems with snmp have been resolve
• proftpd's config file is now generated properly the second time
• minor nits fixed in many places

You can find the release notes or the release. Stay tuned for more updates.

FreeNAS 8 Beta released

iXsystems is pleased to announced FreeNAS 8.0 Beta. FreeNAS 8.0 has undergone a complete rewrite. We've redesigned the GUI to be easier to use and extend. We've upgraded many technologies in the system for improved hardware support, faster I/O, better modularity, and easier upgrades. We trust that you'll find the system easier to use and, in time, much more feature rich than the current FreeNAS offering.

The base system has migrated from FreeBSD 7.x and the m0m0wall build system to FreeBSD 8.1-RELEASE and NanoBSD. The system startup has migrated from the older php scripts to the standard FreeBSD rc.d boot system. We've pushed many of the bug fixes and system improvements back into FreeBSD.

We've rewritten the GUI using Python and Django. We've completely removed the old php system. In addition to Django, we're using Dojango and Dojo to implement AJAX features. The new system is much more modular than the old system. We will use this modularity in a future version for easy integration of custom features into your FreeNAS box.

The installer has been rewritten using pc-sysinstall, the future FreeBSD installation technology. The scripts have a similar feel to the old PHP scripts for users of the current system. The ISO now is only an installer. You can no longer run in live mode from a CDROM.

The installation types have changed; there's no longer an embedded or full install, nor can the image be installed on a data disk. You must now install FreeNAS onto a dedicated device. FreeNAS supports USB flash, CompactFlash, hard drives, ssd or any other mass storage device supported by FreeBSD.

FreeNAS 8.0 features ZFS version 14.

FreeNAS 8.0 beta has retained the core functionality of a storage appliance. The media center features of the box have not been reimplemented in the core FreeNAS package. A media center add-on package will provide this functionality in the future. We've focused on creating a robust, easy-to-use, and extensible system. We're creating the base to allow other types of packages to be added, such as printer support, scanner support, or home automation.

To help prioritize what current features are turned into packages in future FreeNAS releases, please visit http://support.freenas.org/ to provide feedback. Please add feature requests tickets. If a feature you would like to see in FreeNAS already has a ticket please just subscribe to it add a small comment, even if it's a "++." It will help us better judge and meet community needs.

You can download FreeNAS from Source Forge and read all the release notes on our wiki.

Using devfs.conf as a porting aide

I've recently been given the task of moving some software from FreeBSD 4.x to FreeBSD 6.x on a consulting basis. This move was very simple and easy, except for one area: serial device names.

In the great tty-rewrite to make it MP safe that was done, there were a number of name changes that were unavoidable when making the tty layer more uniform between drivers. Unfortunately, sio was one of the drivers that had its names changed. This company used many multi-port expansion cards, so finding all the places where /dev/cuaaX was hard coded was difficult.

To validate the rest of the software, I created the following /etc/devfs.conf file to allow the old names to work while I hunted down all the places they were used.

# 4.x compat names on 6.x
link cuad0 cuaa0
link cuad0.init cuaia0
link cuad0.lock cuala0
link ttyd0.init ttyid0
link ttyd0.lock ttyld0

These blocks were repeated for each of the serial devices in the system (I wrote a quick script to generate them for all 12 devices). One thing to remember is that the 11th device is named "/dev/cuada" not "/dev/cuad10", so you need to print the unit number as hex.

You could easily extend this to FreeBSD 7 the names changed again when FreeBSD moved from sio to uart.

# 6.x names on 7.x system
link cuau0 cuad0
link cuau0.init cuad0.init
link cuau0.lock cuad0.lock
link ttyu0 ttyd0
link ttyu0.init ttyd0.init
link ttyu0.lock ttyd0.lock

again, it is a simple matter to look over the unit numbers necessary.

Note, this will only work for devices that are statically created at boot time. If you have a number of usb serial devices on a newer FreeBSD system that are replacing an older sio based FreeBSD 4.x system, you may need to use devfs.rules to accomplish this transition, which is beyond the scope of this article.

New FreeNAS alpha

After a few unforeseen delays, we have a new FreeNAS snapshot available over at source forge. As an experiment, we are distributing these snapshots using xz as well as bzip2 compressed images. It makes a big difference.

What's new?

First and foremost, we have a completely new GUI look and feel. We've imported dojango into the GUI to take advantage of Dojo JavaScript Toolkit. The flow of the interface is much nicer, it looks better, and we've added additional help to make it easier to use. We think you'll like this new GUI. We've made dozens of improvements over the past few weeks to the GUI. We hope you like the new location for enabling shares.

We've also added support for Apple's AFP protocol to the new GUI. You can now use the GUI to export CIFS, AFP and NFS shares (which we'll be calling Windows, Apple and Unix shares in the GUI). We've added back-end support for FTP as well. The major protocols are now all supported, although some media options are still unimplemented.

This image has a universal booting feature. The first image was difficult to use on some VM systems since they were exclusively SCSI disks (appearing as da0, rather than the ad0 that was hard-coded into the fstab). We've added support for using UFS filesystem labels to nanobsd, so we're now able to produce an image that you can put on any kind of disk that FreeBSD supports and have it boot, since it finds the partitions for /, etc by looking for a label on the filesystem rather than insisting on ad0s1a.

At the present time, we support only adding NEW volumes to the system. If you add a volume to the system, we will always initialize it. This means that you shouldn't (yet) attach old volumes to FreeNAS 8. We will try to correct this problem for the forthcoming FreeNAS 8 beta. While we've made every effort to make things reliable, we've not put this software through a complete QA cycle, so it should be used for testing purposes only.

PC-BSD install without a bootable DVD

Recently, I tried to get PC-BSD onto an older system. This system didn't support booting off of USB, nor did it support swapping in a DVD player for the CD player that was shipped with the system. The computer seemed to be fast enough to support PC-BSD. Since all I really wanted was something that could play some games that my 4-year-old wanted, I wasn't willing to spend a ton of money on a new computer just yet. Better to wait a year or two until we really needed something better.

PC-BSD only installing off a memory stick or off a DVD these days. I opted to go the DVD route because this computer didn't support booting off USB media as far as I could tell (I tried it on two machines: an old HP eVectra 933MHz box, and a slightly newer Dell Optiplex GX270).

The summary of this hack is 'Copy all the files from /boot on the ISO image onto a hard disk that has been made bootable and use a USB expansion box to store the real DVD'.

First, I needed to create a bootable hard disk. I put the hard disk that was going to be installed into the system info a USB expansion box and plugged it into my FreeBSD system. This gave me a disk as 'da0'. This disk had an old Windows installation on it. All commands are run as root. You are advised to triple check your typing, as transposed letters and such might have adverse effects.

Relabeling the Disk

Since I had an old windows install on the disk, I just needed to delete the old windows partition and create a FreeBSD one.

1. gpart delete -i 1 da0
2. gpart add -t \!165 da0
3. gpart create -s bsd da0s1
4. gpart add -t \!7 da0s1

Did the trick. Your milage may vary at this stage. I've also used "gpart delete -i X da0" and "gpart destroy da0" to totally wipe a GPT scheme off the disk in the past. If you do that, you'll also need "gpart create -s mbr da0" before proceeding to step 2 above. There's lots of other ways to skin this cat too involving dd, but since gpart has been added to the system, its so easy to use I prefer it for quick tasks like this so I don't have to remember "do I need to blank the front of the disk, or the end of the disk..."

Making the disk bootable

Next we need to make the disk bootable. This is a lot easier than it used to be in days of yore:

1. gpart bootcode -b /boot/boot0 da0
2. gpart bootcode -b /boot/boot da0s1

We have to make it bootable twice due to the multi-stage boot-loader that FreeBSD has (other systems are likely similar). "boot0" just prompts for which partition to boot (doing the first one by default). "boot" will then load /boot/loader to finish out the booting process and hand control over to FreeBSD's kernel.

Copying The Files

The pc-bsd installer for the DVD just loads a ram disk, and then runs off that. All the early stages of the boot loader need are contained in /boot. This includes the kernel, the third stage boot loader and various config files and scripts. So the next steps are to copy these files from the cd onto the bootable hard disk we just made. This assumes that the mdconfig command below prints "md0" for unit 0. If it prints anything else, you'll need to adjust accordingly. Also, if you are reading this in the future, you may need to change the PCBSD iso name.

1. newfs /dev/da0s1a
2. mount /dev/da0s1a /mnt
3. mdconfig -f PCBSD8.1-x86-DVD.iso
4. mount -t cd9660 /dev/md0 /cdrom
5. mkdir /mnt/boot
6. cd /cdrom/boot
7. tar cf - . | (cd /mnt/boot; tar xvf -)
8. cd /
9. umount /mnt
10. umount /cdrom
11. mdconfig -d -u 0

Moving the disk

At this point, I had to unplug the USB expander from the first system, remove the hard drive and install it into my target box. I also grabbed my UBS DVD player and connected it to the box with a copy of the DVD I burned from the above file. I selected hard disk as the boot media in the bios and let the system boot. It booted off the hard disk, but then started looking for the proper cd device. Luckily for me, it found the DVD on cd1 instead of cd0 (which is what was built-in to the Dell I was booting off of). It then became a 'normal' PC-BSD installation, which is described in detail elsewhere.

Hopefully, people find these instructions useful. Since I only had one of these systems, I didn't create any kind of script to make it easier. Sorry.

[[ Updated to fix problem noted in the comments ]]

FreeNAS 8 -- Update

If you are following the commit messages to FreeNAS, you'll already know that FreeNAS 8 is shaping up nicely. If not, allow me to summarize some of the progress here.

In the past two weeks we've managed to break the gui completely, and put it back together. In the process, we've moved from a fairly static system to a more dynamic DOJO driven system that's much easier to use. It is less cluttered, while still providing escape hatches to the more complicated stuff. In addtion, we've written an installer (really, used the installer from PC-BSD, now part of FreeBSD-current). We've upgraded all the ports to FreeBSD 8.1-RELEASE (basically about 6 weeks newer ports). We'll need to do security updates before beta on some of these as well, so we've added the ability to patch the ports tree (which should also mean we'll retain much of the flexibility that the legacy FreeNAS system has wrt upgrades).

All of this is building to a new alpha snapshot. We're nearing the feature complete phase and want to get at least one more alpha out the door before we go to beta, most likely early next month. Watch this space for updates.

If you've been trying to follow the process at a source code basis, you may have run into a problem. While the build system does a good job at applying new patches to FreeBSD's src and ports tree, it doesn't do a very good job when the patch has been updated. Its unclear to me what the best way to manage this process is, since it can be hard to manage a new ports tree, new src and ports patches, as well as preserve the '-b' functionality of nanobsd which allows for fast builds after an initial build.

So, if you encounter problems updating your tree from an earlier snapshot, there's an easy work around. If you define 'force_update=1' in your environment, then you'll force the do_build.sh script to update your src and ports trees (it is needed if you have a June 24th ports tree from before the cut over). This will force csup to run, which 'fixes' all the files that have been patched. I've enhanced do_build.sh so that it will force all the patches to be reapplied when you do this. It is best to avoid the -n and -b flags to do_build.sh when doing this, since you really want a fresh build after the latest series of changes.

Later this week, we'll have another alpha snapshot ready. The last bit of GUI work should be done tomorrow (we need to add back the ability to control sharing/exporting of volumes which was lost in the great GUI reshuffle). We'll also need to test it to make sure the basics work and then we'll put binaries up to share. The installer may even be ready too, so that might also include ISO images you can boot and use to install the images into a CF.

Finally, we've reworked Olivier Cochard-Labbé's to nanobsd for device label support to be more flexible. As a result, they are now in both FreeNAS, as well as in FreeBSD. The practical upshot is that now a single image can boot off CF (where the root device is /dev/ad0s1a), a VirtualBox container (where it might be /dev/ada0s1a or /dev/ad0s1a) or a USB Stick/VMWare container (where it would be /dev/da0s1a). This should solve some difficulty people reported using VMWare in the first alpha, as well as provide a nice, flexible system to have a single image for all media types going forward. Briefly, we can label things within NanoBSD so they appear as /dev/ufs/FreeNASs1a so that the root filesystem can be found on any underlying media. If you're interested in the details of this, check out glabel and the -L option to tunefs and newfs.

As always, we welcome feedback. We'll have something in place soon to help collect and organize the feedback from the beta tests. I'll announce it after we go live.

My BSDTalk Interview

I just gave Will Backman over at BSDTalk an interview on FreeNAS 8, NanoBSD, ZFS and embedding FreeBSD. You can check it out at the BSDtalk blog. Hope you all enjoy.

Another FreeNAS Alpha snapshot is slated for early this week. Stay tuned. It will include a revamped user interface, local user support, FreeBSD 8.1-RELEASE ports and a few other goodies.

FreeNAS 8 -- First alpha snapshots

My blog has been somewhat quiet lately. I've been busy with other things lately.

The biggest of the other things is FreeNAS. iXsystems stepped up to the plate late last year when FreeNAS was going to become a Linux based platform.

The iXsystems engineering team has moderized FreeNAS in a number of ways. We wanted a platform that was more extensible than the current m0m0wall-based framework allowed. We wanted to create a platform that could be expandable by modules (possibly not even written by us). We wanted to make it easier to upgrade the base FreeBSD release, as well as leverage more base FreeBSD technology that has been integrated into the system since FreeNAS was originally developed.

We've migrated the build to be NanoBSD based. This allows us to leverage the embedded work that has gone into NanoBSD. It also allows us to push some of the features that are important to FreeNAS back into the base FreeBSD distribution. NanoBSD gives us the flexibility that we need. Since we're using the FreeBSD package system to add ports and packages, users will be able to add their own packages (we'll likely expand the basics to use the PBI's that PC-BSD produces for ease of installation). We're using the normal rc.d system, so upgrading is easier as well.

On the GUI side we've moved to using django. This is a python-based front-end to sqlite3 which is extensible and easy to use. While we've kept things rather plain so far, we plan on jazzing things up a bit now that we have the basics working fairly well. Since the GUI is based on django, we'll be able to allow packages to hook into the GUI to present a united interface to the user.

The GUI calls into a middleware layer that manages the services on the box. This middleware is responsible for generating updated config files, as well as starting, stopping and restarting daemons on the system. The same code generates the files at boot as well. Having all the configuration data in one database makes it easier to upgrade from release to release, since you don't have to merge your changes into the config files: the system takes care of that details.

If you'd like to read about the nuts and bolts about trying out the latest snapshot, you can check out my forum post on the topic over at the FreeNAS forums.

What is all the TBEMD stuff

I've been getting questions about what TBEMD means in my commits to FreeBSD head.

First, I'll tell you the name. TBE stands for TARGET_BIG_ENDIAN. For the MIPS and ARM ports, it is an environment variable that you have to set to build these targets for big endian (otherwise they default to little endian). MD is Must Die. A few years ago, there was an action movie whose title was more memorable than the file "Romeo Must Die." So I just stole the name for this branch.

So why must TARGET_BIG_ENDIAN die? It breaks things. For all the other platforms that FreeBSD supports, you just set TARGET and/or TARGET_ARCH to do cross builds. You know everything you need to know from the MACHINE and MACHINE_ARCH in the resulting image. In addition, the build system segregates things so you can build all the targets in one tree..But for mips and arm, you don't know what endian a binary is, the obj tree will collide if you try to build both little endian and big endian binaries at the same time. There's also other, more subtle issues. For example, TARGET_BIG_ENDIAN isn't set on big endian mips, so a native biuldworld tries to build little endian binaries (oops).

So, mips will be moving to mipsel (for little endian) and mipseb (for big endian). Arm will move to arm (for little endian) and armeb (for big endian). You could fill an entire email archive with all the possible other names and why they are better or worse than these names. History, however, trumps all those arguments: these are the names used elsewhere and we're just following convention.

Once tbemd is completely merged, you'll be able to build both endians of MIPS in the same object tree, for example, and all the other issues I've discovered. It also helps with new architectures as we move into powerpc64 (already merged) and mips64 (to be done after tbemd is collapsed). It will also mean we can have different packages for the different endians now.

The dangers of forgetting svn add

I spent a lot of time before the 8.1 release adding code to transparently map COMPAT_IA32 to COMPAT_FREEBSD32. I tested all kinds of different scenarios to make sure that it works, that it got error messages from old config programs, etc.

Today I got a complaint that COMPAT_IA32 doesn't work in 8.1-RELEASE. Linux module was failing to load properly. I thought it was really odd and that the folks complaining must be doing something wrong.

It turns out that I did something wrong. When I originally merged the work, I modified the options file, but didn't add the new options-compat file to stable/8 tree, so it didn't get merged into the releng/8.1 tree used to create 8.1. The result is that COMPAT_IA32 just fails silently in 8.1-RELEASE.

All that hard work ruined because I forgot to do a svn add... How disappointing.

silly vpn hack

I finally got frustrated by the inability of /etc/resolv.conf to do what I'd say in pseudo code:

if domain==example.com; then nameserver=1.2.3.4
else nameserver=4.3.2.1

since I need that when I'm on my VPN to example.com.

So, round one of the hack is to tell dhclient to use my local nameserver, and do it with named.

So, in /etc/dhclient.conf, I have:

interface "wlan0" {
prepend domain-name-servers 127.0.0.1;
}

and I have the following snippets in named.conf:

...
options {
...
forwarders {
4.3.2.1;
};
...
};
zone "example.com" IN {
type forward;
forwarders {
1.2.3.4;
};
};

The only problem I have yet to figure out is how to dynamically update the 4.3.2.1 in /etc/namedb/named.conf as I roam from network to network. There doesn't appear to be a hook in dhclient to say 'run this command when my IP address changes' or 'run this command when I get a new lease' or anything useful like that....

All this is with FreeBSD-current, but these techniques should work back to at least FreeBSD 6.x.

Finally, does anybody know a good way to quote code (as opposed to text) in blogger.com? By default, it seems, all leading whitespace is eaten, which make this post look ugly.

video on FreeBSD with skype

Hans Peter Selasky has been very busy adding support for web cams and other usb video devices. We tried a number of different cameras that we had laying around iX Systems the other day. All of them were recognized as valid video cameras. Some of them worked with pwcview and skype. Some of them had problems with the palettes, or would only work with some resolutions.

I was able to make a Skype call on my FreeBSD to another developer. Unfortunately, the distfile necessary for this isn't freely available. You have to download it from Skype. And it is no longer available from Skype. I got lucky and down loaded it a while ago, thinking I'd play with skype.

If you haven't had a chance to check it out, install the multimedia/webcamd. There's lots of devices that it works with.

Now, the interesting part. Hans Peter Selasky has implemented a user land driver for all of this. He implements a device driver in two parts. One piece in the kernel, which setups the shims to talk to webcamd, and the actual driver in webcamd. He uses the Linux drivers in webcamd. This separates the GPL drivers from Linux from the BSD kernel. So there's no issues with licenses. This is the same technique that's done in the Linux world to implement proprietary drivers and keep them separate from the Linux kernel. This opens the door for a wide array of devices. It is useful to think of this as something similar what Bill Paul did with NDIS and Windows drivers (except in two different address spaces rather than one).

Booting instructions for FreeBSD on Cavium Octeon

Here's a quick note on how to net boot the Cavium EBT3000 board running uboot. The Cavium kernel is still a work in progress as I restore all the fixes I made to an earlier version of this code that I was unable to release.

You'll need to break into the boot sequence for this board. Usually that's just hitting ^C before uboot starts to load the kernel.

Once you have a uboot prompt, similar to below, you'll be ready to start to fix the environment for netbooting.

U-Boot 1.1.1 (Development build) (Build time: Jan 26 2007 - 14:06:35)

EBT3000 board revision major:4, minor:0, serial #: xxxxxxxxx
OCTEON CN38XX-NSP revision: 1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)
DRAM: 2048 MB
Flash: 8 MB
IPD backpressure workaround verified, took 14 loops
Clearing DRAM........ done
BIST check passed.
Net: octeth0, octeth1, octeth2, octeth3
Bus 0 (CF Card): OK

ide 0: Model: SanDisk SDCFB-1024 Firm: Rev 0.00 Ser#: X0308 20050726142909
Type: Removable Hard Disk
Capacity: 977.4 MB = 0.9 GB (2001888 x 512)

Octeon ebt3800#

It couldn't be easier to setup for netbooting. You'll need to setup an tftp server. I had to install the FreeBSD port from net/freebsd-tftpd to get a working tftpd, since there appears to be some issues with the stock FreeBSD tftpd. Once I had that, I set different addresses like so:

set ipaddr 10.0.0.30
set netmask 255.255.255.0
set serverip 10.0.0.10
set gatewayip 10.0.0.1
set bootdelay 0
saveenv

This sets the board's address to 10.0.0.30/24. 10.0.0.1 is the router to the other networks. The tftp command will now use tftp from host 10.0.0.10 to get the named file. saveenv makes these setting permanent. I usually reboot at this point to make sure that the settings took.

Next, you'll need to know how to load a FreeBSD kernel. This is also fairly straight-forward, although finding the right addresses took a bit of guesswork. You are tftping the ELF kernel to a chunk of memory. The bootoctlinux command then takes that image and copies it to the right place and starts executing it. So you have to be sure whereever you load it doesn't clash with uboot's memory use or with where the kernel will ultimately reside.

tftp a800000 kernel.octean1-32
bootoctlinux a800000 numcores=1

As with any u-boot platform, it can often be useful to define macro commands. I did the following:

set fbsd 'tftp a800000 kernel.octeon1-32; bootoctlinux a800000 numcores=1'

so that I don't have to keep typing it over and over. I can just type

run fbsd

to run my next test.

The Cavium port is in a state of flux right now. The latest status as I am preparing this to be published is that there's an interrupt problem that's causing the serial port to be non-responsive after going to multi-user (eg, the login: prompt) and also the network stops responding. Also, I have to comment out the enableintr() call in mips_vector_init around line 365 of sys/mips/mips/machdep.c. I'm trying to hammer these issues out, and I think they are related. Earlier versions of the code worked with earlier versions of FreeBSD, so I'm sure this is just a shift in interfaces that needs to be accounted for.

Here's the latest slightly trimmed log from a boot:

U-Boot 1.1.1 (Development build) (Build time: Jan 26 2007 - 14:06:35)

EBT3000 board revision major:4, minor:0, serial #: xxxxxxxxx
OCTEON CN38XX-NSP revision: 1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)
DRAM: 2048 MB
Flash: 8 MB
IPD backpressure workaround verified, took 32 loops
Clearing DRAM........ done
BIST check passed.
Net: octeth0, octeth1, octeth2, octeth3
Bus 0 (CF Card): OK

ide 0: Model: 512MB CKS Firm: 2N3-0925 Ser#: 2435E7C34A52930007
Type: Removable Hard Disk
Capacity: 491.6 MB = 0.4 GB (1006992 x 512)
Octeon ebt3000# print
bootdelay=0
baudrate=115200
download_baudrate=115200
ls=fatls ide 0
nuke_env=protect off BFBFE000 BFBFffff; erase BFBFE000 BFBFffff
autoload=n
ipaddr=10.0.0.31
gatewayip=10.0.0.1
netmask=255.255.255.0
serverip=10.0.0.16
fbsd=tftp a800000 kernel.octeon1-32; bootoctlinux a800000 numcores=1
fbsd_cf=fatload ide 0 a800000 kernel.imp; bootoctlinux a800000 numcores=1
coremask_override=0xffff
numcores=16
stdin=serial
stdout=serial
stderr=serial

Environment size: 1274/8188 bytes
Octeon ebt3000# run fbsd
octeth0: Down 10Mbs Half duplex, (port 16)
Using octeth0 device
TFTP from server 10.0.0.16; our IP address is 10.0.0.31
Filename 'kernel.octeon1-32'.
Load address: 0xa800000
Loading: octeth0: Up 100Mbs Half duplex, (port 16)
########################
done
Bytes transferred = 3370489 (336df9 hex)
argv[2]: numcores=1
ELF file is 32 bit
Skipping non LOAD program header (type 0x6)
Skipping non LOAD program header (type 0x3)
Skipping non LOAD program header (type 0x70000000)
Allocated memory for ELF segment: addr: 0x1000000, size 0x295a60
Loading .text @ 0x81000000 (2064384 bytes)
Loading .MIPS.stubs @ 0x811f8000 (16 bytes)
Loading .rodata @ 0x811fa000 (33248 bytes)
Loading .reginfo @ 0x812021e0 (24 bytes)
Loading .rodata.str1.4 @ 0x812021f8 (189300 bytes)
Loading set_sysctl_set @ 0x8123056c (3496 bytes)
Loading set_sysinit_set @ 0x81231314 (1684 bytes)
Loading set_sysuninit_set @ 0x812319a8 (952 bytes)
Loading .interp @ 0x81231d60 (13 bytes)
Loading .dynsym @ 0x81231d70 (78320 bytes)
Loading .dynstr @ 0x81244f60 (72163 bytes)
Loading .hash @ 0x81256944 (35984 bytes)
Loading set_kdb_dbbe_set @ 0x8125f5d4 (8 bytes)
Loading set_modmetadata_set @ 0x8125f5dc (288 bytes)
Loading set_cons_set @ 0x8125f6fc (8 bytes)
Loading .data @ 0x8125f704 (116476 bytes)
Loading set_pcpu @ 0x8127be00 (3136 bytes)
Loading .got @ 0x8127ca40 (5296 bytes)
Loading .rld_map @ 0x8127def0 (4 bytes)
Loading .sdata @ 0x8127def4 (12 bytes)
Clearing .bss @ 0x8127df00 (97120 bytes)
## Loading Linux kernel with entry point: 0x81000000 ...
Bootloader: Done loading app on coremask: 0x1
Boot Descriptor Ver: 6 -> 1/2 CPU clock: 500MHz
Dram: 2048 MB Board Type: 2 Revision: 4/0
Octeon Chip: 0 Rev 0/0 Mac Address 00.0F.B7.10.1B.32 (14)
octeon_dram == 80000000
reduced to ram: 2048 MBReal memory bytes is 7ffff000
phys_avail[0] = 1296000 phys_avail[1] = fffffff
realmem_bytes is now at 5ffff000
Next block of memory goes from 20000000 to 7ffff000

Total DRAM Size 0x80000000
Bank 0 = 0x 1296000 -> 0x FFFFFFF
Bank 1 = 0x20000000 -> 0x7FFFF000

physmem: 0x6ed69Cache info:
picache_stride = 4096
picache_loopcount = 8
pdcache_stride = 128
pdcache_loopcount = 64
cpu0: Cavium processor v1.0
MMU: Standard TLB, 32 entries
L1 i-cache: 4 ways of 64 sets, 128 bytes per line
L1 d-cache: 64 ways of 1 sets, 128 bytes per line
Config1=0xbe3303da
Config3=0x80
Physical memory chunk(s):
0x1296000 - 0xfffefff, 248942592 bytes (60777 pages)
0x20000000 - 0x7fffefff, 1610608640 bytes (393215 pages)
Maxmem is 0x7ffff000
KDB: debugger backends: ddb
KDB: current backend: ddb
hz=100 cyl_per_hz:250000 cyl_per_usec:250 freq:250000000 cyl_per_hz:2500000 cyl_per_sec:250000000
Copyright (c) 1992-2010 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-CURRENT #141 r202986:202997M: Mon Jan 25 20:07:07 MST 2010
imp@paco-paco.bsdimp.com:/tmp/imp/be-obj/mips/pe/imp/svn/head/sys/OCTEON1-32 mips
real memory = 1859555328 (1815972K bytes)
Physical memory chunk(s):
0x013ab000 - 0x0fffefff, 247808000 bytes (60500 pages)
0x20000000 - 0x7d830fff, 1568870400 bytes (383025 pages)
avail memory = 1813180416 (1729MB)
mem:
nfslock: pseudo-device
null:
nexus0:
Compact flash found in bootbus region 3 (8 bit).
rgmii0: on nexus0
rgmx0: Ethernet address: 00:0f:b7:10:1b:32
rgmx0: type (null), full duplex
rgmx1: Ethernet address: 00:0f:b7:10:1b:33
rgmx1: type (null), full duplex
rgmx2: Ethernet address: 00:0f:b7:10:1b:34
rgmx2: type (null), full duplex
rgmx3: Ethernet address: 00:0f:b7:10:1b:35
rgmx3: type (null), full duplex
rgmii0: [FILTER]
cf0: on nexus0
clock0: on nexus0
clock0: [FILTER]
obio0 at mem 0x1 flags 0x1 on nexus0
uart1: on obio0
uart1: [FILTER]
uart1: fast interrupt
uart1: console (115200,n,8,1)
uart0: on obio0
uart0: [FILTER]
uart0: fast interrupt
uart0: console (115200,n,8,1)
Device configuration finished.
Reducing kern.maxvnodes 117045 -> 100000
Timecounter "MIPS32" frequency 250000000 Hz quality 800
Timecounters tick every 10.000 msec
GEOM: cf0: partition 2 does not start on a track boundary.
GEOM: cf0: partition 2 does not end on a track boundary.
GEOM: cf0: partition 1 does not start on a track boundary.
GEOM: cf0: partition 1 does not end on a track boundary.
GEOM: cf0s2: geometry does not match label (64h,32s != 16h,63s).
Trying to mount root from ufs:cf0s2a
WARNING: / was not properly dismounted
warning: no time-of-day clock registered, system time will not be set accurately
start_init: trying /sbin/init
Setting hostuuid: 1c99760e-1dd2-11b2-bd6a-00156dc12838.
Setting hostid: 0xb0f278c1.
No suitable dump device was found.
Entropy harvesting:.
Starting file system checks:
t_delta 16.010fac783006c1a6 too long
/dev/cf0s2a: 11100 files, 256973 used, 150946 free (426 frags, 18815 blocks, 0.1% fragmentation)
Mounting local file systems:.
t_delta 15.fece5003dd62224a too short
Setting hostname: ebt3000.
SIOCSTIFFLAGS UP/Running
Starting Network: lo0 rgmx0 rgmx1 rgmx2 rgmx3.
lo0: flags=8049 metric 0 mtu 16384
options=3
inet 127.0.0.1 netmask 0xff000000
rgmx0: flags=8843 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:32
inet 10.0.0.31 netmask 0xffffff00 broadcast 10.0.0.255
media: Ethernet autoselect
rgmx1: flags=8802 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:33
media: Ethernet autoselect
rgmx2: flags=8802 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:34
media: Ethernet autoselect
rgmx3: flags=8802 metric 0 mtu 1500
options=3
ether 00:0f:b7:10:1b:35
media: Ethernet autoselect
Starting devd.
Creating and/or trimming log files.
Starting syslogd.
/etc/rc: WARNING: Dump device does not exist. Savecore not run.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
Clearing /tmp (X related).
Updating motd:.
Starting cron.
Starting inetd.
Starting background file system checks in 60 seconds.

Tue Jan 26 02:07:16 UTC 2010

FreeBSD/mips (ebt3000) (ttyu0)

login:

DLINK DIR-615 REV C1 Redux

After hitting a dry hole with my DIR-615 REV A1, I thought I'd try the DIR-615 REV C1 that I have. It has an Atherose AR9130 in it. This is very similar to the AR7130 that the RouterStation I have running, and I thought it would be a simple matter to use uboot to netboot this code.

Again, I was mistaken. There's no way to break into the boot sequence for this device. The best I can hope for is to create a compressed image that's < 1MB and blow it into the FLASH partition for the kernel under linux. That would work out OK, if I had known good image to start from. Alas, I don't. And there doesn't appear to be an accessible JTAG header that I can use to write a new u-boot. Linux seems to protect u-boot from being overwritten as well.

But the biggest issue of all is that the FLASH is only 4MB in size. It would take some doing to get everything shrunk down that small. The DIR-615 REV A1 at least had 8MB of FLASH, which is doable. Maybe I'll perfect the kernel for the DIR-615 REV A1 on the Marvell eval board I have. Looking at the Linux sources, the deltas between these kernels are small and inconsequential for bootstrapping purposes. Hmmm, the Linux kernel just has support for the DNS-323, now that I look closely, but the required bits of hacking after I have a known good kernel doesn't seem huge. Plus, if I could get one good kernel on the box, then I could put a better uboot on there which would support the network for netbooting...

If I ever make good images for these boards, I'll be sure to post pointers to them here.

DLINK DIR-615 REV A redux

Yesterday I burned a couple of hours playing with the DIR-615 REV A1 that I have had for ages. FreeBSD runs on the Marvell Orion SoC that's inside it now and I thought I'd netboot the box and try to put together an image that can replace my firewall since this little box uses less power and my firewall doesn't need to do *THAT* much.

Unfortunately, it appears as if the network was purposely crippled on this box. Not disabled entirely, but crippled to not work. Sometimes you can get packets out, but generally there's a huge packet loss rate that makes pinging impossible, let alone netbooting.

Talking to other folks that have worked in this area, it appears that this was a deliberate attempt to discourage others from running alternative operating systems on the box.

Since the box will still boot to Linux, there's hope that one can bootstrap the FreeBSD installation from there. When I get some time again, I'll be pursuing that avenue. Maybe I can find the JTAG header for this board and reflash uboot that way, since debugging a new kernel can be difficult to get right in one shot on hardware the kernel has never run on before...

I had planned on publishing instructions for netbooting the thing, but alas, it appears that isn't possible.

Post-mortum on projects/mips branch

Greetings to one and all. As you have read elsewhere, I recently merged all the changes from the projects/mips branch onto head. In other reports, I've made cryptic reference to the branch being damaged. I thought I'd go through all the problems we encountered running this development effort on the projects/mips branch.

First, we created the branch in the normal way:

svn cp svn+ssh://svn.freebsd.org/base/head svn+ssh://svn.freebsd.org/base/projects/mips

and then started committing to it. So far so good.

Then it came time to merge in changes from head. Here is where we made our first mistake. We merged just the kernel changes, and not the entire tree. The next time we merged the entire tree. The effect of this sequence meant that the changes between the creation and the first merge in the rest of the tree (outside of src/sys) were lost. I don't know if this is the result of a buggy svn client, or if they are a fundamental flaw in svn. So when it came time to collapse the results back into head, we couldn't just do a reverse integration, or even a more conventional merge.

Next, we pulled in pieces of the projects/mips tree into head, either by hand or with the svn merge command. Doing both was a mistake, I think. Although svn coped with later merged from head into projects/mips fairly well, occasionally we'd have issues to sort out.

So, when it came time to merge everything back into head, we were left with a number of difficult choices. We opted to copy the new files/directories in the repo (to preserve the history) and merge patches by hand with svn log entries. The latter went really well. There were no problems with it apart from the odd fuzz and .rej file to sort out (no different than a normal merge). The former, however, causes a lot of problems.

First, it created svn:mergeinfo entries on the files I copied over. This conflicts with the project's "merge everything from sys" edict. This happened because I did the copy in the server rather than the client (which was an attempt to preserve history). It was a simple matter to delete these entries when it was brought to my attention.

Next, somehow we created the files on the branch without the required svn:keywords, svn:eol-style and svn:mime-type properties. This caused problems when we went to commit changes to these files. The precommit complained, at this point, that the files lacked svn:keywords, and we had to add $FreeBSD$ by hand (since the copy didn't complaint that they were lacking). Also, I didn't discover these problems on my own. Other sharp-eyed individuals saw the problems and reported them to me.

Finally, since I did the copies on the server, there was no way to batch them. When I copied a directory, I got the whole directory. When I copied a file, I got the file. Each one generated a commit message.

I'm unsure what other damage was lurking in the projects/mips tree, but the lack of properties, the inability to easily use merge info and the missing commits lead me to the conclusion that it was easier to abandon the branch and create a new branch if needed in the future.

The mips team plans to create more branches, that are more for special purposes, that will be shorter lived in the future. We have no further plans to have one monolithic mips branch that acts as a staging ground for commits into head.

Hack to allow automatic wired/wireless failover with lagg on FreeBSD

For years I've had a love/hate relationship with my wireless card. On the one hand, it allows me to roam. On the other I like being able to plug into the wired Ethernet when I'm sitting at my desk since I get better throughput and lower latency. So for years, I just loaded and unloaded kernel modules to make dhclient (and/or wpa_supplicant) do its thing. This changed the IP address of my box, which messed up NFS mounts until I went back to the old way.

Recently, Josh Paetzel came up with a way to use lagg to do automatic failover. Inspired by the lagg man page, he created a number of rc.conf lines that would do this automatically. Xin Li and I refined the trick a bit. To completely integrate this into FreeBSD, I suppose that the rc.d scripts would need to be enhanced to automatically generate these lines when appropriate. However, the hack is so easy to do now, I'm going to share it with you here. Xin Li is writing up a longer description of how to set this up for the handbook. Let's say you have a ath0 wireless interface, and a re0 wired interface, just add the following to your /etc/rc.conf file:

• ifconfig_re0="up"


• ifconfig_ath0=`ifconfig re0 ether`


• ifconfig_ath0="ether ${ifconfig_ath0##*ether }"


• wlans_ath0="wlan0"


• ifconfig_wlan0="WPA"


• cloned_interfaces="lagg0"


• ifconfig_lagg0="laggproto failover laggport re0 laggport wlan0 DHCP"

This bit of code brings up re0, programs ath0 with the ethernet MAC address of re0 (so your MAC doesn't change when you switch over), creates a wlan0 interface from ath0, causes lagg0 to be created and configured to do the fail over.

Hope this tip is useful...

Revision at Wed Jan 13 16:00:00 UTC 2010:

Note: "ifconfig re0 ether" doesn't work in versions prior to 8.0. The following should fix it, but I don't have a system to test it on:

ifconfig_ath0=`ifconfig re0 ether`

ifconfig_ath0="${ifconfig_ath0##*ether}"

ifconfig_ath0="ether ${ifconfig_ath0%%inet*}"

This sequence avoids using grep, awk and/or sed to accomplish this (and thus touch /usr too early for systems where / and /usr are on different partitions).

Also corrected the bwi0 reference in the feedback...

Revision at Mon Jan 18 23:00:00 UTC 2010:
Corrected () vs {} confusion in the above...

FreeBSD/mips updated

The base/projects/mips branch has been merged into base/head. The merge is complete and the sanity tests have passed. The code has booted on both a Ubiquiti RouterStation (big endian) as well as in gxemul (little endian).

The branch lived for one year, minus a day, and accumulated much work:

• A new port to the Atheros AR71xx series of processors. This port supports the RouterStation and RouterStation PRO boards from Ubiquiti. Other boards should work with minimal tweaking. This port should be considered as nearing production quality, and has been used extensively by the developers. The primary author of this port is Oleksandr Tymoshenko (gonzo@freebsd.org).


• A new port to the sibyte BCM1250 SoC on the BCM91250 evaluation board (aka SWARM). This port is reported to be stable, but this hardware is a little old and not widely available. The primary author of this port is Neel Natu (neel@freebsd.org). Only one core is presently supported.


• A port, donated by Cavium, to their Octeon and Octeon plus series of SoC (CN3xxx and CN5xxx). This code is preliminary, supporting only a single core right now. It has been lightly tested on the CN3860 evaluation board only in 32-bit mode. Warner Losh (imp@freebsd.org) has been driving the efforts to get this code into the tree.


• A port, donated by RMI, to their XLR series of SoCs. This port is single core only as well. The code reaches multi-user but should be considered beta quality for the moment. Randal Stewart (rrs@freebsd.org) has been driving the efforts to integrate this into the tree.


• Preliminary support for building a mips64 kernel from this source base. More work is needed here, but at least two kernels successfully build in 64-bit mode (OCTEON1 and MALTA64).


• Very early support for N32 and N64 ABIs


• Support for booting compressed kernels has been added (gonzo@).


• Improved support for debugging


• Improved busdma and bus_space support


• Many bug fixes


• More types of MIPS cores are recognized


• Expanded cache handling for newer processors


• Beginning of a port to the alchemy au1XXX cpus is present but experimental.


• Work on SMP is underway to support multicore processors like the sibyte, Octeon and XLR processors.

I'm sure there are minor items I've forgotten. If so, please forgive any omission on my part...

The branch had been updated incorrectly several times over the past year, and the damage was too much to repair. We've retired the branch and will do further mips development in "head" for the time being. If you have a checked out tree, the suggested way to update the projects/mips tree you have is to do a "svn switch svn://svn.freebsd.org/base/head" in that tree.

I'd like to thank everybody that has contributed time, code or hardware to make FreeBSD/mips better.

We are still investigating how feasible merging all this work into stable/8 will be, as it represents a huge leap forward in code stability and quality.

As development proceeds, I'll keep posting updates. In addition, I hope to have some mini "how-to" wiki pages done for people that want to try it out.