Arm testing

Testing before the branch

So, tonight I tested 4 images that were 12.0 snapshots. I downloaded all the arm images that the re@ produces. The version string was 20180719-r336479.

GUMSTIX is impossible. There's no hardware it could run on, so I ignored it. It has an GUMSTIX XSCALE kernel, but a new GUMSTIX Duovero uboot and armv4 user land. Complete no go.

WANDBOARD and RPI2 booted on their hardware. I was able to dd the images and put them into my existing test bed. They just worked (but I haven't done more than just test boot, so caveat emptor).

I recently bought a new Raspberry Pi 3 B. I had a couple of RPi 3 (v1.2). I couldn't get any of them to boot at all on the RPI3 arm64 image. No output at all. [Update: thinking it may be a bad uSD, so will try again]

I also tried my old Pandaboard. It didn't work: no valid DTB was found in ubldr. [update: after selecting a good DTB, I get a core dump attaching ti_sdma_attach when calling device_get_softc()\

Later, I tried the CUBIEBOARD config on an actual Cubieboard. It worked too.

[UPDATE] PINE64 worked on my PINE64 kick-starter board. Woot!

On a related note, it was a sad day. The RPi3 B replaced my old Atmel AT91SAM9G20 board. It's my last Atmel hardware that I've turned off (I still have a ton of old Atmel gear, and some newer armv7 A5-based ones, but none of it is powered on: FreeBSD never supported the new gear, and the old gear barely works anymore).


Recovering git repo

I had a crash shortly after updating my git tree to master. When I rebooted, git was confused:
% git status
fatal: not a git repository (or any of the parent directories): .git
Needless to say, my heart skipped a beat. I have a ton of un-backed-up branches in this tree. There's many things that can cause this, according to a quick web search. However, a college told me they've seen this and the most common problem is with .git/HEAD. Looking at mine, it was 0 bytes long. This file has pointer to the latest branch checked out. Since I'd just checked out master, I was able to fix this by creating .git/HEAD:
ref: refs/heads/master
and life was good.



How to get a memory mapped serial console

Memory mapped uart

So, a friend was bringing up FreeBSD on a new system. It didn't have a traditionally mapped UART, but a memory mapped one, like we have in the embedded world. After stumbling around, I thought I'd document how to get a serial port in a situation like this.

First, he had linux running. It recognized the serial port. And it worked in UEFI.

In Linux, we first found the ttys:
% dmesg | grep tty
AMDI0020:00:ttyS4 at MMIO 0xfedc9000 (irq = 3, base_baud = 30000000) is a 16550A
AMDI0020:01:ttyS5 at MMIO 0xfedca000 (irq = 4, base_baud = 30000000) is a 16550A
ttyS4 at MMIO 0xfedc9000 (irq = 3, base_baud = 30000000) is a 16550A
ttyS5 at MMIO 0xfedca000 (irq = 4, base_baud = 30000000) is a 16550A
So, this is weird. Or rather, it's normal if you are used to ARM s16550's memory mapped, or some PCIe, PCI and CardBus serial cards.

It turns out that even if we don't have a driver for this that attaches automatically (uart works, but needs some glue on this platform), you can have a serial port console none-the-less. At the boot loader OK prompt:
OK set hw.uart.console="mm:0xfedc9000,rs:2"
What's the "rs:2"? If you look at the source it's Register Shift. It means that the cadence of registers isn't 1, but 4. It also means that each register is 32-bits, but only the lower 8 bits are valid. That's also typical in ARM and MIPS embedded processors.

So now you know.


FreeBSD Lua Loader (started in GSoC 2014) about to land

Just a quick note. I've been working on making the /stand environment easier to deal with and integrate into. After months of work on and off, the using Lua as a boot loader extension language is ready to roll into -current.

A preview review can be found here.


Setting up a SVN mirror

Setting up the svn mirror

Just following the steps in
in the Setting up a svn mirror (5.4.7) but have found there's some issues...

First step, grab the svn seed file
% fetch ftp://ftp.freebsd.org/pub/FreeBSD/development/subversion/svnmirror-base-r290116.tar.xz
this takes about 30 minutes. Also grabbed svnmirror-ports-r400410.tar.xz for a ports mirror. These file names are different than the instructions.

Next extract this somewhere (say /home/svnmirror/base) and then update it with "svnsync sync file:///home/svnmirror/base" and then set it to updating. You can do similar things for docs and ports. These files are a couple of years old now, so the sync still takes a several of hours. I believe that this is due to the protocol round trips required to extract it being sensitive to the latency between my machines and the FreeBSD svn server.

I use the following shell script as a cron job to keep things up to date.
lock="lockf -s -t 0 -k /tmp/svn-sync-mutex"

${lock} svnsync sync ${URL}/base >> ${logdir}/svn-base.log 2>&1
${lock} svnsync sync ${URL}/ports >> ${logdir}/svn-ports.log 2>&1
${lock} newsyslog -r -f ${logdir}/newsyslog.conf


FreeBSD cumulative commit graphs

FreeBSD Commit Graph

I've updated an old graph from a talk I gave in 2010 to have data up through today. The rate of development seems fairly steady.

When last I did this release, stable branches were flatter and lived longer. Now there's a fair amount of activity after the branch that continues for years.

Note: The hash marks are generally releases. I've removed the 'artificial' commits that were needed as a result of the CVS to SVN migration the project did a few years ago. The 2.0.5 branch has been omitted, and the 2.2.9 release (never tagged in the repo and < 10 changes different than 2.2.8). I had release numbers on the hash marks in earlier revs, but that's too busy now.


Reading in my Rainbow Floppy Collection

Recently, I acquired a kryoflux board to help read in the Venix floppies... I finally have it up and running.

I've managed to read in most of my collection of ~300 Rainbow floppies. 95% of them were actual Rainbow disks. The other 5% were either completely blank, or some variety of IBM format (including 3.5" 720k on a 5.25" floppy!) with a couple of 'robin' formatted ones just to keep you on your toes. I didn't wind up seeing any of the very rate dual-sided RX-50's (they exist, but are super rare because this was a home-brew job not an official thing).

While it is possible just to do
dtc -ffoo.img -i4
It takes a long time since it's reading the back-side that has no data on it. Using
dtc -ffoo.img -g0 -i4
is all that's needed to go fast. Some caveats should be observed.

Nearly every track has extra data in at least one of the sectors. This threw me for a loop, but is harmless (as stated in the manual).

I had some issues with old, ill-maintained floppy drives. Bought a refurbished one and all my troubles just went away. I had several old drives and based on the good floppy drive, I could find known good floppy diskettes and was able to refurbish one of my old units. I had the best luck with the TEAC FD55-GFR drives, btw. Not sure it's worth while to refurb the rest of the units, though, since I have something that's working...

Keep an eye on the number of sectors reported. It's usually 10. If it isn't, stop right away. It isn't a Rainbow / RX-50 formatted disk. If it's 8 or 9 sectors, it's likely an 320k or 360k IBM floppy. Removing -g0 and adding -k2 is useful for these diskettes (saves time more than anything. If it's 15 sectors, it's likely a 1.2MB floppy. Had 2 or 3 of those mixed in with these diskettes despite never having a machine that produced them.

As these disks age, you may need to try multiple times. I have some disks, though (maybe 1%) that I can't read back certain tracks on even with that. The default is good, but trying more helps with a similar number (so -t10 -tc5). I don't know if there's a way to retry individual sectors, or if dtc is internally buffering good sectors from previous retires or not. Previous programs I've had on the Rainbow for extraction had great luck retrying sectors multiple times when the full track read didn't work...

Some disks are copy protected. Not sure how best to preserve these disks. The -i4 format won't work with, for example, Lotus 1-2-3 system disks because tracks 78 and 79 have sectors numbered up to 12 and are missing sectors (they do this with a special format that I believe omits sectors 9 and 10 and instead has sectors 11 and 12). There are other copy protected disks that have deleted sectors since the Rainbow could easily read those, but they were hard to create from standard interfaces. Don't know if -i2 images would help, or if I have to go all the way down to -i0 to ensure proper imaging. The copy protected disks, however, were all broken in the '80s, so preserving them isn't so interesting to me. There's also a couple of disks that have data on track 82, but it's marked as track 79 and it's unclear what this data is. I know there are several old formats that preserve this info (TELDISK and DISKIMAGE being two that I've run into most).

cpmtools is able to extract data from CP/M formatted disks (use dec_pro format). It understands all about the interleaving you need to do on tracks 2+ and can work on raw kyroflux images.
As for MS-DOS disks, I wrote a small utility to convert the physical layout to a logical one, and to (optionally) replace sector 0 with a standard boot record with the BPB in it (Rainbow floppies have Z80 boot code which has a different starting byte (f3) and doesn't have the tables expected. Doing this, both mtools and FreeBSD's msdosfs can copy files easily enough. But so far it's not reversible, so you can't (yet) use it to create floppy images. I'll be testing two tricks (just prepending the magic sector and appending sector 0 at the end) to see if that might help. I can post a pointer if there's interest.

Venix disks are a mixed bag as far as data extraction, but are otherwise boring: 80 tracks, 1 side, 10 sectors per track. Just had brief access to the distribution diskettes, mostly prior to having kryoflux (and the reason for buying it). Ironically, I couldn't get the kryoflux working due to the drive issues I discussed above before I had to return the distribution diskettes. But I was able to copy them natively on my Rainbow...

Finally, there's still 2 or 3 disks that I've been able to read in that I have no clue what the format is. They are kinda sorta CP/M, but also kinda-sorta MS-DOS from looking at the first two tracks, but neither quite works to decode them. Physically, they RX-50. They don't look like ODS-2 nor v7 unix file systems, but that's harder to know for sure. They are unbootable (tracks 0 and 1 are 0xe5, the erasure pattern for RX-50s).

My only real complaint is that it would be nice if dtc would spit out a summary at the end, including tracks unreadable. I have to keep an eye on the logs otherwise. Not a huge deal, to be sure.