Lining up

Turns out I didn't need to look for exotic formats to solve this problem.

I had physical copies of the disks with 10 sectors per track. Since I couldn't get tar to line up, I posited that there must be missing sectors. Turns out I was missing something, but it wasn't sectors. That turned out to be a dry well.

What's going on is actually a lot simpler. I managed to get Venix installed on my Rainbow (I'll write up how in my next blog, it's fun to login to my rainbow from upstairs while it's down in the basement).

On the Rainbow 100, under both CP/M and MS-DOS there's an interleave of 2, but no per-track offset. This is done by having the logical sectors ride on top of the physical. So, if I write logical sectors in the order 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, it gets put on the disk in sectors with the labels 1, 6, 2, 7, 3, 8, 4, 9, 5, 10. Why go through this crazy arrangement? For performance. This helps sequential performance. After I read sector 1, I have half a disk rotation (at 300rpm or 5/s that's about 10ms) to queue up the read for sector 2. If I have a slow system, this can help performance a lot, since I don't have to wait for the disk to rotate a full rotation if I'm not fast enough to setup the transfer for second immediately. This is often impossible on slower systems since the inter sector gap is maybe 20-40 bytes. 40 bytes transfers at 250kHz, 320bits at 250kHz is about 100 micro seconds. At 4.8MHz that the Rainbow runs at, this is 5000 instructions. But since there's latency between when the transfer is complete to the controller, and the system interrupts this can easily have less than 10 microseconds to respond. On the Rainbow, it takes tens of microseconds to setup the command to read the floppy. Given such a tight time budget, you can get a big gain by requiring only 100ms to get the next sector instead of 200ms.

This is well known and well documented in the Rainbow community. I had to cope with it when I wrote IMPDRIVE back years ago by translating the requested sector so that it would read the proper sector when reading 3.5" floppies which don't do this.

There's another way to get better performance. MS-DOS and CP/M don't do this. But Venix does. If you want to reduce the amount of time that you have to wait for sector 1 to spin by the head when stepping from the previous track, you can skew the sectors another way. If you shift the start of the interleave sequence above to be something like 8, 4, 9, 5, 10, 1, 6, 2, 7, 3. This gives you half a rotation to get the head moved and settled. If sector 1 is right under the head by the time the head is done moving and the host gets its command to the floppy controller, we don't have to wait an average of half a rotation.

So to sort out this, I dd'd one of the disks under Venix ono the disk. Tar was able to read it readily there, but couldn't on raw images. So, I wrote a program to undo these effects. Once I did that, I was able to read the images with tar.

Once I was able to decode the images, I discovered that the floppies I'd read out under dos using rbimg. icreate.exe is used to read it out, and iwrite.exe can be used to recreate them. These days, this is the gold standard to preserve disks. Well, non-copy protected disks. If you want to preserve the disk in all its glory, use a disk imaging solution like Kryoflux.

Not lining up

Close, but not quite

So, it looks like I'm getting closer. I read the disks with the 'physical' read option, which did sector skewing. Now that I've looked at them, I think that was a mistake. It looks like they are meant to be read in a logical mode, so I'm redoing a few to analyze again.

So I wrote a program to try to sort that out. It did work, but not quite. It got things almost right, but over-shot the mark when it came to getting the second or third file from the tar ball. It looked for all the world like there were missing sectors or something.

On a lark, I decided to try to read 11 sectors instead of 10 that RX-50's are usually formatted at. And I was able to read all 11, and the data for the 11th doesn't seem to match anything else. But we still have a mismatch, and I'm trying to chase that down. Well, 11th sector for tracks 2 and larger. So, I need to try to figure out why I'm not seeing extra data between the files.

So it looks like I'll have to do this again to get all the bits.... I think I need to puzzle this out with some friendly files that have text (include files) in them. maybe there's 12 sectors, though that would be unprecedented for these drives. You can do 800k like the RX-50, and also 880k for the amiga, but I don't think you could fit much more than that on a track, but who knows, maybe there is 12 sectors and I'm messing up.

Then again maybe I should just wait for the Kryoflux to show up so I can image the entire disk, including the boot disk.

Peering into the Venix Disks with read errors

As you may recall from last time, I had 5 errors on two disks I'm trying to read for the Rainbow Venix Floppy project.

The specific errors were on two disks. 76/3, 77/7 and 78/5 on the "User 3" disk and 78/5 and 79/5 on "User 2". The program I was using to read the disks didn't report the kind of error, so I'll have to augment it to do so.

This is very concerning to me, so I though I'd take a closer look what's on the disks. It appears that the disks are much simpler than I'd though. The first two tracks (10k) is nothing at all. Then we have what appears to be a TAR file after that. So, it looks like, worst case, we have at most 5 files that are corrupted, and more likely only a couple. The distance on User 2 is only 5k, while the distance on User 3 is 11k. There's a good chance that there will just be one file that's affected.

If I wanted to try to extract these disks, there's another wrinkle. I thought a simple dd to remove the first 10k would do the trick. However, there's a snag. I read these in the physical mode. The RX-50s, at least as used on the Rainbow, have a sector interleaving starting with track 2. The actual sectors are labeled as 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 on the disk (that's the order I read them out in). However, it appears that these sectors correspond to 1, 6, 2, 7, 3, 8, 4, 9, 5, 10, so I'll have to write a stupid filter to de-interlace them. Otherwise TAR throws all kinds of fits.

Unix v7 Tar files, btw, have fixed 512-byte headers. Wasteful, to be sure, but great for data recovery. A quick grep from hexdump shows what's on the 'User 1' disk:

00001400  2f 75 73 72 2f 61 64 6d  2f 00 00 00 00 00 00 00  |/usr/adm/.......|

00001800  2f 75 73 72 2f 62 69 6e  2f 00 00 00 00 00 00 00  |/usr/bin/.......|

00001c00  2f 75 73 72 2f 62 69 6e  2f 61 64 62 00 00 00 00  |/usr/bin/adb....|

00009200  2f 75 73 72 2f 62 69 6e  2f 72 61 6e 6c 69 62 00  |/usr/bin/ranlib.|

0000c600  2f 75 73 72 2f 62 69 6e  2f 63 61 6c 00 00 00 00  |/usr/bin/cal....|

0000e000  2f 75 73 72 2f 62 69 6e  2f 61 63 00 00 00 00 00  |/usr/bin/ac.....|

00010800  2f 75 73 72 2f 62 69 6e  2f 62 63 00 00 00 00 00  |/usr/bin/bc.....|

00011400  2f 75 73 72 2f 62 69 6e  2f 64 6f 73 63 6f 70 79  |/usr/bin/doscopy|

00014400  2f 75 73 72 2f 62 69 6e  2f 63 62 00 00 00 00 00  |/usr/bin/cb.....|

00014600  2f 75 73 72 2f 62 69 6e  2f 64 63 00 64 63 00 2d  |/usr/bin/dc.dc.-|

00015000  2f 75 73 72 2f 62 69 6e  2f 63 61 6c 65 6e 64 61  |/usr/bin/calenda|

00016c00  2f 75 73 72 2f 62 69 6e  2f 63 6f 6c 00 00 00 00  |/usr/bin/col....|

0001a200  2f 75 73 72 2f 62 69 6e  2f 64 6f 73 00 00 00 00  |/usr/bin/dos....|

00020400  2f 75 73 72 2f 62 69 6e  2f 64 65 72 6f 66 66 00  |/usr/bin/deroff.|

00021e00  2f 75 73 72 2f 62 69 6e  2f 65 67 72 65 70 00 00  |/usr/bin/egrep..|

00026a00  2f 75 73 72 2f 62 69 6e  2f 66 67 72 65 70 00 00  |/usr/bin/fgrep..|

00027a00  2f 75 73 72 2f 62 69 6e  2f 67 72 61 70 68 00 00  |/usr/bin/graph..|

0002be00  2f 75 73 72 2f 62 69 6e  2f 69 70 6c 6f 74 00 00  |/usr/bin/iplot..|

00031a00  2f 75 73 72 2f 62 69 6e  2f 6c 65 78 00 00 00 00  |/usr/bin/lex....|

00039e00  2f 75 73 72 2f 62 69 6e  2f 73 70 6c 69 6e 65 00  |/usr/bin/spline.|

0003c400  2f 75 73 72 2f 62 69 6e  2f 6c 70 73 74 6f 70 00  |/usr/bin/lpstop.|

0003f800  2f 75 73 72 2f 62 69 6e  2f 6d 34 00 00 00 00 00  |/usr/bin/m4.....|

00042a00  2f 75 73 72 2f 62 69 6e  2f 6e 65 71 6e 00 00 00  |/usr/bin/neqn...|

0004a600  2f 75 73 72 2f 62 69 6e  2f 6e 72 6f 66 66 00 00  |/usr/bin/nroff..|

00054600  2f 75 73 72 2f 62 69 6e  2f 73 70 65 6c 6c 00 00  |/usr/bin/spell..|

00054c00  2f 75 73 72 2f 6c 69 62  2f 74 6d 61 63 2f 74 6d  |/usr/lib/tmac/tm|

00055600  2f 75 73 72 2f 62 69 6e  2f 79 61 63 63 00 00 00  |/usr/bin/yacc...|

so there's lex, yacc, and several familiar filters. Transferring from the Rainbow is being super slow since I'm sending them via old-reliable KERMIT, so I've only looked at the first file.

All in all, I'd say this is encouraging news. I'll transfer the other disks and see what's missing to see if it is worth trying to delve further into the missing bits or not. If nothing else, it means we can use whatever data we can for those 5 sectors and the reduced functionality may be almost nothing. And there's a chance we can get a second copy of the affected files from the boot disk or something.

Rainbow 100 Venix/86R Disks Found

Recently, there was a post on an obscure retro-computing blog. It said that someone had acquired a Rainbow 100, which isn't so unusual (though there's only a few on the market). The part that was interesting was that it said it included VENIX disks. After some email back and forth, the owner sent the disks to me for reading.

After fighting for a week with PCs (more on that in another blog), I decided to search to see if there was a Rainbow native solution. Turns out that there was something called RBIMG by PrintStar (Jeff Armstrong) which I was able to get going on my old Rainbow 100B. It came with source because the Venix disks were mostly good, but some sectors requires a number of retries. It appears to be a Version 7 Unix for the PC.

I've read in the Boot disk, a bunch of Xfer Disks (not sure what they are), All the User and System utilities and an BSW extension disks. That's the good news. The bad news is that I still have a few sectors that aren't doing so hot. User3 disk has 3 errors, User2 disk has 2 errors and the boot disk has 1 error.

The Boot disk's one error, though, seems intentional. I've actually booted the Venix disk on my Rainbow, and it comes up. It also tells me the serial number after what sounds like a read error. I'm not yet sure what scheme that Venix uses, so that one might be solvable. It's cool to see a unix kernel that's only 45k in size.

So I'm 5 errors away from having a full set of disks. My thought was that maybe these disks were produced with a 1.2MB drive instead of a real RX-50, so diversity being good, I'm going to try to bring up an old PC I have sitting on the E-Waste pile. I have newer ones, but the BIOS doesn't support 1.2MB drives in those, which may be a clue that 1.2MB drives won't work at all in them (the literature on the web is spotty on this topic, but seems to indicate that latter-day systems cost-reduced 360k/1.2MB support out of the controller). More on those trials and tribulations in a newer post.

Installing FreeBSD-current with 11.0R installation image

Just a quick blog to document a trick.

Boot the install image (or even just a boot-only image). Get a Shell. At the shell, type

# env UNAME_r=12.0-CURRENT bsdinstall auto

and it will get the latest 12.0-current build installed. Thanks to Allan Jude for alerting me of this trick.

Making Cables

Tonight I assembled my cables I'll need to read my Rainbow 100B ST-251-1 hard drive I've had for a long time to get a backup.

I recently purchased a MFM ST-506 emulator which supports reading and writing old hard disks. To read the ST-252-1 I need the old standard two-cable (20-pin and 34-ping) to connect it to the emulator. This will give me a final backup, and also make it easy for me to snag the data from the drive.

Once I get this done, I'll switch to using the emulator to access the drive so I'll have continuous backups. It will also make it possible for me to explore Venix if the Venix disks that I just got are still good.... Still need to read them. I've hit a few snags in reading them from FreeBSD, which appears to have broken ISA DMA in 11.x, so I'll have to install 10.x instead of -current.

We'll see how it goes...

Compiling sdcc for FreeBSD

In preparation for a possible new project, I needed to build sdcc, a compiler that targets a bunch of 8-bit architectures. You can find information about sdcc at their sourceforge page: http://sdcc.sourceforge.net/

Usually, this is just a port away. But due to looking at the wrong tree, I thought I had to compile it myself. The port and package are great, and are what people should normally be using, but the problems showed the need to use a trick or three to get the job done. I thought I'd document them here, since they are minimal and easy to write up.

FreeBSD's compiler doesn't default to having /usr/local/include in its include path, or /usr/local/lib in its library path. Normally, one would try to get around this by just add --prefix /usr/local to the configure command. However, that doesn't work for the sdcc stuff (since it is already the default). You have to explicitly add more things to the command line. After some digging, it turns out it's only a few variables to worry about:

./configure CC=clang CXX=clang++ CPPFLAGS="-I/usr/local/include -L/usr/local/lib" CFLAGS="-I/usr/local/include -L/usr/local/lib"

 The key ones here are CPPFLAGS and CFLAGS.

With that, I have a build sdcc. But it's a pain to install. The port / package is much easier for that reason alone.