20170511

v7 compilation for simple .c's in /usr/src/cmd.

After about 4 and a half hours of build time, I have results to share. There's 110 .c files in /usr/src/cmd. 85 of these compile and link (it's unclear if they run). 25 have a compile or link error:

  • accton, sa (lack of acct in libc)
  • ar, nm, prof, ranlib, size, strip (mismatch between NMAGIC/OMAGIC and ARMAGIC_X)
  • arcv (obsolete conversion program written in old style)
  • clri, dcheck, dump, dumpdir, icheck, mkfs, ncheck, quot, restor (inode differences)
  • dmesg (message buffer issues?)
  • getty (freaks out compiler)
  • graph (missing symbols for curses and libm stuff)
  • osh (old shell? Written in funky style)
  • ps, pstat (proc definition differences)
  • tc (silly compiler issue, trivial to fix, but I have not Tek4015 terminals)
So 10 classes of issues for the affected files. Not too terrible. But there's other issues that will need looking into. 'ld' compiles fine, but it works on pdp11 a.out files, for example. cc compiles, but doesn't have the venix specific switches. I've written a shell script to do this, not a Makefile, and the commands used aren't quite right. Venix appears to support different amounts of stack under the data segment (but without docs, it's hard to know what flags I need), but none of my binaries have those. There's NMAGIC and OMAGIC binaries that have different meanings, and so on. All this makes the 'file foo /bin/foo' differ. Plus I haven't done the more complicated commands, nor have I poked at the compiler apart from cc, nor the kernel really at all, nor the libraries, nor the .y files. And factor.s and prime.s are pdp-11 assembler, so I'd have to pull those in from somewhere.

But given that almost all the normal commands compiled just fine, and many appear to work gives me hope that I might be able to reconstruct the sources used to build at last parts of the system. For some reason, that's appealing to me, but I can't really articulate a reason why, or come up with why that might be useful. Though it seems like a fun background project to celebrate the 40th anniversary of Version 7 in a couple of  years. Though running it on a 35 year old computer at that point may limit its appeal...

Rebuilding v7 sources on Venix, early results

I just started rebuilding the v7 sources on my Venix Rainbow on a lark. I'll post a full writeup, but here's a two early things I've learned.

First, it takes 1 minute to compile cat.c into cat. Yes, one full minute. On my normal server, it takes .1s of elapsed time. This is a 600x speedup since then, likely more since clang isn't known for being gentle on system resources.

Second, some compiler error messages are classic:
"getty.c", line 12: compiler error: insane structure member list
I have always thought getty to be rather opaque to use. The code is somewhat simple, but that simplicity doesn't lead to easy understanding in this case.

Most things not dealing with the filesystem are building w/o a problem. This is a pleasant surprise. Though the speed at which they are building isn't going to set any records. About 5% have issues that prevent compilation or linkage out of the box, though the problems look easy to fix. I started the build at 9:30. It's now 2 hours later and we're to building 'grep'. We're on pace for a rebuild of just the trivial foo.c -> foo programs taking 5-6 hours. This isn't for the more complicated programs like make or tar that have their own subdirectory.


20170507

Rainbow Venix with BSW Enhancements

I've taken some time to look at the Venix disks. I've managed to read them all after retrying the read errors enough. I've created new disks and installed it on my system. I'll go through all that in a forthcoming post since there's lots of fiddly bits.

This post I'll expose what BSW is. It's the Boston Software Works. It was a company that was run by Larry Campbell from 1985 until 1995. They had a number of different products over the years. Through about 1988 they offered their enhanced version of Venix for $800 a copy (not their choice on the price).  It was normal Venix, only better. Larry posted the following to mod.newprod on Usenet back in December1986 describing what it was. I've edited the formatting a bit, and trimmed the headers:
The Boston Software Works (BSW) is now distributing an improved version of VENIX/Rainbow.  VENIX/Rainbow is a complete UNIX(tm) system for the DEC Rainbow 100, developed by VenturCom, Inc. of Cambridge, Mass. It is a V7-based system, with extensions such as plot(3) drivers for the Rainbow color graphics board and DEC LA50 printer, semaphores, and shared memory.
Standard VENIX/Rainbow requires and supports the DEC RD51 winchester disk (10MB) only.
The Boston Software Works enhancements to VENIX/Rainbow include:
  1. Support for any disk physically connectible to the Rainbow. Our development system runs a 70MB Micropolis disk.
  2. Virtual consoles, as found in VENIX on IBM-compatible machines, and also in XENIX and Microport System V/AT.
  3. Support for the NEC V20 chip, which replaces the Intel 8088 for a 10-15% performance gain.
  4. Support for simultaneous use of a monochrome display (for terminal/console use) and a color graphics display (for graphics).
  5. Ports (on an as-is basis) of several public domain programs, such as the Usenet news software, rn, smail, and Jove, an excellent EMACS subset.
  6. Support for the CHS dual-winchester controller.
The price (single copy) for VENIX/Rainbow is $795 and includes a complete manual set.
I've installed both the original Venix/Rainbow and this enhanced version. The enhanced version is the way to go. It knows how to cope with the larger disk I installed in my Rainbow, and the installation process was much less painful (though there were some up-front gotchas that I wound up having to re-run WUTIL to get through).
These are all cool things. The only issue that I've seen is that the .o's aren't included so it's impossible to rebuild the kernel with these neat, new features without significant hassle. The other problem is that while I can get stable file transfer on MS-DOS with LCTERM at 9600 baud, I can't with Venix. 4800 is error prone even. Messing with the serial chip seems to also trigger the dreaded Interrupts Off message, so I've not messed with it too much. Still don't know if that's my Rainbow going bad, or other issues (have some spare parts arriving soonish). The maximum throughput is only 240-300 CPS anyway with LCTERM, so maybe I'll just run at 2400 or 3600 baud.
Not listed above: it kinda seems like it is reading my clikclok. I did have to fix the 'date' command to grok Y2k though (well, since I don't have the sources, I used the Unix v7 sources from TUHS). If this keeps up, I'll put together a distribution of useful things from V7 :).
I'll blog more when I've run through another install and worked out how to boot in MESS (it has a bug when booting directly off the hard drive, but maybe the BOOT program will fix that).

[[ Edited May 8, 2017 to supply link to original mod.newprod article on google groups ]]

20170501

Rainbow 100 Windows 1.0 disks, redux

Rainbow 100 Windows 1.0 Disks

OK. I never thought I'd find myself writing a blog about Microsoft Windows of all things. However, that's what I'm doing.

A while ago, on the cctalk mailing list I mentioned that I had Windows 1.0 disks for the Rainbow. I didn't think anything of it until someone contacted me about it. I'd never looked at them in the 15 years they've been in my collection. You needed the Graphics Option card, and I don't have one so I just filed it away in the back of my brain. I never looked into the situation further.

So, it turns out that the publicly available Rainbow Windows 1.0 disks are really not quite right. They are a collision between the Rainbow 100 version, and the VAXmate version. And evidentially, neither quite work. This collection has been cargo culted around in the three decades since they became available, but they didn't work. it was well known, but the archives contained what they contained and successive generations of hackers tried, and failed, to disentwingle the resulting mess. The mess is still available at archives of archives of archives, for example at Classic CMP there's a copy of the "goodnight" archive which is an incomplete copy of the various BBS archives that existed in the last 80's and early 90's.

Sometime in 2001, Arun Welsh and I corresponded about the DEC Rainbow. I forget the details, but if memory serves, he had a ton of disks that he was looking to get rid of and wanted to see them find a good home. The details are frankly fuzzy after 16 years. I got these disks and took out a few of the interesting bits, but mostly it was a collection of files that were available elsewhere so I didn't look too closely at the collection. But DEC Windows 1.03 stuck in my brain for some reason.

Recently, that boast I made in cctalk generated some email requesting a copy. Since I had my Rainbow out and was messing with it, I thought I'd oblige them. Once upon a time, I could read Rainbow disks, but that was a dozen hardware iterations ago, so I fired up my old machine and read the files in. There's two sets. One was Rainbow MS Windows and was 7 disks. It contained what appears to be a copy of the Goodnight archives with the entertwingled set of files. The other was 4 disks of DEC Windows 1.03.

The other one proved more interesting. It was a different set of files. It came with a letter tucked into the one of the disks giving permission to upload the set of disks to any BBS (including the recipient's BBS, who wasn't Arun, I've forgotten his connection to this mystery). If there's interest, I can post this letter.

Early repots suggest this is a good copy. If they prove true, I wonder how hard it would be to get things corrected in all the online archives.

I've uploaded to a github repo I've created. These files are under 'decwin' while the putative copy of the goodnight archives is under 'rbwin'. I've also uploaded a copy of wutil 3.2 source and binaries. I plan on uploading the Venix binaries there as well, once I get things sorted out and post instructions on how to install them either on real hardware, or in an emulator.

20170421

Lining up

Turns out I didn't need to look for exotic formats to solve this problem.

I had physical copies of the disks with 10 sectors per track. Since I couldn't get tar to line up, I posited that there must be missing sectors. Turns out I was missing something, but it wasn't sectors. That turned out to be a dry well.

What's going on is actually a lot simpler. I managed to get Venix installed on my Rainbow (I'll write up how in my next blog, it's fun to login to my rainbow from upstairs while it's down in the basement).

On the Rainbow 100, under both CP/M and MS-DOS there's an interleave of 2, but no per-track offset. This is done by having the logical sectors ride on top of the physical. So, if I write logical sectors in the order 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, it gets put on the disk in sectors with the labels 1, 6, 2, 7, 3, 8, 4, 9, 5, 10. Why go through this crazy arrangement? For performance. This helps sequential performance. After I read sector 1, I have half a disk rotation (at 300rpm or 5/s that's about 10ms) to queue up the read for sector 2. If I have a slow system, this can help performance a lot, since I don't have to wait for the disk to rotate a full rotation if I'm not fast enough to setup the transfer for second immediately. This is often impossible on slower systems since the inter sector gap is maybe 20-40 bytes. 40 bytes transfers at 250kHz, 320bits at 250kHz is about 100 micro seconds. At 4.8MHz that the Rainbow runs at, this is 5000 instructions. But since there's latency between when the transfer is complete to the controller, and the system interrupts this can easily have less than 10 microseconds to respond. On the Rainbow, it takes tens of microseconds to setup the command to read the floppy. Given such a tight time budget, you can get a big gain by requiring only 100ms to get the next sector instead of 200ms.

This is well known and well documented in the Rainbow community. I had to cope with it when I wrote IMPDRIVE back years ago by translating the requested sector so that it would read the proper sector when reading 3.5" floppies which don't do this.

There's another way to get better performance. MS-DOS and CP/M don't do this. But Venix does. If you want to reduce the amount of time that you have to wait for sector 1 to spin by the head when stepping from the previous track, you can skew the sectors another way. If you shift the start of the interleave sequence above to be something like 8, 4, 9, 5, 10, 1, 6, 2, 7, 3. This gives you half a rotation to get the head moved and settled. If sector 1 is right under the head by the time the head is done moving and the host gets its command to the floppy controller, we don't have to wait an average of half a rotation.

So to sort out this, I dd'd one of the disks under Venix ono the disk. Tar was able to read it readily there, but couldn't on raw images. So, I wrote a program to undo these effects. Once I did that, I was able to read the images with tar.

Once I was able to decode the images, I discovered that the floppies I'd read out under dos using rbimg. icreate.exe is used to read it out, and iwrite.exe can be used to recreate them. These days, this is the gold standard to preserve disks. Well, non-copy protected disks. If you want to preserve the disk in all its glory, use a disk imaging solution like Kryoflux.

20170418

Not lining up

Close, but not quite

So, it looks like I'm getting closer. I read the disks with the 'physical' read option, which did sector skewing. Now that I've looked at them, I think that was a mistake. It looks like they are meant to be read in a logical mode, so I'm redoing a few to analyze again.

So I wrote a program to try to sort that out. It did work, but not quite. It got things almost right, but over-shot the mark when it came to getting the second or third file from the tar ball. It looked for all the world like there were missing sectors or something.

On a lark, I decided to try to read 11 sectors instead of 10 that RX-50's are usually formatted at. And I was able to read all 11, and the data for the 11th doesn't seem to match anything else. But we still have a mismatch, and I'm trying to chase that down. Well, 11th sector for tracks 2 and larger. So, I need to try to figure out why I'm not seeing extra data between the files.

So it looks like I'll have to do this again to get all the bits.... I think I need to puzzle this out with some friendly files that have text (include files) in them. maybe there's 12 sectors, though that would be unprecedented for these drives. You can do 800k like the RX-50, and also 880k for the amiga, but I don't think you could fit much more than that on a track, but who knows, maybe there is 12 sectors and I'm messing up.

Then again maybe I should just wait for the Kryoflux to show up so I can image the entire disk, including the boot disk.

20170417

Peering into the Venix Disks with read errors

As you may recall from last time, I had 5 errors on two disks I'm trying to read for the Rainbow Venix Floppy project.

The specific errors were on two disks. 76/3, 77/7 and 78/5 on the "User 3" disk and 78/5 and 79/5 on "User 2". The program I was using to read the disks didn't report the kind of error, so I'll have to augment it to do so.

This is very concerning to me, so I though I'd take a closer look what's on the disks. It appears that the disks are much simpler than I'd though. The first two tracks (10k) is nothing at all. Then we have what appears to be a TAR file after that. So, it looks like, worst case, we have at most 5 files that are corrupted, and more likely only a couple. The distance on User 2 is only 5k, while the distance on User 3 is 11k. There's a good chance that there will just be one file that's affected.

If I wanted to try to extract these disks, there's another wrinkle. I thought a simple dd to remove the first 10k would do the trick. However, there's a snag. I read these in the physical mode. The RX-50s, at least as used on the Rainbow, have a sector interleaving starting with track 2. The actual sectors are labeled as 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 on the disk (that's the order I read them out in). However, it appears that these sectors correspond to 1, 6, 2, 7, 3, 8, 4, 9, 5, 10, so I'll have to write a stupid filter to de-interlace them. Otherwise TAR throws all kinds of fits.

Unix v7 Tar files, btw, have fixed 512-byte headers. Wasteful, to be sure, but great for data recovery. A quick grep from hexdump shows what's on the 'User 1' disk:

00001400  2f 75 73 72 2f 61 64 6d  2f 00 00 00 00 00 00 00  |/usr/adm/.......|
00001800  2f 75 73 72 2f 62 69 6e  2f 00 00 00 00 00 00 00  |/usr/bin/.......|
00001c00  2f 75 73 72 2f 62 69 6e  2f 61 64 62 00 00 00 00  |/usr/bin/adb....|
00009200  2f 75 73 72 2f 62 69 6e  2f 72 61 6e 6c 69 62 00  |/usr/bin/ranlib.|
0000c600  2f 75 73 72 2f 62 69 6e  2f 63 61 6c 00 00 00 00  |/usr/bin/cal....|
0000e000  2f 75 73 72 2f 62 69 6e  2f 61 63 00 00 00 00 00  |/usr/bin/ac.....|
00010800  2f 75 73 72 2f 62 69 6e  2f 62 63 00 00 00 00 00  |/usr/bin/bc.....|
00011400  2f 75 73 72 2f 62 69 6e  2f 64 6f 73 63 6f 70 79  |/usr/bin/doscopy|
00014400  2f 75 73 72 2f 62 69 6e  2f 63 62 00 00 00 00 00  |/usr/bin/cb.....|
00014600  2f 75 73 72 2f 62 69 6e  2f 64 63 00 64 63 00 2d  |/usr/bin/dc.dc.-|
00015000  2f 75 73 72 2f 62 69 6e  2f 63 61 6c 65 6e 64 61  |/usr/bin/calenda|
00016c00  2f 75 73 72 2f 62 69 6e  2f 63 6f 6c 00 00 00 00  |/usr/bin/col....|
0001a200  2f 75 73 72 2f 62 69 6e  2f 64 6f 73 00 00 00 00  |/usr/bin/dos....|
00020400  2f 75 73 72 2f 62 69 6e  2f 64 65 72 6f 66 66 00  |/usr/bin/deroff.|
00021e00  2f 75 73 72 2f 62 69 6e  2f 65 67 72 65 70 00 00  |/usr/bin/egrep..|
00026a00  2f 75 73 72 2f 62 69 6e  2f 66 67 72 65 70 00 00  |/usr/bin/fgrep..|
00027a00  2f 75 73 72 2f 62 69 6e  2f 67 72 61 70 68 00 00  |/usr/bin/graph..|
0002be00  2f 75 73 72 2f 62 69 6e  2f 69 70 6c 6f 74 00 00  |/usr/bin/iplot..|
00031a00  2f 75 73 72 2f 62 69 6e  2f 6c 65 78 00 00 00 00  |/usr/bin/lex....|
00039e00  2f 75 73 72 2f 62 69 6e  2f 73 70 6c 69 6e 65 00  |/usr/bin/spline.|
0003c400  2f 75 73 72 2f 62 69 6e  2f 6c 70 73 74 6f 70 00  |/usr/bin/lpstop.|
0003f800  2f 75 73 72 2f 62 69 6e  2f 6d 34 00 00 00 00 00  |/usr/bin/m4.....|
00042a00  2f 75 73 72 2f 62 69 6e  2f 6e 65 71 6e 00 00 00  |/usr/bin/neqn...|
0004a600  2f 75 73 72 2f 62 69 6e  2f 6e 72 6f 66 66 00 00  |/usr/bin/nroff..|
00054600  2f 75 73 72 2f 62 69 6e  2f 73 70 65 6c 6c 00 00  |/usr/bin/spell..|
00054c00  2f 75 73 72 2f 6c 69 62  2f 74 6d 61 63 2f 74 6d  |/usr/lib/tmac/tm|
00055600  2f 75 73 72 2f 62 69 6e  2f 79 61 63 63 00 00 00  |/usr/bin/yacc...|
so there's lex, yacc, and several familiar filters. Transferring from the Rainbow is being super slow since I'm sending them via old-reliable KERMIT, so I've only looked at the first file.

All in all, I'd say this is encouraging news. I'll transfer the other disks and see what's missing to see if it is worth trying to delve further into the missing bits or not. If nothing else, it means we can use whatever data we can for those 5 sectors and the reduced functionality may be almost nothing. And there's a chance we can get a second copy of the affected files from the boot disk or something.

20170415

Rainbow 100 Venix/86R Disks Found

Recently, there was a post on an obscure retro-computing blog. It said that someone had acquired a Rainbow 100, which isn't so unusual (though there's only a few on the market). The part that was interesting was that it said it included VENIX disks. After some email back and forth, the owner sent the disks to me for reading.

After fighting for a week with PCs (more on that in another blog), I decided to search to see if there was a Rainbow native solution. Turns out that there was something called RBIMG by PrintStar (Jeff Armstrong) which I was able to get going on my old Rainbow 100B. It came with source because the Venix disks were mostly good, but some sectors requires a number of retries. It appears to be a Version 7 Unix for the PC.

I've read in the Boot disk, a bunch of Xfer Disks (not sure what they are), All the User and System utilities and an BSW extension disks. That's the good news. The bad news is that I still have a few sectors that aren't doing so hot. User3 disk has 3 errors, User2 disk has 2 errors and the boot disk has 1 error.

The Boot disk's one error, though, seems intentional. I've actually booted the Venix disk on my Rainbow, and it comes up. It also tells me the serial number after what sounds like a read error. I'm not yet sure what scheme that Venix uses, so that one might be solvable. It's cool to see a unix kernel that's only 45k in size.

So I'm 5 errors away from having a full set of disks. My thought was that maybe these disks were produced with a 1.2MB drive instead of a real RX-50, so diversity being good, I'm going to try to bring up an old PC I have sitting on the E-Waste pile. I have newer ones, but the BIOS doesn't support 1.2MB drives in those, which may be a clue that 1.2MB drives won't work at all in them (the literature on the web is spotty on this topic, but seems to indicate that latter-day systems cost-reduced 360k/1.2MB support out of the controller). More on those trials and tribulations in a newer post.

20170411

Installing FreeBSD-current with 11.0R installation image

Just a quick blog to document a trick.

Boot the install image (or even just a boot-only image). Get a Shell. At the shell, type
# env UNAME_r=12.0-CURRENT bsdinstall auto
and it will get the latest 12.0-current build installed. Thanks to Allan Jude for alerting me of this trick.

Making Cables

Tonight I assembled my cables I'll need to read my Rainbow 100B ST-251-1 hard drive I've had for a long time to get a backup.

I recently purchased a MFM ST-506 emulator which supports reading and writing old hard disks. To read the ST-252-1 I need the old standard two-cable (20-pin and 34-ping) to connect it to the emulator. This will give me a final backup, and also make it easy for me to snag the data from the drive.

Once I get this done, I'll switch to using the emulator to access the drive so I'll have continuous backups. It will also make it possible for me to explore Venix if the Venix disks that I just got are still good.... Still need to read them. I've hit a few snags in reading them from FreeBSD, which appears to have broken ISA DMA in 11.x, so I'll have to install 10.x instead of -current.

We'll see how it goes...

20170405

Compiling sdcc for FreeBSD

In preparation for a possible new project, I needed to build sdcc, a compiler that targets a bunch of 8-bit architectures. You can find information about sdcc at their sourceforge page: http://sdcc.sourceforge.net/

Usually, this is just a port away. But due to looking at the wrong tree, I thought I had to compile it myself. The port and package are great, and are what people should normally be using, but the problems showed the need to use a trick or three to get the job done. I thought I'd document them here, since they are minimal and easy to write up.

FreeBSD's compiler doesn't default to having /usr/local/include in its include path, or /usr/local/lib in its library path. Normally, one would try to get around this by just add --prefix /usr/local to the configure command. However, that doesn't work for the sdcc stuff (since it is already the default). You have to explicitly add more things to the command line. After some digging, it turns out it's only a few variables to worry about:


./configure CC=clang CXX=clang++ CPPFLAGS="-I/usr/local/include -L/usr/local/lib" CFLAGS="-I/usr/local/include -L/usr/local/lib"
 The key ones here are CPPFLAGS and CFLAGS.

With that, I have a build sdcc. But it's a pain to install. The port / package is much easier for that reason alone.