20211002
Spelling Fixes -- Some Advice
20210828
A new path: vm86-based venix emulator
Venix Emulator Update
Enter VM86VENIX
Tracking down a silly bug
% ls
ls: Sig 44
%
That's weird. So I added tracing. I tried the cc command, which in this version is a simple program that orchestrates all the different parts of the compiler using fork, exec, dup and the strategic close/open pair to setup stdin etc. All the tracing looked good as well, we'd see something like:
123: fork() 124
123: wait()
124: ... lots of stuff
124: exit 0
123: wait pid 124 status 0
and then 123 would proceed to delete all the temp files and exit. It was like it was getting an error, despite its children exiting without an error. Every time it was like this, but only when I ran the optimizer. When I'd re-run the ls test, I'd get different Sig values as well.
Available 8086 compilers in 1985
The wait(2) system call...
SYNOPSIS
wait(statusp)
int *statusp;
which shows wait taking a pointer. So I assumed this was what the kernel received in the first arg that's passed into the kernel (DX). I assumed that DS:DX pointed to an integer where I'd return the status. Most of the time, DX was something that looked like a pointer on the stack, so I just did a copyout. The problem is that's not right.
So, I took another look at the manual. At the end of the man page I saw:
8086 BX=7
int 0xf1
AX = pid of process
DX = status
and then it hit me. DX isn't a pointer to anything. After int 0xf1 AX is the pid of the process (the normal return value) and DX is the status. Disassembling wait.o confirmed this:if statusp is not 0, dx is copied back to *statusp. Doh! The classic pointer vs value mistake. Fixing my implementation to take out the copyout and replace it with setting DX in the processor context made pcc work. And cc worked. And the silly test programs I wrote in the middle to debug things worked. Woo Hoo!
Once I fixed this, all weird combinations of compilations suddenly worked for me. I could optimize, strip, etc and there were no oddities.
Conclusion
20210501
On Updating QEMU's bsd-user fork
QEMU bsd-user
20210416
Customizing Emacs for Git Commit Messages
Customizing Emacs for Git Commit Messages
(add-hook 'find-file-hook 'imp-git-hook)
will do the job nicely. Next up, we need to define this function to do something useful when run (indeed, failure to define it will result in an error when visiting all files).
(defun imp-git-hook ()
(when (string= (file-name-base buffer-file-name) "COMMIT_EDITMSG")
(freebsd-git-setup)))
buffer-file-name is a local variable that has the full path name to the buffer, if any. file-name-base is like basename(1) in that it returns the name of the file without the extension, rather than its whole path.
But what is 'freebsd-git-setup'? It's a little function I wrote to set the fill column to 72 (I usually have it at 80, but that produces commit messages that are just a bit wide when git adds leading spaces) and adds a sponsorship tag to my commits:
(defun freebsd-git-setup ()
(save-excursion
(setq fill-column 72)
(if (re-search-forward "^Sponsored by:" nil t)
t
if (re-search-forward "^\n#" nil t)
(replace-match "\nSponsored by:\t\tNetflix\n\n#")))))
But it only adds the sponsorship tag if one isn't there yet.
This is a proof-of-concept function. No doubt it will have to evolve over time. The project adds 'Differential Revision' tags as the last tag in a commit message because differential requires (required?) that. And it wouldn't be good to add this for non-FreeBSD commits, so I'll have to find a way to filter for that... But I thought this would be useful, if nothing else than to my future self as a roadmap for how to do things like this.
20210131
EPSON QX-10 20MB Hard disk
EPSON QX-10 20MB Hard Disk
PT.COM for EPSON QX-10 PeachText 5000 date changed - 02/03/84
and similar references to the QX-10 or EPSON CP/M.
So, this was from a Epson QX-10 CP/M system. Looks to be a soft-water service company from South Bend Indiana. All their books and correspondence from the mid 1980s was on it, along with some interesting disk support software. There's even some bits of Z80 assembler, but they are too disjointed to know what they were for.
I've not been able to get cpmtools to read the disk in a structured way, however, so it's hard to share just the interesting bits. Still working on it.
If you have one of these machines, or are interested in preserving software from it, please let me and we may be able to work something out.
20201006
How to Recover From a BIOS Upgrade
Recovering From Firmware Upgrade
How to Recover
C:\WINDOWS\system32> mountvol w: /sC:\WINDOWS\system32>w:W:\> cd EFI\Microsoft\BootW:\EFI\Microsoft\Boot> ren bootmgfw.efi bootmgfw-back.efiW:\EFI\Microsoft\Boot> copy W:\EFI\FreeBSD\loader.efi bootmgfw.efiW:\EFI\Micorsoft\Boot>
% sudo efibootmgr -vBoot to FW : falseBootCurrent: 0001Timeout : 0 secondsBootOrder : 0001, 2001, 2002, 2003+Boot0001* Windows Boot Manager HD(1,GPT,f859c46d-19ee-4e40-8975-3ad1ab00ac09,0x800,0x82000)/File(\EFI\Microsoft\Boot\bootmgfw.efi)nvd0p1:/EFI/Microsoft/Boot/bootmgfw.efi /boot/efi//EFI/Microsoft/Boot/bootmgfw.efiBoot2001* EFI USB DeviceBoot2002* EFI DVD/CDROMBoot2003* EFI NetworkUnreferenced Variables:%
% sudo efibootmgr --create --loader /boot/efi/EFI/freebsd/loader.efi --kernel /boot/kernel/kernel --activate --verbose --label FreeBSDBoot to FW : falseBootCurrent: 0001Timeout : 0 secondsBootOrder : 0000, 0001, 2001, 2002, 2003Boot0000* FreeBSD HD(1,GPT,f859c46d-19ee-4e40-8975-3ad1ab00ac09,0x800,0x82000)/File(\EFI\freebsd\loader.efi)nvd0p1:/EFI/freebsd/loader.efi /boot/efi//EFI/freebsd/loader.efiHD(6,GPT,68f0614d-c322-11e9-857a-b1710dd81c0d,0x7bf1000,0x1577e000)/File(boot\kernel\kernel)nvd0p6:boot/kernel/kernel /boot/kernel/kernel+Boot0001* Windows Boot Manager HD(1,GPT,f859c46d-19ee-4e40-8975-3ad1ab00ac09,0x800,0x82000)/File(\EFI\Microsoft\Boot\bootmgfw.efi)nvd0p1:/EFI/Microsoft/Boot/bootmgfw.efi /boot/efi//EFI/Microsoft/Boot/bootmgfw.efiBoot2001* EFI USB DeviceBoot2002* EFI DVD/CDROMBoot2003* EFI NetworkUnreferenced Variables:%
% cd /boot/efi/EFI/Microsoft/Boot% sudo mv bootmgfw-back.efi bootmgfw.efi%
20201001
20200919
FreeBSD Subversion to Git Migration: Pt 1 Why?
FreeBSD moving to Git: Why
Why?
Apache has moved on, so has llvm
Better CI support
Better Merging
Robust Mirroring
Features from 3rd Party Sites
Improved User-Submitted Patch Tracking and Integration
Collaboration
Skill Set
Developers are already moving
Downsides
Conclusion
20200831
How to transport a branch from one git tree to another
git branch transport
How to export / import the branch
The Hack
bad-tree% git format-patch --stdout main..foo | \ssh remote "(cd some/path && git checkout main && \git checkout -b foo && git am)"
so that's it. We format the patches for everything from the mainline through the end of foo. It doesn't matter where 'foo' is off main. The main.. notation is such that it does the right thing (see gitrevisions(7) for all you could do here).
On the receiving side, we cd to git svn repo (or really any repo, this technique generalizes if you're moving patches between projects, though you may need to apply odd pipeline filters to change paths), make sure we have a clean tree with 'git checkout main' (though I suppose I could make that more robust). We create a new branch foo and then we apply the patch. Easy, simple, no muss, no fuss.
BTW I use '&&' above in case I've fat fingered anything, or there's already a foo branch, etc, it will fail as early as possible.
But what about when things go wrong...
One Last Hack
cd freebsdgit format-patch --stdout main..zstd sys/contrib/openzfs | \sed -e 's=/sys/contrib/openzfs/=/=g' | \(cd ../zfs ; git checkout master && git branch -b zstd && \git am)
This nicely moved the patches from one tree to the other to make it easier for me to create my pull request. Your milage may vary, and I've had to play around with the filter to make sure I didn't catch unwanted things in it... I've not taken the time to create a regexp for the lines that I need to apply the sed to for maximum safety, but so far I've gotten lucky that the above path isn't an any of the files I want to transport this way...
Final Though
Postscript
20200816
A 35-year-old bug in patch found in efforts to restore 29 year old 2.11BSD
A 35 Year Old Bug in Patch.
What is the bug?
*** /usr/src/bin/sh/mac.h.old Fri Dec 24 18:44:31 1982--- /usr/src/bin/sh/mac.h Mon Jan 18 08:45:24 1993****************** 59,63 ****#define RQ '\''#define MINUS '-'#define COLON ':'-- #define MAX(a,b) ((a)>(b)?(a):(b))--- 59,61 ----
--------------------------|*** /usr/src/bin/sh/mac.h.old Fri Dec 24 18:44:31 1982|--- /usr/src/bin/sh/mac.h Mon Jan 18 08:45:24 1993--------------------------Patching file usr/src/bin/sh/mac.h using Plan A...No such line 62 in input file, ignoringHunk #1 succeeded at 53 with fuzz 1 (offset -6 lines).done
Which looks odd. Why is it complaining about a line that isn't there? why did it misapply the patch 6 lines earlier? It thinks it succeeded, but really added back the MAX macro line too early.
Where is the bug?
****************** 59,61 ****--- 59,63 ----#define RQ '\''#define MINUS '-'#define COLON ':'++ #define MAX(a,b) ((a)>(b)?(a):(b))
****************** 59,61 ****--- 59,63 ----
The Fix
Side note: Old Style Context Diffs
FreeBSD Fix
Conclusion
Addendem
20200806
Recovering 2.11BSD, fighting the patches
Recovering 2.11BSD, fighting the patches
- The kernel logger (/dev/klog)
- The namei cache and argument encapsulation calling sequence
- readv(2)/writev(2) as system calls rather than emulation/compatibility routines
- Shadow password file implementation (the May 1989 4.3BSD update)
- A TMSCP (TK50/TU81) driver with standalone support (bootblock and standalone driver)
- Pronet and LH/DH IMP networking support
- The portable ascii archive file format (ar, ranlib)
- The Unibus Mapping Register (UMR) handling of the network was rewritten to avoid allocating excessive UMRs.
- The necessary mods to the IP portion of the networking were made to allow traceroute (which is present in 2.11BSD) to run.
- Long filenames in the file system
The Problem(s)
Fix:Apply the patch "/tmp/dif" (multiple files updated by it) andinstall the new scripts ":splfix.mfps" and "splfix.movb+mfps".
#! /bin/sh -vcd /usr/src/usr.binrm lint/*zcat PORT/lint.tar.Z | tar xf -cd lintrm :yyfix llib-*cp -p /tmp/libs libs
The Path Forward
The Slog Backward
The Framework
The /GENALLSYS script is obsolete and has been removed

FreeBSD git Primer for Users
Today's blog is actually a preview of a git primer I'm writing for the FreeBSD project. It covers what a typical user will need, including those relatively rare users that may have some changes to the base. Please let me know what you think, and ways it can be improved. I'm keen on especially clear and useful pointers for the topics as well.
Also note: The cgit-beta mirror mentioned below is currently for testing purposes only.
If you have a lot of suggests, you can make them directly on the original for this on hackmd.
Scope
If you want to download FreeBSD, compile it from sources and generally keep up to date that way, this primer is for you. If you are looking to do more with the tree, contribute back, or commit changes, then you will need to wait for a later blog where I cover that. It covers getting the sources, updating the sources, how to bisect and touches briefly on how to cope with a few local changes. It covers the basics, and tries to give good pointers to more in-depth treatment for when the readers finds the basics insufficient.
Keeping Current With FreeBSD src tree
First step: cloning a tree. This downloads the entire tree. There’s two ways to download. Most people will want to do a deep clone of the repo. However, there are times that you may wish to do a shallow clone.
Branch names
The branch names in the new git repo are similar to the old names. For the stable branches, they are stable/X where X is the major release (like 11 or 12). The main branch in the new repo is ‘main’. The main branch in the old github mirror is ‘master’. Both reflecting the defaults of git at the time they were created. The main/master branch is the default branch if you omit the ‘-b branch’ or ‘–branch branch’ options below.
Repositories
At the moment, there’s two repositories. The hashes are different between them. The old github repo is similar to the new cgit repo. However, there are a large number of mistakes in the github repo that required us to regenerate the export when we migrated to having a git repo be the source of truth for the project.
The github repo is at https://github.com/freebsd/freebsd.git
The new cgit beta repo is at https://cgit-beta.freebsd.org/src.git
These will be $URL in the commands below.
Note: The project doesn’t use submodules as they are a poor fit for our workflows and development model. How we track changes in third-party applications is discussed elsewhere and generally of little concern to the casual user.
Deep Clone
A deep clone pulls in the entire tree, as well as all the history and branches. It’s the easiest to do. It also allows you to use git’s worktree feature to have all your active branches checked out into separate directories but with only one copy of the repository.
is how you make a deep clone. ‘branch’ should be one of the branches listed in the previous section. It is optional if it is the main/master branch. dir is an optional directory to place it in (the default will be the name of the repo you are clone (freebsd or src)).
You’ll want a deep clone if you are interested in the history, plan on making local changes, or plan on working on more than one branch. It’s the easiest to keep up to date as well. If you are interested in the history, but are working with only one branch and are short on space, you can also use --single-branch to only download the one branch (though some merge commits will not reference the merged-from branch which may be important for some users who are interested in detailed versions of history).
Shallow Clone
A shallow clone copies just the most current code, but none or little of the history. This can be useful when you need to build a specific revision of FreeBSD, or when you are just starting out and plan to track the tree more fully. You can also use it to limit history to only so many revisions.
This clones the repository, but only has the most recent version in the respository. The rest of the history is not downloaded. Should you change your mind later, you can do ‘git fetch --unshallow’ to get the old history.
Building
Once you’ve downloaded, building is done as described in the handbook, eg:
so that won’t be coverd in depth here.
Updating
To update both types of trees uses the same commands. This pulls in all the revisions since your last update.
will update the tree. In git, a ‘fast forward’ merge is one that only needs to set a new branch pointer and doesn’t need to re-create the commits. By always doing a ‘fast forward’ merge/pull, you’ll ensure that you have an identical copy of the FreeBSD tree. This will be important if you want to maintain local patches.
See below for how to manage local changes. The simplest is to use --autostash on the ‘git pull’ command, but more sophisticated options are available.
Selecting a Specific Version
In git, the ‘git checkout’ command can not only checkout branches, but it can also checkout a specific version. Git’s versions are the long hashes rather than a sequential number. You saw them above in the conflict when it said it couldn’t apply “646e0f9cda11”.
When you checkout a specific version, just specify the hash you want on the command line (the git log command can help you decide which hash you might want):
and you have that checked out.
However, as with many things git, it’s not so simple. You’ll be greeted with a message similar to the following:
where the last line is generated from the hash you are checking out and the first line of the commit message from that revision. Also, a word about hashes: they can be abbreviated. That’s why you’ll see them have different lengths in different commands or their outputs. These super long hashes are often unique after 6 or 10 characters, so git lets you abbreviate and is somewhat inconsistent about how it presents them to users.
Bisecting
Sometimes, things go wrong. The last version worked, but the one you just updated to does not. A developer may ask to bisect the problem to track down which commit caused the regression.
If you’ve read the last section, you may be thinking to yourself “How the heck do I bisect with crazy version numbers like that?” then this section is for you. It’s also for you if you didn’t think that, but also want to bisect.
Fortunately, one uses the ‘git bisect’ command. Here’s a brief outline in how to use it. For more information, I’d suggest https://www.metaltoad.com/blog/beginners-guide-git-bisect-process-elimination or https://git-scm.com/docs/git-bisect for more details. The man page is good at describing what can go wrong, what to do when versions won’t build, when you want to use terms other than ‘good’ and ‘bad’, etc, none of which will be covered here.
‘git bisect start’ will start the bisection process. Next, you need to tell a range to go through. ‘git bisect good XXXXXX’ will tell it the working version and ‘git bisect bad XXXXX’ will tell it the bad version. The bad version will almost always be HEAD (a special tag for what you have checked out). The good version will be the last one you checked out.
A quick aside: if you want to know the last version you checked out, you should use ‘git reflog’:
shows me moving the working tree to the master branch (a816…) and then updating from upstream (to 5ef0…). In this case, bad would be HEAD (or 5rf0bd68) and good would be a8163e165. As you can see from the output, HEAD@{1} also often works, but isn’t foolproof if you’ve done other things to your git tree after updating, but before you discover the need to bisect.
Back to git bisect. Set the ‘good’ version first, then set the bad (though the order doesn’t matter). When you set the bad version, it will give you some statistics on the process:
You’d then build/install that version. If it’s good you’d type ‘git bisect good’ otherwise ‘git bisect bad’. You’ll get a similar message to the above each step. When you are done, report the bad version to the developer (or fix the bug yourself and send a patch). ‘git bisect reset’ will end the process and return you back to where you started (usually tip of main). Again, the git-bisect manual (linked above) is a good resource for when things go wrong or for unusual cases.
Ports Considerations
The ports tree operates the same way. The branch names are different and the repos are in different locations.
The github mirror is at https://github.com/freebsd/freebsd-ports.git
The cgit mirror is https://cgit-beta.freebsd.org/src.git
As with ports, the ‘current’ branches are ‘master’ and ‘main’ respectively. The quarterly branches are named the same as in FreeBSD’s svn repo.
Coping with Local Changes
Here’s a small collections of topics that are more advanced for the user tracking FreeBSD. If you have no local changes, you can stop reading now (it’s the last section and OK to skip).
One item that’s important for all of them: all changes are local until pushed. Unlike svn, git uses a distributed model. For users, for most things, there’s very little difference. However, if you have local changes, you can use the same tool to manage them as you use to pull in changes from FreeBSD. All changes that you’ve not pushed are local and can easily be modified (git rebase, discussed below does this).
Keeping local changes
The simplest way to keep local changes (especially trivial ones) is to use ‘git stash’. In its simples form, you use ‘git stash’ to record the changes (which pushes them onto the stash stack). Most people use this to save changes before updating the tree as described above. They then use ‘git stash apply’ to re-apply them to the tree. The stash is a stack of changes that can be examined with ‘git stash list’. The git-stash man page (https://git-scm.com/docs/git-stash) has all the details.
This method is suitable when you have tiny tweaks to the tree. When you have anything non trivial, you’ll likely be better off keeping a local branch and rebasing. It is also integreated with the ‘git pull’ command: just add ‘–autostash’ to the command line.
Keeping a local branch
It’s much easier to keep a local branch with git than subversion. In subversion you need to merge the commit, and resolve the conflicts. This is managable, but can lead to a convoluted history that’s hard to upstream should that ever be necessary, or hard to replicate if you need to do so. Git also allows one to merge, along with the same problems. That’s one way to mange the branch, but it’s the least flexible.
Git has a concept of ‘rebasing’ which you can use to avoids these issues. The ‘git rebase’ command will basically replay all the commits relative to the parent branch at a newer location on that parent branch. This section will briefly cover how to do this, but will not cover all scenarios.
Create a branch
Let’s say you want to make a hack to FreeBSD’s ls command to never, ever do color. There’s many reasons to do this, but this example will use that as a baseline. The FreeBSD ls command changes from time to time, and you’ll need to cope with those changes. Fortunately, with git rebase it usually is automatic.
The commit will pop you into an editor to describe what you’ve done. Once you enter that, you have your own local branch in the git repo. Build and install it like you normally would, following the directions in the handbook. git differs from other version control systems in that you have to tell it explicitly which files to use. I’ve opted to do it on the commit command line, but you can also do it with ‘git add’ which many of the more in depth tutorials cover.
Time to update
When it’s time to bring in a new version, it’s almost the same as w/o the branches. You would update like you would above, but there’s one extra command before you update, and one after. The following assumes you are starting with an unmodified tree. It’s important to start rebasing operations with a clean tree (git usually requires this).
This will bring up an editor that lists all the commits in it. For this example, don’t change it at all. This is typically what you are doing while updating the baseline (though you also use the git rebase command to curate the commits you have in the branch).
Once you’re done with the above, you’ve move the commits to ls.c forward from the old version of FreeBSD to the newer one.
Sometimes there’s merge conflicts. That’s OK. Don’t panic. You’d handle them the same as you would any other merge conflicts. To keep it simple, I’ll just describe a common issue you might see. A pointer to a more complete treatment can be found at the end of this section.
Let’s say the includes changes upstream in a radical shift to terminfo as well as a name change for the option. When you updated, you might see something like this:
which looks scary. If you bring up an editor, you’ll see it’s a typical 3-way merge conflict resolution that you may be familiar with from other source code systems (the rest of ls.c has been omitted):
The new code is first, and your code is second. The right fix here is to just add a #undef COLORLS_NEW before #ifdef and then delete the old changes:
save the file. The rebase was interrupted, so you have to complete it:
which tells git that ls.c has changed and to continue the rebase operation. Since there was a conflict, you’ll get kicked into the editor to maybe update the commit message.
If you get stuck during the rebase, don’t panic. git rebase --abort will take you back to a clean slate. It’s important, though, to start with an unmodified tree.
For more on this topic, https://www.freecodecamp.org/news/the-ultimate-guide-to-git-merge-and-git-rebase/ provides a rather extensive treatment. It goes into a lot of cases I didn’t cover here for simplicity that are useful to know since they come up from time to time.
Updating to a New FreeBSD Branch
Let’s say you want to main the jump from FreeBSD stable/12 to FreeBSD current. That’s easy to do as well, if you have a deep clone.
and you are done. If you have a local branch, though, there’s one or two caveats. First, rebase will rewrite history, so you’ll likely want to do something to save it. Second, jumping branches tends to encounter more conflicts. If we pretend the example above was relative to stable/12, then to move to main, I’d suggest the following:
What the above does is checkout no-color-ls. Then create a new name for it (no-color-ls-stable-12) in case you need to get back to it. Then you rebase onto the main branch. This will find all the commits to the current no-color-ls branch (back to where it meets up with the stable/12 branch) and then it will replay them onto the main branch creating a new no-color-ls branch there (which is why I had you create a place holder name).