Recently, I started hacking a TiVo HR10-250 that I've had for a million years unhacked.
Back when I first got it, there were dozens of threads of hundreds of messages in about 10 different forums you needed to read in order to hack the goofy thing. Everybody did their own thing, it was manual and a big pain.
Then came zipper (you can find all the details out at this link). It makes it very easy to hack the TiVos in general. However, there was just one problem. The ISO authoring tools it used were windows-based. Since I don't generally run on a Windows machine, this was a problem for me. So I tried to do the steps from the batch file by hand, and it didn't work out so well.
So I rewrote the batch file in a unix script running on FreeBSD. FreeBSD's iso extraction feature of tar made this process hassle free. I can build a new ISO image from an old one using only one port (mkisofs) and no privs are needed at any step. I thought about trying to teach tar how to create an iso image, but since mkisofs does such a good job, and there are *SO* many knobs and dials that need to be implemented to do that right, I just let mkisofs do its thing. I just can't get over how easy FreeBSD's tar made it to break apart the old bootable cd that does the hacking to add the zipper files. Kudos!
I've uploaded the script to the TiVo Community Forum thread about zipper if you are interested in looking into it.
20091105
20091104
T-Mobile and iPhone 3.1.2 firmware update issue
Recently, I used PwnageTool on my mac to update my 2G iPhone from 2.2.1 to 3.1.2. I've been using T-Mobile since day one with my iPhone without hassle. Well, except for the upgrade hassles that I have from time to time. Once I unlocked the SIM, I was able to use the SIM from my T-Mobile DASH in the phone.
When I upgraded this time, however, there was a problem. My data connections stopped working. Since 3.1.2 was otherwise a nice upgrade for me, I went looking for the problem. It turns out that the bundled carrier files that describe what you can and can't do with the iPhone basically said no data.
Carrier files that are correct for T-Mobile, and the data plan I have, can be found iPhone-Dev.ca under the IPCC section of the web page. Instructions for loading the IPCC file were there as well. For T-Mobile, you'll need to call T-Mobile to find the right APN to use. For my plan it turned out to be wap-apn. Your milage may vary, but worst case would involve trying all 4 of the files.
While not a programming hack, this should come in handy for anybody that's in a similar situation. Alas, I have no experience with the 3G or 3GS phones, and problems with the upgrades for those phones seem to be more widely covered (and blogged about).
When I upgraded this time, however, there was a problem. My data connections stopped working. Since 3.1.2 was otherwise a nice upgrade for me, I went looking for the problem. It turns out that the bundled carrier files that describe what you can and can't do with the iPhone basically said no data.
Carrier files that are correct for T-Mobile, and the data plan I have, can be found iPhone-Dev.ca under the IPCC section of the web page. Instructions for loading the IPCC file were there as well. For T-Mobile, you'll need to call T-Mobile to find the right APN to use. For my plan it turned out to be wap-apn. Your milage may vary, but worst case would involve trying all 4 of the files.
While not a programming hack, this should come in handy for anybody that's in a similar situation. Alas, I have no experience with the 3G or 3GS phones, and problems with the upgrades for those phones seem to be more widely covered (and blogged about).
20090514
Broadcom BCM43xx support (bwi) committed
The BCM43xx driver from DragonFlyBSD has been ported to FreeBSD. Andrew Thomas and others have done the port. I've pushed it into the tree, and plan on fixing the one or two quirks with my card.
The driver is far from perfect. 802.11a isn't support. Some older BCM4306 cards don't support channels 1, 2 or 3. It is rumored not to work at all for some people.
The driver was written by Sepherosa Ziehau for DragonFlyBSD. He used the specs available at http://bcm-specs.sipsolutions.net/Specification to write the driver. The driver is written against the v3 firmware. The v4 firmware is needed for support of newer cards, and many people have made noise about porting to that, but nobody has emerged yet to do that work.
The driver will be in FreeBSD 8.0. A back port of the driver to 7.x is difficult because it relies on all the new VAP work that Sam Leffler has done that is not in 7.x (although backports exist for 7.x, they won't be committed to the FreeBSD tree because it breaks binary compatibility).
You need the net/bwi-firmware-kmod port installed for the firmware. Although the binary .o file that the firmware is extracted from is apparently easy to distribute, distributing derived works of that .o file is not. So, there's a port to download, extract the firmware and make a kld for the driver to access.
A man page will be committed soon.
The driver is far from perfect. 802.11a isn't support. Some older BCM4306 cards don't support channels 1, 2 or 3. It is rumored not to work at all for some people.
The driver was written by Sepherosa Ziehau for DragonFlyBSD. He used the specs available at http://bcm-specs.sipsolutions.net/Specification to write the driver. The driver is written against the v3 firmware. The v4 firmware is needed for support of newer cards, and many people have made noise about porting to that, but nobody has emerged yet to do that work.
The driver will be in FreeBSD 8.0. A back port of the driver to 7.x is difficult because it relies on all the new VAP work that Sam Leffler has done that is not in 7.x (although backports exist for 7.x, they won't be committed to the FreeBSD tree because it breaks binary compatibility).
You need the net/bwi-firmware-kmod port installed for the firmware. Although the binary .o file that the firmware is extracted from is apparently easy to distribute, distributing derived works of that .o file is not. So, there's a port to download, extract the firmware and make a kld for the driver to access.
A man page will be committed soon.
20090505
Belkin F6D4230 "Enhanced Wireless Router"
Just got the Belkin Enhance Wireless Router F6D4230-4. Looks like it has a ralink RT3052 or RT3050 under the hood. This is a MIPS24KEc based SoC, with a full Linux GPL tarball available from Belkin's web site.
I haven't had a chance to open the box yet to see what else is inside, and this is inferred entirely from the Linux config file that was in the GPL compliance package.
Not sure what I'm going to do with this box...
I haven't had a chance to open the box yet to see what else is inside, and this is inferred entirely from the Linux config file that was in the GPL compliance package.
Not sure what I'm going to do with this box...
20090423
ExpressCard
Now that I've managed to knock down the number of bugs in NEWCARD to a more reasonable level, I've started looking at ExpressCard again.
As you may know, FreeBSD supports ExpressCard right now for usb devices. They all work, and there's no additional work needed here, except maybe to make more drivers. In fact, I recently purchased an ExpressCard to CardBus adapter (one that lets me plug in an expressCard into a CardBus slot) and it worked with all the usb-based ExpressCards that I have laying around with no changes.
The problem with ExpressCard for PCIe-based devices is one of resource allocation. If the BIOS allocates the resource, then the ExpressCard works without hot-plug. If the BIOS doesn't, then we can see the device with pciconf -l, as well as probe the device. However, since there's no resources setup for the PCI bridge, attempts for the driver to allocate them fail. There's also some bus numbering issues as well.
I've started to attack the resource problem. I'll keep people posted
As you may know, FreeBSD supports ExpressCard right now for usb devices. They all work, and there's no additional work needed here, except maybe to make more drivers. In fact, I recently purchased an ExpressCard to CardBus adapter (one that lets me plug in an expressCard into a CardBus slot) and it worked with all the usb-based ExpressCards that I have laying around with no changes.
The problem with ExpressCard for PCIe-based devices is one of resource allocation. If the BIOS allocates the resource, then the ExpressCard works without hot-plug. If the BIOS doesn't, then we can see the device with pciconf -l, as well as probe the device. However, since there's no resources setup for the PCI bridge, attempts for the driver to allocate them fail. There's also some bus numbering issues as well.
I've started to attack the resource problem. I'll keep people posted
20090421
Update my old CardStatus page...
If anybody still cares about 16-bit PC Cards and 32-bit CardBus cards still, I've updated the latest testing I've done.
You can find it here. I think it does more to show what a big pack-rat I am, but maybe someone will find it useful. It shows I'm down to one or two ed cards that aren't working for me yet...
You can find it here. I think it does more to show what a big pack-rat I am, but maybe someone will find it useful. It shows I'm down to one or two ed cards that aren't working for me yet...
20090413
3Com 3C1 works
Years ago, I bought a 3Com 3C1 CF card. This card was an early attempt by 3Com to minimize the power usage of the traditional 3C589 line of cards. These cards were important for the early palm-sized PDAs that were on the market. Through a number of different connections, I was able to get documentation on the 3c1. At the time I got it, I barely understood network drivers or even how the hardware worked. The documentation I got from 3com had pages for the flow charts for transmit and ISR, but the flowcharts weren't actually included on those pages. My connections couldn't get a better copy of the documentation, always saying it was coming soon.... The descriptions of the commands had some vague hints, but I never could make them connect. This was around 1999 or so.
Over the years, I kept trying to make this card work on FreeBSD. I considered it unfinished business from my past and I kept trying to find time to work on it. I never could find the time, or if I had the time my brain was burned out from the deliverables I had to produce for work. Over the years I've acquired all the cards that I could find in the 3c5xx line and made them work, but never got the 3c1 working.
Recently I stumbled across the 3c1 documentation and had a few minutes to read it. It was a nice change of pace from all the other stuff I'd been working on lately. After reading through it, I realized that there were a few key items that I'd not properly implemented before. There was a cryptic new command to turn on and off the TX PLL. So I had 30 minutes the other night and started reading the document again.
In the interim between when I started looking at the 3c1 documentation and now I did a lot of work in the high precision time and frequency world. This world is all about PLLs and controlling the oscillators/clocks to an insanely high degree of precision. So this clicked in my brain "Hmm, TX PLL sounds like it controls the clock used to do the transmission." So everywhere we enabled TX in the driver, I added a call to turn on TX PLL. This got me started: dhclient worked! I was able to transmit packets. This was the first. However, it stopped working after a while. So I searched through the 3c1 docs. There was nothing about PLL enable and disable, except for the documentation of the command. It appears there's an automatic timeout in the hardware which was turning off the PLL. So, an additional enabling of the PLL in start did the trick. I only enabled the PLL when we're not actively transmitting (which means we were idle for some unknown period of time). I could refine it further if I knew what the timeout of the PLL (which isn't documented), but there appears to be no ill effects doing it all the time the TX unit has been idle.
Ah, one more piece of unfinished business from the past is now complete. I can move on to other more interesting things...
Over the years, I kept trying to make this card work on FreeBSD. I considered it unfinished business from my past and I kept trying to find time to work on it. I never could find the time, or if I had the time my brain was burned out from the deliverables I had to produce for work. Over the years I've acquired all the cards that I could find in the 3c5xx line and made them work, but never got the 3c1 working.
Recently I stumbled across the 3c1 documentation and had a few minutes to read it. It was a nice change of pace from all the other stuff I'd been working on lately. After reading through it, I realized that there were a few key items that I'd not properly implemented before. There was a cryptic new command to turn on and off the TX PLL. So I had 30 minutes the other night and started reading the document again.
In the interim between when I started looking at the 3c1 documentation and now I did a lot of work in the high precision time and frequency world. This world is all about PLLs and controlling the oscillators/clocks to an insanely high degree of precision. So this clicked in my brain "Hmm, TX PLL sounds like it controls the clock used to do the transmission." So everywhere we enabled TX in the driver, I added a call to turn on TX PLL. This got me started: dhclient worked! I was able to transmit packets. This was the first. However, it stopped working after a while. So I searched through the 3c1 docs. There was nothing about PLL enable and disable, except for the documentation of the command. It appears there's an automatic timeout in the hardware which was turning off the PLL. So, an additional enabling of the PLL in start did the trick. I only enabled the PLL when we're not actively transmitting (which means we were idle for some unknown period of time). I could refine it further if I knew what the timeout of the PLL (which isn't documented), but there appears to be no ill effects doing it all the time the TX unit has been idle.
Ah, one more piece of unfinished business from the past is now complete. I can move on to other more interesting things...
20090402
More PC Card enhancements
Every few months/years I take all the cards that I have in my collection and start to run through them testing to see what's working in FreeBSD and what's broken. Over the past year or two I've neglected this task as I've been involved in other things. After my last trip to Akihabara, I've been playing with this stuff, so I thought I'd pull out the cards again. However, for the moment, I'm restricting things to the cards supported by the ed driver.
I've already blogged about the AX88x90 enhancements. The only additional enhancement is that I've eliminated some code that duplicates reading the NIC from what is just memory. I'll be committing that soon.
I've finally implemented the workaround for fullduplex disabling SQE in the DL10019 parts. This gives slightly better performance on the parts that are affected, but I'm having trouble measuring it. I'm not sure that it is worth the hassle.
I've also made a number of cards work by tweaking the MII initialization. I think I have a good way to kick the PHYs that are on these cards to get them working (this has helped to get three cards going).
I'm approaching 100% of the ne-2000ish cards working. I'll do a final cleanup of my patches and try to get them into the tree. It is time to move on to the CardBus cards. Most of them appear to work, once certain resource allocation issues are addressed. At the very least I need to test them all to see how they work. I may skip this and move directly and address the resource issues there.
I've already blogged about the AX88x90 enhancements. The only additional enhancement is that I've eliminated some code that duplicates reading the NIC from what is just memory. I'll be committing that soon.
I've finally implemented the workaround for fullduplex disabling SQE in the DL10019 parts. This gives slightly better performance on the parts that are affected, but I'm having trouble measuring it. I'm not sure that it is worth the hassle.
I've also made a number of cards work by tweaking the MII initialization. I think I have a good way to kick the PHYs that are on these cards to get them working (this has helped to get three cards going).
I'm approaching 100% of the ne-2000ish cards working. I'll do a final cleanup of my patches and try to get them into the tree. It is time to move on to the CardBus cards. Most of them appear to work, once certain resource allocation issues are addressed. At the very least I need to test them all to see how they work. I may skip this and move directly and address the resource issues there.
20090330
AX88x90 updates
After focusing so much time on that old Toshiba card, I thought I'd see if I could get something more modern working as well. I've had one card in my bag knocking around in my bag based on the AX88790 chip. This is the corega FEther II-PXD. I saw it on sale in Akihabara when I was there a few weeks ago, and this card came from Akihabara a few years ago from a friend in Japan. So I thought I'd try it out.
Well, at first it didn't work too well. The MII code was finding gobs of PHYs that were phantoms. This in turned caused bad things to happen when dhclient(8) was run.
There turned out to be three root causes. First, the mii bit-bang code was wrong for reads, yielding bad register reads. Second, the AX88790 has an odd quirk for talking to its internal PHY, so more code was needed to cope with that. Third, the reset code for the NE2000 was also causing problems, so I rewrote it to be AX88x90 reset code. Finally, the internal phy for the AX88790 has a few quirks in silicon that need software workarounds.
Once I'd corrected these problems, the card started working again. Since most of this data is readily available on the net (the asix web site has datasheets for these parts), I'll skip over all the blow-by-blow details. I hopefully had enough comments in the code for people to follow along, as well as the datasheet should make what I've done easy to see.
Now, the arduous task of testing all the AX88x90 cards in my collections... The target card now works correctly, and I believe all of the ones in my collection will work too. Maybe I'll get to the point were all my ed-based PC Cards in my collection will work (except for the Mitsubushi B8895 which I've given up on, and the ones I don't have proper dongles for).
Well, at first it didn't work too well. The MII code was finding gobs of PHYs that were phantoms. This in turned caused bad things to happen when dhclient(8) was run.
There turned out to be three root causes. First, the mii bit-bang code was wrong for reads, yielding bad register reads. Second, the AX88790 has an odd quirk for talking to its internal PHY, so more code was needed to cope with that. Third, the reset code for the NE2000 was also causing problems, so I rewrote it to be AX88x90 reset code. Finally, the internal phy for the AX88790 has a few quirks in silicon that need software workarounds.
Once I'd corrected these problems, the card started working again. Since most of this data is readily available on the net (the asix web site has datasheets for these parts), I'll skip over all the blow-by-blow details. I hopefully had enough comments in the code for people to follow along, as well as the datasheet should make what I've done easy to see.
Now, the arduous task of testing all the AX88x90 cards in my collections... The target card now works correctly, and I believe all of the ones in my collection will work too. Maybe I'll get to the point were all my ed-based PC Cards in my collection will work (except for the Mitsubushi B8895 which I've given up on, and the ones I don't have proper dongles for).
20090327
Toshiba LANCT00A PC Card
When I was in Akihabara last, I got a lot of PC Cards from the junk vendors for 100 Yen. One of these cards was the Toshiba LANCT00A. This card didn't probe when I first got it. FreeBSD didn't support it. This was a challenge.
As long-time reads of this blog may know, I sometimes go out of my way to hack on really oddball hardware that somehow I get it into my head needs to work. Never mind that I have a dozen other projects cooking that is more relevant to FreeBSD and its user base. Never mind that some of this hardware was obsolete when I started hacking on FreeBSD's PC Card layer. The cool thing is that I can find just enough documentation to be interesting on these cards.
This Toshiba card was like that. Google searches for information about it turned up a half dozen posts to different FreeBSD and Linux mailing lists asking about it, plus some speculation about what's in the card. One brave soul even took his card apart and reported it had a NS DP83902 under the hood. However, nobody that I can find ever got this card working under FreeBSD or Linux. Checks of the current Linux kernel was no help. Checks of OpenBSD and NetBSD also yielded no results. There was a windows driver available (actually an NDIS driver of unknown version).
So I was printing out random values for this card. At one time the FreeBSD PC Card probe routine for ed tried to probe WD80x3 cards as well as NE2000 cards. I removed this about 4 years ago after failing to find any cards that needed this, used it, etc. However, this card is almost a WD80x3 clone. I was printing the board type in the attach routine, trying to track down why attach was failing. I saw that it was 20 decimal or 0x14. A quick scan of the if_edreg.h file showed this was one of the Toshiba cards. The comments said it was a PCETC board. I immediately thought "Hmmm, isn't this a Toshiba card?" and then wondered if the 'C' in LANCT00A meant it was the same silicon revision as the 'C' in the PCETC. So I hacked up a quick little test into the if_ed_pccard probe routine.
It recognized it the card when I did that! Success. Well, almost. There was a snag. The memory allocated for this card was at 0x88000000. This didn't pass through the sanity test in the driver designed for ISA cards, so it didn't work. A little more hacking and I was getting a sane MAC address and the card was attaching. It dhcped its address w/o incident.
However, all is not right in mudville. As soon as I started using TCP, I started getting sbdrop panics. It looks like the packets that the WD80x3 code is pumping out aren't quite rightly aligned, or something (I have never had a sbdrop panic, so I'll have to investigate what that means when I have some time). This isn't too terribly surprising, since I haven't tested the ISA cards in years. Maybe something broke for the memory copyout path, but not for the PIO copyout path. I'll have to track that down another night.
[[ UPDATE: It looks like it was the copyin path. There was a bug in it that caused 2x too many bytes to be read into memory, causing the corruption. So now it works well enough for me to update this post and commit the change. ]]
Now, what does this gain FreeBSD? Not much, in the grander scheme of things. I'd be surprised if any of my readers actually had one of these cards and started using it as a result of my work. However, it was a relaxing hour or two of my life. Given my current job stress level, this has great value to me. Also, I think this is one of the first PC Cards to work with both I/O and Memory windows mapped at the same time from the CFE entry. While there are some other cards that manually allocate memory ranges, I don't think any of them have it in their CIS.
Postscript
Well, while I was doing research for this blog entry, I stumbled accross http://linux.toshiba-dme.co.jp/linux/eng/download.htm which has a tos_cs for download. There's a lot of voodoo magic in that driver that I'll have to investigate. This driver is 1000 lines long, and was for Linux 2.0.35. It was never integrated into Linux, so I'd stand by my statement that Linux doesn't support it. Just for kicks I tried building it under 2.6.29, and it didn't compile anymore.
My changes to the ed driver in FreeBSD are more like 20-25 lines of code. There's magic kick of a magic offset in shared memory that isn't present in FreeBSD's ed driver today. I'll keep that in mind in case I have the card wedge on me under load or something.
As long-time reads of this blog may know, I sometimes go out of my way to hack on really oddball hardware that somehow I get it into my head needs to work. Never mind that I have a dozen other projects cooking that is more relevant to FreeBSD and its user base. Never mind that some of this hardware was obsolete when I started hacking on FreeBSD's PC Card layer. The cool thing is that I can find just enough documentation to be interesting on these cards.
This Toshiba card was like that. Google searches for information about it turned up a half dozen posts to different FreeBSD and Linux mailing lists asking about it, plus some speculation about what's in the card. One brave soul even took his card apart and reported it had a NS DP83902 under the hood. However, nobody that I can find ever got this card working under FreeBSD or Linux. Checks of the current Linux kernel was no help. Checks of OpenBSD and NetBSD also yielded no results. There was a windows driver available (actually an NDIS driver of unknown version).
So I was printing out random values for this card. At one time the FreeBSD PC Card probe routine for ed tried to probe WD80x3 cards as well as NE2000 cards. I removed this about 4 years ago after failing to find any cards that needed this, used it, etc. However, this card is almost a WD80x3 clone. I was printing the board type in the attach routine, trying to track down why attach was failing. I saw that it was 20 decimal or 0x14. A quick scan of the if_edreg.h file showed this was one of the Toshiba cards. The comments said it was a PCETC board. I immediately thought "Hmmm, isn't this a Toshiba card?" and then wondered if the 'C' in LANCT00A meant it was the same silicon revision as the 'C' in the PCETC. So I hacked up a quick little test into the if_ed_pccard probe routine.
It recognized it the card when I did that! Success. Well, almost. There was a snag. The memory allocated for this card was at 0x88000000. This didn't pass through the sanity test in the driver designed for ISA cards, so it didn't work. A little more hacking and I was getting a sane MAC address and the card was attaching. It dhcped its address w/o incident.
However, all is not right in mudville. As soon as I started using TCP, I started getting sbdrop panics. It looks like the packets that the WD80x3 code is pumping out aren't quite rightly aligned, or something (I have never had a sbdrop panic, so I'll have to investigate what that means when I have some time). This isn't too terribly surprising, since I haven't tested the ISA cards in years. Maybe something broke for the memory copyout path, but not for the PIO copyout path. I'll have to track that down another night.
[[ UPDATE: It looks like it was the copyin path. There was a bug in it that caused 2x too many bytes to be read into memory, causing the corruption. So now it works well enough for me to update this post and commit the change. ]]
Now, what does this gain FreeBSD? Not much, in the grander scheme of things. I'd be surprised if any of my readers actually had one of these cards and started using it as a result of my work. However, it was a relaxing hour or two of my life. Given my current job stress level, this has great value to me. Also, I think this is one of the first PC Cards to work with both I/O and Memory windows mapped at the same time from the CFE entry. While there are some other cards that manually allocate memory ranges, I don't think any of them have it in their CIS.
Postscript
Well, while I was doing research for this blog entry, I stumbled accross http://linux.toshiba-dme.co.jp/linux/eng/download.htm which has a tos_cs for download. There's a lot of voodoo magic in that driver that I'll have to investigate. This driver is 1000 lines long, and was for Linux 2.0.35. It was never integrated into Linux, so I'd stand by my statement that Linux doesn't support it. Just for kicks I tried building it under 2.6.29, and it didn't compile anymore.
My changes to the ed driver in FreeBSD are more like 20-25 lines of code. There's magic kick of a magic offset in shared memory that isn't present in FreeBSD's ed driver today. I'll keep that in mind in case I have the card wedge on me under load or something.
20090325
Dumpster diving for PC Cards at Akihabara Junk Shops
When I was in Japan two weeks ago, I took a trip to Akihabara and found a few junk shops. Well, these places are just second hand stores for used computer hardware and software. There is quite the range of shops. Some are nice, neat, well lit and have plenty of room to move about. Others aren't quite so nice and are very cramped. The further you get away from the street level and from the main streets, the less polished these stores are. The harder it is for a non-Japanese speaker like myself finds it to do business. This was my first trip to Akihabara that I didn't have Japanese speakers to help me out, so I didn't get too far away from the beaten path. I managed to find about 20 PC Cards at these shops, and spent only 2500 yen for them.
First, the cards that worked. The CyQ've ELA-010 and NEC PC-9801N-J12 both worked with the ed driver without changes to the drivers. The TDK LAC-CD02x worked with the fe driver, again without changes. In a way, these were the disappointing cards in the lot. While I didn't have examples of them in my collection, I also didn't expand FreeBSD's support for PC Cards at all.
I was able to modify the fe driver to support two additional cards that I found. One was the Hitachi HT-4840-11. I had to update the fe driver to select an alternative CFE. What's a CFE? It is the Configuration Function Entry in the CIS. These nodes in the CIS describe the resources that the card will use, as well as control different ways the hardware interacts with the system. The PC Card infrastructure usually selects the proper CFE based on the available resources. However, there are cases like this where some CFEs don't work for whatever reason and alternatives need to be selected. That's what happened here. The HT-4840-11 CFE entries less than 10 appear to be designed to be used in a PC-9801 and PC-9821 laptop. They don't work in my modern HP laptop at all. Entries 10 and above all work fine.
The second fe card I was able to make work was the CONTEC CNETPC. Its first entry had a similar problem to the first few of the Hitachi HT-4840-11, but the rest were good. A quick modification to my earlier modification for the Hitachi and I was supporting both. Well, with one caveat. I can't get this card to get any link with any of the hubs that I have. I don't know if this is because the dongles got mixed up over time and I didn't have a good dongle, or if other parts of the card are 'blown out' or otherwise messed up. I had several cards like this.
Next up was a Toshiba JT Ethernet card. It worked out of the box with the ed driver. However, there was an issue. The MAC address showed up as 00:20:16:ff:ff:ff which seemed odd. Especially since 002016016b31 was labelled on the card. Digging into this deeper showed that the card has the bogus-looking address in the CIS. None of the other standard ways to get a MAC produced anything other than 0's. There's no aux entries in the CIS that could be mined for this data either. Since that's the case, I'm almost convinced that this is working the best it can given the circumstances. Also, there was no good dongle for this card...
There were a number of other boring cards. These were cards that I was able to add to the drivers more easily. The NextComK.K NC5310B Ver1.0 seems to work with the fe driver, but I don't know if the driver I have works or not. Also the MACNICA ME1-JEIDA worked with the ed driver, but the dongle didn't seem to be the right one.
Now, on to the odd-balls.
The NEC PC-9801N-J02R is a card made especially for PC-9801 and PC-9821 laptops. I've never been able to make this work with the smc driver that's in the tree. The card has a sonic chip in it, and it should just work. However, it is extremely limited to where it can be mapped in the I/O space (only at 0x880!), and none of the laptops I've ever had worked with this address.
The TDK LAC-CD011 I found turns out to be rare. At one time they were expensive, but no longer. Sadly, I didn't get dongles for this card. There's sources on the web that document it not working with the fe driver, but that it may have the MB86960A inside, which we might be able to make work. Plus, the NetBSD mbe driver supports the CD010, so I may fiddle with the fe driver to see if I can make this work...
The Toshiba LANCT00A doesn't work out of the box at all. The only information about this card on the net today is that once there was a web page that Toshiba said it was a custom-built card. Chances are good that there will never be a driver for this card.
I'll take a look at the other cards another time. They are even weirder, I fear...
First, the cards that worked. The CyQ've ELA-010 and NEC PC-9801N-J12 both worked with the ed driver without changes to the drivers. The TDK LAC-CD02x worked with the fe driver, again without changes. In a way, these were the disappointing cards in the lot. While I didn't have examples of them in my collection, I also didn't expand FreeBSD's support for PC Cards at all.
I was able to modify the fe driver to support two additional cards that I found. One was the Hitachi HT-4840-11. I had to update the fe driver to select an alternative CFE. What's a CFE? It is the Configuration Function Entry in the CIS. These nodes in the CIS describe the resources that the card will use, as well as control different ways the hardware interacts with the system. The PC Card infrastructure usually selects the proper CFE based on the available resources. However, there are cases like this where some CFEs don't work for whatever reason and alternatives need to be selected. That's what happened here. The HT-4840-11 CFE entries less than 10 appear to be designed to be used in a PC-9801 and PC-9821 laptop. They don't work in my modern HP laptop at all. Entries 10 and above all work fine.
The second fe card I was able to make work was the CONTEC CNETPC. Its first entry had a similar problem to the first few of the Hitachi HT-4840-11, but the rest were good. A quick modification to my earlier modification for the Hitachi and I was supporting both. Well, with one caveat. I can't get this card to get any link with any of the hubs that I have. I don't know if this is because the dongles got mixed up over time and I didn't have a good dongle, or if other parts of the card are 'blown out' or otherwise messed up. I had several cards like this.
Next up was a Toshiba JT Ethernet card. It worked out of the box with the ed driver. However, there was an issue. The MAC address showed up as 00:20:16:ff:ff:ff which seemed odd. Especially since 002016016b31 was labelled on the card. Digging into this deeper showed that the card has the bogus-looking address in the CIS. None of the other standard ways to get a MAC produced anything other than 0's. There's no aux entries in the CIS that could be mined for this data either. Since that's the case, I'm almost convinced that this is working the best it can given the circumstances. Also, there was no good dongle for this card...
There were a number of other boring cards. These were cards that I was able to add to the drivers more easily. The NextComK.K NC5310B Ver1.0 seems to work with the fe driver, but I don't know if the driver I have works or not. Also the MACNICA ME1-JEIDA worked with the ed driver, but the dongle didn't seem to be the right one.
Now, on to the odd-balls.
The NEC PC-9801N-J02R is a card made especially for PC-9801 and PC-9821 laptops. I've never been able to make this work with the smc driver that's in the tree. The card has a sonic chip in it, and it should just work. However, it is extremely limited to where it can be mapped in the I/O space (only at 0x880!), and none of the laptops I've ever had worked with this address.
The TDK LAC-CD011 I found turns out to be rare. At one time they were expensive, but no longer. Sadly, I didn't get dongles for this card. There's sources on the web that document it not working with the fe driver, but that it may have the MB86960A inside, which we might be able to make work. Plus, the NetBSD mbe driver supports the CD010, so I may fiddle with the fe driver to see if I can make this work...
The Toshiba LANCT00A doesn't work out of the box at all. The only information about this card on the net today is that once there was a web page that Toshiba said it was a custom-built card. Chances are good that there will never be a driver for this card.
I'll take a look at the other cards another time. They are even weirder, I fear...
20090319
AsiaBSDCon 2009
I just presented a paper at AsiaBSDCon 2009 on the FreeBSD/mips status. There's a lot going on with the FreeBSD/mips project these days. You can find out most of the details from my paper, which is available in letter or a4 format.
In summary, there's a number of new ports underway. There's Cavium Octeon, Atheros 7xxx/9xxxx, RMI Au1xxxx and a couple others.
AsiaBSDCon is always fun. There's many Japanese developers that don't get a chance to go to other conferences who I only get to see in Japan. This conference allows me to see them, as well as talk to them to see what they are using BSD in general, and FreeBSD specifically. The Japanese have often lead the way in many areas, and seeing their current projects is very interesting.
Also, I could cut and paste large parts of my paper here, but I'm not sure of the value of doing that. Can people that have strong opinions leave feedback on this issue.
In summary, there's a number of new ports underway. There's Cavium Octeon, Atheros 7xxx/9xxxx, RMI Au1xxxx and a couple others.
AsiaBSDCon is always fun. There's many Japanese developers that don't get a chance to go to other conferences who I only get to see in Japan. This conference allows me to see them, as well as talk to them to see what they are using BSD in general, and FreeBSD specifically. The Japanese have often lead the way in many areas, and seeing their current projects is very interesting.
Also, I could cut and paste large parts of my paper here, but I'm not sure of the value of doing that. Can people that have strong opinions leave feedback on this issue.
20090129
Melman Lives!
My son got a toy Melman at McDonalds in a happy meal a while ago. He love the thing. He pounds it on EVERYTHING. When you pound it, Melman says one of three phrases. Recently, however, Melman's voice started to give out. So I thought I'd replace the batteries. Should be easy, right, since those toys are just held together with a couple of screws.
Turns out it wasn't quite so easy. The screws are triangular. This means that a regular screw driver wouldn't work. I tried forcing it with some precision screw drivers I had, but they were all just the wrong size. A few google searches later and I found a few places that have them. The cheapest one sells bits for about $3.00. Currently, there's a catalog link here, but who knows how long it will last. I also found amermeida who sells through amazon that sells a screw driver for the triangle recessed screws for $2.00, but with $5.00 for shipping. I wound up ordering there.
Once the screw driver arrived, I removed the screws. There were L738H cells in place that were drained. I didn't have any of these cells on hand, and it was a cold night, so I raided my watch battery drawer in my shop. I had Energizer 392's. These were about 1-2mm thinner than the L738, but were close enough. I was able to bend the battery holder a bit and they slid right in. I snapped Melman back together, and I had a happy 2-year-old again.
Maybe I'll be able to repair other toys with this screw driver. I hope so, since I don't think the original toy itself cost $7.00. There's a couple of cars that he has which are held together with the same type of screw.
More on FreeBSD/mips another time...
Turns out it wasn't quite so easy. The screws are triangular. This means that a regular screw driver wouldn't work. I tried forcing it with some precision screw drivers I had, but they were all just the wrong size. A few google searches later and I found a few places that have them. The cheapest one sells bits for about $3.00. Currently, there's a catalog link here, but who knows how long it will last. I also found amermeida who sells through amazon that sells a screw driver for the triangle recessed screws for $2.00, but with $5.00 for shipping. I wound up ordering there.
Once the screw driver arrived, I removed the screws. There were L738H cells in place that were drained. I didn't have any of these cells on hand, and it was a cold night, so I raided my watch battery drawer in my shop. I had Energizer 392's. These were about 1-2mm thinner than the L738, but were close enough. I was able to bend the battery holder a bit and they slid right in. I snapped Melman back together, and I had a happy 2-year-old again.
Maybe I'll be able to repair other toys with this screw driver. I hope so, since I don't think the original toy itself cost $7.00. There's a couple of cars that he has which are held together with the same type of screw.
More on FreeBSD/mips another time...
20090127
Alchemy Au1550 status update
I'm in the very early stages of bringing FreeBSD/mips on the Alchemy Au1550. Plat'Home was kind enough to put an OpenMicroServer into my hands for the port. After months of sitting idle, I've started putting the board to use again. I test booted my first kernel tonight. It is still early days, but I thought I'd mention that I'd committed the very preliminary code to the project/mips tree in svn.
As an aside, the FreeBSD/mips main development has moved from the so-called "mips2" tree in perforce to the projects/mips tree in svn. This move allows us to more easily share the work in progress mips ports that people have been asking for in the past. We'll see how well this works out and see if theory matches reality or not.
As an aside, the FreeBSD/mips main development has moved from the so-called "mips2" tree in perforce to the projects/mips tree in svn. This move allows us to more easily share the work in progress mips ports that people have been asking for in the past. We'll see how well this works out and see if theory matches reality or not.
20090125
More Video Creation hacks
I believe that I've blogged before about my video creation on FreeBSD. Here's some interesting tips for DIVX players that I've found.
First, to recap: I tape a lot of hockey games with my DV camera. I use
Recently, I had my DVD player of many years die on me. It refused to play disks. So I went out gone one that would play DivX disks, as well as normal DVDs. This one also has a USB port on it. Through trial and error, I discovered that I could save myself a lot of disks by putting the videos I wanted to test on a flash drive.
DivX videos, for those that don't know, are basically MPEG-4 AVI encoded rather than MPEGE-2 encoded like DVD movies. While DVD movies can play as low as about 4 or 5kbps, they are generally encoded at more like 10kbps. At that rate, the average single layer disc can hold about 2 hours of video. DivX movies are encoded at more like 1-2kbps, so are about 3-6 times smaller than their DVD encoded counterparts. Of course, these are gross generalizations, but good enough for people to understand.
Now, my problem is that I have about 18 hours of hockey games that I'd like to distribute to the parents of the other players. For my son's benefit, I had made one DVD per game, but with 11 other parents, the thought of making 200 DVDs filled me with dread: it was too much time, too much hassle and it was starting to get expensive.
So, I started experimenting with DivX encoded videos. I tried to get ffmpeg working, but ran into a number of snags that I basically punted on. I had used mplayer for years to watch other videos and DVDs, so I thought about using mencoder. kino had a mode that would encode the DV movies to something else, so I played around with it. I created two test videos. One was at 854x480. This is what you get when you film in 16:9 aspect ratio with a DV camera that has a resolution of 720x480. Google for anamorphic for all the reasons for this, but they can be summarized as non-square pixels converted to square pixels. For the curious, I used:
One of these days, I'll need to learn how to do the fancy two-pass encoding to ring every bit of quality I can out of my source material. But for now, I'm doing good to get things encoded. I also need to find some way to tame the insane complexity of this problem. The above commands were basically minor tweaks to ones I found in kino's scripts to encode the video.
As for the DivX disk, I just copied the files over, and they played in alphabetical order. No way I could see to create a menu. No way to encode them in the .divx format that has menus and such in it, since I couldn't find the equivalent to dvdauthor for divx movies (please leave comments if you know of one for FreeBSD/Linux). For the hockey games for the parents, this is likely good enough. For other things, I'd like to do better...
First, to recap: I tape a lot of hockey games with my DV camera. I use
fwcontrol -R name.dv -M dvto read in the tapes. I then use the kino port to do some editing and convert the .dv files into mpeg 2 files for the DVDs of the games I make. I convert the mpeg2 files to dvds using dvdauthor. When I want to be fancy, I use dvdstyler to create themes for the DVDs.
Recently, I had my DVD player of many years die on me. It refused to play disks. So I went out gone one that would play DivX disks, as well as normal DVDs. This one also has a USB port on it. Through trial and error, I discovered that I could save myself a lot of disks by putting the videos I wanted to test on a flash drive.
DivX videos, for those that don't know, are basically MPEG-4 AVI encoded rather than MPEGE-2 encoded like DVD movies. While DVD movies can play as low as about 4 or 5kbps, they are generally encoded at more like 10kbps. At that rate, the average single layer disc can hold about 2 hours of video. DivX movies are encoded at more like 1-2kbps, so are about 3-6 times smaller than their DVD encoded counterparts. Of course, these are gross generalizations, but good enough for people to understand.
Now, my problem is that I have about 18 hours of hockey games that I'd like to distribute to the parents of the other players. For my son's benefit, I had made one DVD per game, but with 11 other parents, the thought of making 200 DVDs filled me with dread: it was too much time, too much hassle and it was starting to get expensive.
So, I started experimenting with DivX encoded videos. I tried to get ffmpeg working, but ran into a number of snags that I basically punted on. I had used mplayer for years to watch other videos and DVDs, so I thought about using mencoder. kino had a mode that would encode the DV movies to something else, so I played around with it. I created two test videos. One was at 854x480. This is what you get when you film in 16:9 aspect ratio with a DV camera that has a resolution of 720x480. Google for anamorphic for all the reasons for this, but they can be summarized as non-square pixels converted to square pixels. For the curious, I used:
mencoder sample.mpeg -cache 8192 -aspect 16:9 -vf harddup,pp=ci,scale=854:480 \to encode the DVD-quality images into divx. This encoding resulted in a factor of 3-4 saving in file sizes. This was fairly good, but didn't get my 40GB of mpeg's to fit on one DVD. In addition, the DVD player I bought didn't like this resolution at all. I figured other people's DVD players wouldn't like it either. Next, I tried a reduced resolution version. I used
-ovc xvid -oac mp3lame -lameopts preset=192 -xvidencopts bitrate=2048 -ffourcc DX50 \
-o high.avi
mencoder - -demuxer rawdv -quiet -cache 8192 -aspect 16:9 -vf harddup,pp=ci,scale=426:240 \which resulted in a space savings more like 6-10, which puts the encoding project within striking distance of one DVD. It is well under the DL DVD range, and might be very close to the single layer limit of 4.7GB. We'll have to see how the encoding goes. It is moving about 4x faster than the previous encoding as well.
-af-adv force=1 -srate 44100 -ovc xvid -oac mp3lame -lameopts preset=medium \
-xvidencopts fixed_quant=4 -ffourcc DX50 -o medium.avi
One of these days, I'll need to learn how to do the fancy two-pass encoding to ring every bit of quality I can out of my source material. But for now, I'm doing good to get things encoded. I also need to find some way to tame the insane complexity of this problem. The above commands were basically minor tweaks to ones I found in kino's scripts to encode the video.
As for the DivX disk, I just copied the files over, and they played in alphabetical order. No way I could see to create a menu. No way to encode them in the .divx format that has menus and such in it, since I couldn't find the equivalent to dvdauthor for divx movies (please leave comments if you know of one for FreeBSD/Linux). For the hockey games for the parents, this is likely good enough. For other things, I'd like to do better...
20090123
Quick Mac OS X hack: Time Machine
I have two macs in my house. One has a 750GB drive on it for backups, plus its normal disk. It had been setup for Time Machine before and was working well. I also have a MacBook Pro that needs to be backed up. I didn't want to swap the disk back and forth between the two, since that would significantly erode the value of Time Machine's automatic backups. I didn't see any place where this was well documented, so I thought a quick note would be helpful to people. Most of the references I've seen have talked about different hacks to make this happen, or that it is only on a specialized Apple product that I didn't have.
I'm running Mac OS X 10.5.6 on both machines.
First, I went to the store and bought a 750GB drive. I plugged it into the mac mini that I have sitting on my desk. I configured it for Time Machine on the Mac Mini. I did a couple of time machine backups.
Next, I exported 750GB disk using the 'Sharing.' I clicked on 'File Sharing' tab to enable it. I clicked on the '+' button under the "Shared Folders" menus. I selected the volume that I had setup for Time Machine. I made sure that I was sharing via "AFP" in the "Options..." dialog.
Once I had things shared this way, I went to the other mac. I went to "Finder" and selected "Go->Connet to Server.." tab. Here I explicitly typed "afp://ipaddr/" and connected. Once I was connected this way, I proceeded to enable Time Machine. The remote disk appeared on the list of choices, so I just selected it and started it up.
I discovered that I have a 100Mbps switch, rather than a GigE switch. This made the initial backup very slow, on the order of 10GB/hr, which means I'm looking at a 10 hour backup time. Maybe it makes sense to head off to staples/best buy to get a GigE switch to plug them both into.
I've not tried to do this wirelessly, so I don't know how painful that will be. I've also not tried to connect this to the Samba server that I have running FreeBSD. I can normally see that volume on the macs without needing to do the manual step.
UPDATE: A GigE switch does help quite a bit. It was more like 6GB/hr with the 100Mbps switch, but with GigE I'm getting about 22GB/hr... So I have about 3 hours left in the backup now... 22GB/hr is about 6MB/s which seems way too slow (the hard disks are limited to about 15MB/s on each side, and the network to 125MB/s, so why the fall off in speed).
I'm running Mac OS X 10.5.6 on both machines.
First, I went to the store and bought a 750GB drive. I plugged it into the mac mini that I have sitting on my desk. I configured it for Time Machine on the Mac Mini. I did a couple of time machine backups.
Next, I exported 750GB disk using the 'Sharing.' I clicked on 'File Sharing' tab to enable it. I clicked on the '+' button under the "Shared Folders" menus. I selected the volume that I had setup for Time Machine. I made sure that I was sharing via "AFP" in the "Options..." dialog.
Once I had things shared this way, I went to the other mac. I went to "Finder" and selected "Go->Connet to Server.." tab. Here I explicitly typed "afp://ipaddr/" and connected. Once I was connected this way, I proceeded to enable Time Machine. The remote disk appeared on the list of choices, so I just selected it and started it up.
I discovered that I have a 100Mbps switch, rather than a GigE switch. This made the initial backup very slow, on the order of 10GB/hr, which means I'm looking at a 10 hour backup time. Maybe it makes sense to head off to staples/best buy to get a GigE switch to plug them both into.
I've not tried to do this wirelessly, so I don't know how painful that will be. I've also not tried to connect this to the Samba server that I have running FreeBSD. I can normally see that volume on the macs without needing to do the manual step.
UPDATE: A GigE switch does help quite a bit. It was more like 6GB/hr with the 100Mbps switch, but with GigE I'm getting about 22GB/hr... So I have about 3 hours left in the backup now... 22GB/hr is about 6MB/s which seems way too slow (the hard disks are limited to about 15MB/s on each side, and the network to 125MB/s, so why the fall off in speed).
20090122
Minor at91 hacking
The recent changes to the SD/MMC stack broke AT91. It seems like there was an assumption that was bogusly baked into the driver. We had assumed that the only data transfer commands would be 512 bytes.
The problem is that the newer stack does commands that are 16 and 64 bytes long. These aren't really needed for basic support of the stack, but to get things like SDHC and high speed transfers working, they are needed.
I've fixed this, and now things work again. I've also taken this opportunity to do some minor cleanups to the at91 mmc/sd driver.
The problem is that the newer stack does commands that are 16 and 64 bytes long. These aren't really needed for basic support of the stack, but to get things like SDHC and high speed transfers working, they are needed.
I've fixed this, and now things work again. I've also taken this opportunity to do some minor cleanups to the at91 mmc/sd driver.
Subscribe to:
Posts (Atom)