tag:blogger.com,1999:blog-65254691918506909572024-03-12T19:10:15.294-07:00persistent.infoMihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.comBlogger486125tag:blogger.com,1999:blog-6525469191850690957.post-30275827295952889342024-02-28T23:13:00.000-08:002024-02-29T21:05:23.694-08:00Infinite Mac: The Case Of The Missing Text<p>I had <a href="https://blog.persistent.info/2023/12/dingusppc.html#dppc-cpu-bugs">previously mentioned</a> that I ended up fixing some <a href="https://github.com/dingusdev/dingusppc">DingusPPC</a> CPU emulation bugs. This was my first time working at this (lower) level of the <a href="https://infinitemac.org/">Infinite Mac</a> emulators, and I thought it would be interesting to write up such an investigation.</p>
<p>There was an issue that was immediately apparent when I first got DingusPPC building: while starting up from the 7.1.2 Disk Tools floppy would result in a functional-looking Finder, doing the same from the 7.1.2 install CD would end up with many rendering glitches — notably no text, but also missing icons.</p>
<table style="margin: 0 auto; table-layout: fixed">
<tr>
<td style="width: 50%; padding: 4px; vertical-align: bottom; text-align: center; font-style: italic">
<img src="https://persistent.info/images/dingusppc-712-floppy.png" width="640" height="480" alt="System 7 Finder window"><br>
7.1.2 Disk Tools Finder
</td>
<td style="width: 50%; padding: 4px; vertical-align: bottom; text-align: center; font-style: italic">
<img src="https://persistent.info/images/dingusppc-712-cd.png" width="640" height="480" alt="System 7 Finder window with no text or icons drawn"><br>
7.1.2 Install CD Finder
</td>
</tr>
</table>
<p>Classic System Software is mostly a closed-source black box (outside of <a href="https://github.com/elliotnunn/supermario/">leaks</a>), and while DingusPPC is open source, it’s a codebase that is still pretty foreign to me. In the absence of other ideas, I figured that creating a more reduced test case would be a good use of time. This reminded me of my <a href="https://issues.chromium.org/issues/41247259#comment3">investigations</a> <a href="https://issues.chromium.org/issues/40597328#comment6">into</a> <a href="https://issues.chromium.org/issues/41203721#comment17">Chromium/WebKit</a> <a href="https://issues.chromium.org/issues/40805884#comment5">rendering bugs</a>, where half the battle was reducing a complex web page down to a minimal test case.</p>
<p>I examined the two system folders and observed that the CD-ROM one was quite a bit larger, with additional extensions, a different enabler¹, and a bigger System suitcase.</p>
<table style="margin: 0 auto; table-layout: fixed">
<tr>
<td style="width: 50%; padding: 4px; vertical-align: top; text-align: center;">
<img src="https://persistent.info/images/dingusppc-712-floppy-system-folder.png" width="346" height="217" alt="System Folder from the 7.1.2 Disk Tools floppy, list view"></td>
<td style="width: 50%; padding: 4px; vertical-align: top; text-align: center;">
<img src="https://persistent.info/images/dingusppc-712-cd-system-folder.png" width="367" height="298" alt="System Folder from the 7.1.2 CD, list view">
</td>
</tr>
</table>
<p>I created a read-write copy of the CD-ROM system folder and used a native build of Basilisk II to operate on it, removing or switching out files one a time to morph it into the floppy version, and looking for any differences when booting it in DingusPPC. It turned out that switching out the “Minimal PowerPC Enabler” on the floppy with the full “PowerPC Enabler” one from the CD-ROM was the key difference — with the full enabler text would not render.</p>
<p>I then opened the two enablers in ResEdit and compared them — the full one had a bunch more resources. I thus repeated the process, deleting extra resources one at a time to morph the full enabler into the minimal one.</p>
<table style="margin: 0 auto; table-layout: fixed">
<tr>
<td style="width: 50%; padding: 4px; vertical-align: top; text-align: center;">
<img src="https://persistent.info/images/dingusppc-minimal-enabler.png" width="410" height="417" alt="ResEdit view of resources in the Minimal PowerPC Enabler"></td>
<td style="width: 50%; padding: 4px; vertical-align: top; text-align: center;">
<img src="https://persistent.info/images/dingusppc-full-enabler.png" width="410" height="483" alt="ResEdit view of resources in the regular PowerPC Enabler">
</td>
</tr>
</table>
<p>It turned out that the culprit was the <code>ntrb</code> resources, specifically removing ID 16 (<code>StdTextTraps</code>) and 18 (<code>QuickDrawTextTraps</code>) would restore text rendering. They contain PowerPC executable code — the giveaway is the <code>Joy!peff</code> prefix that is the <a href="https://web.archive.org/web/20020219190852/http://developer.apple.com/techpubs/mac/runtimehtml/RTArch-91.html">header</a> of the <a href="https://en.wikipedia.org/wiki/Preferred_Executable_Format">Preferred Executable Format</a>. “Traps” refers to the system/<a href="https://en.wikipedia.org/wiki/Macintosh_Toolbox#On_68k_systems">Toolbox</a> calls. My assumption is that the ROM of the first generation of Power Macintoshes was finalized relatively early, and contained mostly 68K code that would need to be emulated². The PowerPC enabler contained native implementations of more of the Toolbox, which were <a href="https://github.com/elliotnunn/supermario/blob/9dd3c4bef84df2ea30f5ec2c5e97b043e8267b3f/base/SuperMarioProj.1994-02-09/OS/StartMgr/Boot3.a#L1436-L1448">loaded in</a> via these <code>ntrb</code> resources. These particular resources had native implementations of <a href="https://developer.apple.com/library/archive/documentation/mac/pdf/Text.pdf">the text subsystem</a>, which was very much in line with where the buggy behavior was.</p>
<p>Between <a href="https://github.com/fuzziqersoftware/resource_dasm">resource_dasm</a> and some help from DingusPPC Discord member <a href="https://github.com/joevt">joevt</a> I was able to get a list of the 14 Toolbox traps that were being reimplemented as native PowerPC versions. The one that immediately stood out was <code>StdTxMeas</code> – here’s a description of it from <a href="https://developer.apple.com/library/archive/documentation/mac/pdf/Text.pdf">Inside Macintosh: Text</a>:</p>
<blockquote>The QuickDraw text routines use two of the bottleneck routines extensively—one to measure text (<code>StdTxMeas</code>) and one to draw it (<code>StdText</code>). Most of the high-level QuickDraw text routines call the low-level routines. The use of bottleneck routines provides flexibility to QuickDraw and applications that need to alter or augment the basic behavior of QuickDraw.</blockquote>
<p>While I had significantly narrowed where things were going wrong, I was reaching the limits of what I could via basic file system and ResEdit hacking. Ideally I would have a simple test program that just called <code>StdTxMeas</code>, so that I could examine its behavior in isolation.</p>
<p>It was thus time to crack open <a href="https://en.wikipedia.org/wiki/CodeWarrior">CodeWarrior</a> and write a simple test program. I used the <a href="https://developer.apple.com/library/archive/samplecode/SillyBalls/Listings/SillyBalls_c.html#//apple_ref/doc/uid/DTS10000735-SillyBalls_c-DontLinkElementID_4">SillyBalls sample code</a> as a starting point — there certainly used to be a lot of boilerplate in getting a basic window up on the screen. I created a simple test function that draws a string and then its bounding box as determined by <code>StdTxMeas</code>. Lacking the ability to render debugging text, I used <code>FrameRect</code> as the output mechanism.<p>
<pre class="prettyprint">
void TestStdTxMeas(WindowPtr mainWindowPtr) {
Rect measureRect;
Str255 testString = "\pHello World";
Point number = {1, 1};
Point denom = {1, 1};
FontInfo fontInfo;
int width;
SetPort(mainWindowPtr);
MoveTo(20, 20);
TextSize(12);
DrawString(testString);
width = StdTxMeas(testString[0], testString + 1, &number, &denom, &fontInfo);
// +1 for the width so that if we get a zero we can still see the rect.
SetRect(&measureRect, 20, 8, 20 + width + 1, 24);
FrameRect(&measureRect);
}</pre>
<p>I set the program as the startup item (so that I wouldn’t have to navigate blindly in a half-drawn Finder window) and observed its behavior under both emulated and native implementation of the text traps.</p>
<table style="margin: 0 auto; table-layout: fixed">
<tr>
<td style="width: 50%; padding: 4px; vertical-align: top; text-align: center;">
<img src="https://persistent.info/images/dingusppc-test-correct.png" width="253" height="271" alt="QuickDrawTester window showing expected output of "Hello World" surrounded by a rectangle"></td>
<td style="width: 50%; padding: 4px; vertical-align: top; text-align: center;">
<img src="https://persistent.info/images/dingusppc-test-incorrect.png" width="253" height="271" alt="QuickDrawTester window showing incorrect output with just a black line">
</td>
</tr>
</table>
<p>Sure enough, it looked like when using native PowerPC code, <code>StdTxMeas</code> would end up returning 0 instead of the expected value. I suspected a bug in DingusPPC’s PowerPC CPU emulation, but it was unclear where. I had some false starts:</p>
<ol>
<li>I tried to disassemble the <code>StdTxMeas</code> implementation – I got lost when going through too many layers of indirection due to the <a href="https://orangejuiceliberationfront.com/universal-procedure-pointers">calling convention</a>.</li>
<li>I looked at its <a href="https://github.com/elliotnunn/CubeE/blob/1c8bd4ce45620ccc2df0426a6ba43d31a62f5e37/QuickDraw/Text.a#L1140-L1362">68K implementation</a> and checked some of the intermediate building blocks – the <a href="https://github.com/jeeb/mpc-be/blob/master/include/qt/Quickdraw.h#L7672C6-L7672C20">width table</a> looked correct.</li>
<li>I tried to get <a href="https://en.wikipedia.org/wiki/MacsBug">MacsBug</a> running so that I could single-step through it – I was not able to get it to succesfully suspend execution under DingusPPC.</li>
</ol>
<p>As a variant of attempt 3, I decided to make DingusPPC output the instruction stream that it was executing, hopefully there would be a clue where the problem was. To <a href="https://www.swiftbysundell.com/wwdc2018/getting-started-with-signposts/">signpost</a> the instructions that represented the implementation of <code>StdTxMeas</code>, I did two otherwise no-op divisions by specific literals (the <code>MoveTo</code> call is to ensure that the compiler didn’t decide to optimize them out).</p>
<pre class="prettyprint">
GetPen(&pen);
pen.h = pen.h / 32749;
width = StdTxMeas(testString[0], testString + 1, &number, &denom, &fontInfo);
pen.v = pen.v / 32719;
MoveTo(pen.h, pen.v);</pre>
<p>Checking the CodeWarrior dissembly I could see the expected <code><a href="https://stackoverflow.com/a/11147432">li</a></code> (load immediate) and <code><a href="https://www.ibm.com/docs/en/aix/7.3?topic=set-divw-divide-word-instruction">divw</a></code> (divide word) op codes:</p>
<pre class="prettyprint">
// pen.h = pen.h / 32749;
000000A0: A861003A lha r3,58(SP)
000000A4: 38807FED li r4,32749
000000A8: 7C6323D6 divw r3,r3,r4
000000AC: B061003A sth r3,58(SP)</pre>
<p>I could thus look for the 32,749 divisor in DingusPPC’s <a href="https://github.com/dingusdev/dingusppc/blob/0c3f399de3e7198616f555b5378eac1fb69ae692/cpu/ppc/ppcopcodes.cpp#L619-L646">divw implementation</a> and then set a flag to trigger its <a href="https://github.com/dingusdev/dingusppc/blob/master/cpu/ppc/ppcdisasm.h">built-in disassembler</a> (and turn it off when the divison by 32,719 occurred). </p>
<p>Out of this I had a list of 635 instructions that were executed, which was a lot, but still made for a starting point. Looking at counts of the types of instructions, there were 59 unique ones, with <a href="https://www.ibm.com/docs/en/aix/7.3?topic=set-lwz-l-load-word-zero-instruction">loads</a> and <a href="https://www.ibm.com/docs/en/aix/7.3?topic=set-stw-st-store-instruction">stores</a> being the most common. I figured that the most popular instructions were least likely to be buggy, otherwise more things would have been broken. Looking at the bottom of list, two stood out: <code><a href="https://www.ibm.com/docs/en/aix/7.3?topic=set-nabs-negative-absolute-instruction">nabs</a></code> (negative absolute) and <code><a href="https://www.ibm.com/docs/en/aix/7.3?topic=set-doz-difference-zero-instruction">doz</a></code> (difference or zero). They both had no coverage in DingusPPC’s <a href="https://github.com/dingusdev/dingusppc/tree/master/cpu/ppc/test">test suite</a>³ and were obscure — they are actually part of the POWER instruction set that was the predecesor of PowerPC, and were only <a href="https://www.ibm.com/docs/en/aix/7.3?topic=programs-power-family-instructions-deleted-from-powerpc">included in the PowerPC 601</a> chip to help with IBM’s migration to it.</p>
<p>Looking at <a href="https://github.com/dingusdev/dingusppc/blob/35c86ad6bf99ef88f0db78bc89cd3f8c27f3af0b/cpu/ppc/poweropcodes.cpp#L226C5-L226C48">the implementation of <code>nabs</code></a>, it seemed relatively straightforward — it would always set the sign bit to to <code>1</code>, thus making the number always negative:</p>
<pre class="prettyprint">ppc_result_d = (0x80000000 | ppc_result_a);</pre>
<p>However, negative numbers are not actually represented as positive numbers with the sign bit set, they use <a href="https://en.wikipedia.org/wiki/Two%27s_complement">two’s complement</a>. Using <a href="https://jvns.ca/">Julia Evans</a>’s handy <a href="https://jvns.ca/">integer.exposed</a> we can see the representation of <a href="https://integer.exposed/#0x0000007b">123</a>, <a href="https://integer.exposed/#0xffffff85">-123</a> and <a href="https://integer.exposed/#0x8000007b">123 with its sign bit flipped</a>, and thus why this implementation is not right.</p>
<p>I switched it out to the more readable explicit sign test and negation:<br/>
<pre class="prettyprint">ppc_result_d = ppc_result_a & 0x80000000 ? ppc_result_a : -ppc_result_a;</pre>
<p>Things still weren’t working, but as I had previously mentioned, <code>doz</code> was also suspicious (and it was the instruction that was executed right before <code>nabs</code>). I noticed a copy/paste error in which register the result was being stored in, and once I <a href="https://github.com/dingusdev/dingusppc/commit/1b4de3b64eba97694813476aa23f9313ec343379">fixed</a> that, everything began to work!</p>
<p style="text-align: center">
<img src="https://persistent.info/images/dingusppc-712-cd-working.png" width="640" height="480" alt="Correctly rendered System 7.1.2 Finder, showing the contents of the "Power Macintosh CD" drive">
</p>
<p>This whole investigation took about a week’s worth of evening hacking time, and there was a bit of luck involved in the end. I very easily could have ended up on a wild goose chase looking at some of the other infrequent instructions, and given up before I got to <code>nabs</code> and <code>doz</code>. Or the implementation bugs could have been much more subtle. However, it worked out, and it was really satisfying to be able to <a href="https://github.com/dingusdev/dingusppc/pull/67">contribute</a> to the DingusPPC core.</p>
<ol style="border-top: solid 1px #aaa; padding-top: 3px;" class="footnotes">
<li><a href="https://web.archive.org/web/20091213155035/https://support.apple.com/kb/TA28948">System enablers</a> were used to patch in support for Macintosh models released after a base version of the OS shipped.</li>
<li>Much of the Toolbox code was written in 68K assembly and so could not be easily ported when Apple made the 68K-to-PowerPC transition in 1994. <a href="https://en.wikipedia.org/wiki/Mac_68k_emulator">Emulation</a> was thus the solution not just for older software, but also for large parts of the OS too. It was not until the transition to Mac OS X in 2001 that the need for 68K emulation went away.</li>
<li>The DingusPPC test suite is partially based on <a href="https://github.com/lioncash/DolphinPPCTests">the one</a> for the <a href="https://github.com/lioncash/DolphinPPCTests">Dolphin</a> emulator. Since that emulates a G3-class CPU, it does not need to worry about the quirks of first-generation PowerPC CPUs.</li>
</ul>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com1tag:blogger.com,1999:blog-6525469191850690957.post-47554667977785297102023-12-13T08:30:00.000-08:002024-02-25T11:43:50.122-08:00Infinite Mac: DingusPPC Explorations<p>One of my goals for <a href="https://infinitemac.org/">Infinite Mac</a> is to learn more about computer architecture and other fundamentals. While fiddling with actual old hardware is not as interesting to me (the one classic Mac I own is a PowerBook 550c that I turn on about once a year), I do enjoy operating at lower levels of the stack than I normally encounter. Interacting with the emulators that I’ve ported to Emscripten has definitely exposed me to more of this world, but it’s still been pretty superficial. Even the <a href="https://blog.persistent.info/2023/01/infinite-mac-2022-in-review.html#basilisk-ii-fpu">FPU fidelity tweaks</a> from last year amounted to just changing some build flags once I understood what was going on. </p>
<p>That being said, I’m not sure that spending more time with the current set of emulators would be the best use of my time. Basilisk II, SheepShaver and Mini vMac are mature (old enough <a href="https://www.gryphel.com/c/news/nva0.html#n010228">to drink</a> or <a href="https://github.com/cebix/macemu/commit/8e491572ca420b22ff0366e594ce5b9248928e91">rent a car</a>) codebases. That in and of itself is not bad, but the original authors are gone, the vision has drifted, they have fragmented into <a href="https://github.com/kanjitalk755/macemu/">forks</a> or states of <a href="https://www.emaculation.com/forum/viewtopic.php?t=11570">semi-abandonment</a> and have fundamental limitations (<a href="https://github.com/mihaip/macemu/blob/infinite-mac-kanjitalk755/BasiliskII/src/rom_patches.cpp#L682-L702">aggressive ROM patching</a> and <a href="https://www.emaculation.com/forum/viewtopic.php?t=11428">lack of an MMU</a>).</p>
<p>With that in mind, I went looking for other options. The <a href="https://docs.google.com/spreadsheets/d/1us6SCBgVs8NqbxofJXTmHDeK3nKQJpcgya2nWC9_t2w/edit">Mac Emulation Matrix</a> spreadsheet lists many choices, and I was particularly interested in options that would eventually allow (early) Mac OS X to be runnable. The obvious choice was <a href="https://github.com/mihaip/infinite-mac/issues/72">QEMU</a> – it has very broad guest OS support and is very actively developed. However, it’s also a beast of a project to build and navigate around; it didn’t seem like it would be something I would able to make much progress on while working on it for a few hours a week. There is also some work on <a href="https://github.com/mamedev/mame/commits/master/src/mame/apple/macpdm.cpp">Power Macintosh support</a> in MAME, but it’s a similar large codebase, and it seemed to be <a href="https://www.emaculation.com/forum/viewtopic.php?p=73012&hilit=mame#p73012">much further behind QEMU</a>, compatibility-wise. I also briefly considered <a href="https://github.com/sebastianbiallas/pearpc">PearPC</a> (which is much more focused on early PowerMacs than QEMU), but it’s also in a state of abandonment (development mostly stopped in 2005, with a brief resurrection in 2015).</p>
<p>I then came across <a href="https://github.com/dingusdev/dingusppc">DingusPPC</a>, development of which has been going on for a few years (chronicled in this <a href="https://www.emaculation.com/forum/viewtopic.php?t=10263">Emaculation thread</a> and these <a href="https://www.reddit.com/r/emulation/search/?q=DingusPPC&restrict_sr=1&sort=new">Reddit posts</a>). It had picked up steam recently, getting to a <a href="https://www.emaculation.com/forum/viewtopic.php?p=74449#p74449">somewhat usable state</a>, with some hope of <a href="https://www.emaculation.com/forum/viewtopic.php?p=76602#p76602">booting Mac OS X</a> too. The codebase was much smaller than QEMU’s (as was the resulting binary - 1.5MB vs. 17MB) and seemed reasonably modern. I crossed my fingers that this was <a href="https://marc.info/?l=kfm-devel&m=104197092318639&w=2">a KHTML-vs.-Gecko-in-2003 decision</a> and decided to give DingusPPC a try.</p>
<p>My first task was figuring out what worked under DingusPPC — it emulates <a href="https://github.com/dingusdev/dingusppc/tree/master/machines">lots of machines</a> but some are more fleshed out than others. I settled on the <a href="https://github.com/dingusdev/dingusppc/blob/master/machines/machinepdm.cpp">Power Macintosh 6100</a> running System 7.1.2 (via the Disk Tools floppy) both because it seemed to have had the most attention from the developers and because it’s a combination that’s unsupported by SheepShaver or QEMU (both of which emulate PCI-era Macs). Once I had a working native build and machine/OS configuration, I started to work on getting it to build under Emscripten. DingusPPC doesn’t have much cross-platform support beyond what’s provided for free by SDL, and SDL’s Emscripten compatibility layer does not work for Infinite Mac (it assumes direct DOM access is possible from the generated code, but I run all emulators in a worker for better performance). I <a href="https://github.com/dingusdev/dingusppc/commit/1f7edfdb3b9f513ab5e4271aedda85a65f7ee299">ended up</a> with an approach inspired by <a href="https://www.chromium.org/developers/design-documents/conventions-and-patterns-for-multi-platform-development/">Chromium</a>’s, where some classes have per-platform implementations which are included automatically based on filename suffixes.</p>
<p>Usually when bringing up a new emulator I implement video output first, because it makes debugging much easier — if the ROM is being read correctly I should at least see a happy or sad Mac. I <a href="https://github.com/mihaip/dingusppc/commit/d7469d49d388b13c4a374e4e1f633b880af6cf2b">did the same thing</a> with DingusPPC (DPPC from here on out), but I was disappointed to end up with a black screen. </p>
<p style="text-align: center">
<img src="https://persistent.info/images/pdm-black-screen.png" width="640" height="480" alt="Black screen when tryig to boot DingusPPC"><br>
<i>Black screens - it’s not just for graphics programmers.</i>
</p>
<p>To get a high-level overview of what was happening, I enabled DPPC’s <a href="https://github.com/dingusdev/dingusppc/blob/01c031284f25cebce35ddcca17b3e647afbfd49a/cpu/ppc/ppcemu.h#L338-L345">CPU profiling support</a> and added a periodic dump when <a href="https://github.com/mihaip/dingusppc/blob/dbc22411389f0220f022900a666dad732418df44/core/hostevents_js.cpp#L27-L32">host events were read</a> (i.e. every 11ms). I could then compare the counters when booting the same configuration in both the native and Emscripten builds, and see where the divergence occurred. It became apparent that after around 300ms the Emscripten build was stuck in a loop, since the number of supervisor instructions remained flat, while it would occasionally increase in the native one.</p>
<p>I was stumped for a while until I increased the logging level to maximum verbosity, and noticed that a <code>AMIC: Sound out DMA control set to 0x0</code> <a href="https://github.com/dingusdev/dingusppc/blob/01c031284f25cebce35ddcca17b3e647afbfd49a/devices/ioctrl/amic.cpp#L533">line</a> was logged right before the divergence started. I had seen that the native build <a href="https://github.com/dingusdev/dingusppc/blob/fd92d86954835777b09d80845f711f52278253ae/devices/sound/soundserver_cubeb.cpp#L185">starts a background thread</a> to <a href="https://github.com/dingusdev/dingusppc/blob/fd92d86954835777b09d80845f711f52278253ae/devices/sound/soundserver_cubeb.cpp#L120">pull data from the sound DMA channel</a>, but I had left all of the sound support in the Emscripten build <a href="https://github.com/mihaip/dingusppc/blob/dbc22411389f0220f022900a666dad732418df44/devices/sound/soundserver_js.cpp">stubbed out</a>, since it’s usually something that I tackle much later. In this case sound turns out the load-bearing — if the DMA channel does not get drained, the boot process hangs. I added <a href="https://github.com/mihaip/dingusppc/commit/f476d1fc91705a3b8e2f00b8c218d8e4906c40aa">a basic polling loop</a> to read from the DMA channel and send it to the JS side, and I got the flashing question mark I was expecting:</p>
<p style="text-align: center">
<img src="https://persistent.info/images/pdm-flashing-question-mark.png" width="640" height="480" alt="Flashing question mark boot screen">
</p>
<p>The reason why the flashing question was expected is because DPPC had no way to read the startup floppy disk image — it needs to be bridged to the <a href="https://blog.persistent.info/2023/08/infinite-mac-cd-roms.html">JS APIs</a> that handle streaming of content over the network. I <a href="https://github.com/dingusdev/dingusppc/commit/d4c9db7fcf6d944c6af864c68be5c6d38128ce89">moved</a> all of DPPC’s disk image reading behind an abstraction, <a href="https://github.com/mihaip/dingusppc/commit/4d357dffb5e08153c5be1030f84c2779d1317b83">added</a> a JS-backed implementation, and then I had a successful boot from the floppy.</p>
<p style="text-align: center">
<img src="https://persistent.info/images/pdm-welcome.png" width="640" height="480" alt="Welcome to Power Macintosh startup screen"><br>
<i>System 7.1.2 was the only release to say “Welcome to Power Macintosh” when starting up one of those machines.</i>
</p>
<p>Once I had the system booting, input was the next task. I was able to send mouse updates pretty easily, but it became immediately apparent that the position was handled differently. Other emulators I’ve ported operate in terms of absolute coordinates, but DingusPPC pretends to have a physical mouse attached, and only sends deltas to the guest OS. While <a href="https://github.com/mihaip/infinite-mac/commit/6f87ed4d9a2a2591ad6641e2ba06faa7a3a240ad">adding support for deltas</a> was easy, I ran into the issue that the guest OS treats them as raw inputs, and then computes an extra acceleration curve based on them, thus movements were not 1:1. For now I chose to <a href="https://github.com/mihaip/infinite-mac/commit/228e8ca022d98c15c4be25b5714ccdc0704a60e2">implement pointer lock</a> support (another case where <a href="https://developer.mozilla.org/en-US/docs/Web/API/Pointer_Lock_API">a modern web API</a> makes things easier), but the likely better long term fix is to make DingusPPC support the extended ADB protocol, which includes the ability to send absolute coordinates (originally meant for graphics tablets).</p>
<p>I then went to make keyboard input work, which seemed <a href="https://github.com/mihaip/dingusppc/commit/f465669c024e3eee95ca274d5be0270a13311667">easy enough</a>, but it didn’t seem to work. A quick check in the native build showed that it wasn’t working there either, because the ADB keyboard implementation was <a href="https://github.com/dingusdev/dingusppc/blob/cd9ccb66ed70548a491f696b6dba45cca5a072b2/devices/common/adb/adbkeyboard.cpp#L47-L49">actually a stub</a>. I figured this was a good first contribution to upstream to the DPPC project, so I began to spend some quality time with <a href="https://web.archive.org/web/20210321183851/https://developer.apple.com/library/archive/technotes/hw/hw_01.html">tech note HW01</a> and <a href="https://developer.apple.com/library/archive/documentation/mac/pdf/Devices/ADB_Manager.pdf">the relevant chapter</a> of Inside Macintosh. The protocol was straightforward enough, but it took me a while to catch on to the fact that there was <a href="https://github.com/dingusdev/dingusppc/commit/73272b28ddbed7f567f56e1da6f9aac533da2ca1#diff-3695c5571a108889aa80b91cdd4c6689541f0561e52bd5a9f5ebdb1aa0a556e4">a separate polling loop</a> that needed to be triggered. I did eventually <a href="https://github.com/dingusdev/dingusppc/pull/56">succeed</a>, and the keyboard now works in both the native and Infinite Mac builds of DPPC (though I did have to make some <a href="https://github.com/dingusdev/dingusppc/commit/d45bba924d728931a5b1de4c50195eda865b7b25">followup</a> <a href="https://github.com/dingusdev/dingusppc/commit/d08b486db0a060a4da3885cd46ed1489bb805598">fixes</a>).</p>
<p style="text-align: center">
<img src="https://persistent.info/images/pdm-keyboard.png" width="640" height="480" alt="Key Caps desk accessory">
</p>
<p>During all of this development I observed that DingusPPC was extremely slow (as in, more than 3 minutes to go from boot to the Finder running). While this could be attributed to its CPU emulator being a pure interpreter and the fact that it more accurately emulates the timings of a floppy drive, the Emscripten build was particularly slow. I ran a profile and was surprised to see that a lot of time was being spent going between JavaScript and WebAssembly.</p>
<table style="margin: 0 auto; table-layout: fixed">
<tr>
<td style="width: 50%; padding: 4px; vertical-align: bottom; text-align: center; font-style: italic">
<img src="https://persistent.info/images/dingusppc-wasm-profile1.png" width="425" height="124" alt="Bottom-profile results showing top functions"><br>
Top Functions
</td>
<td style="width: 50%; padding: 4px; vertical-align: bottom; text-align: center; font-style: italic">
<img src="https://persistent.info/images/dingusppc-wasm-profile2.png" width="425" height="199" alt="Flame graph showing call stacks"><br>
Sampled call stacks
</td>
</tr>
</table>
<p>While I do end up calling to JavaScript from the input, disk and display implementations, those hops are relatively rare. These were happening in the core CPU emulation loop, and thus every instruction. The <code>js-to-wasm</code> and <code>wasm-to-js</code> functions and the bouncing back-and-forth is <a href="https://emscripten.org/docs/porting/setjmp-longjmp.html">done by Emscripten</a> to implement <code>setjmp</code>/<code>longjmp</code> support, and DPPC uses those to <a href="https://github.com/dingusdev/dingusppc/blob/0e0de638d4f493d7ecfea6976f0b709d32b94673/cpu/ppc/ppcexceptions.cpp#L134-L136">implement exceptions</a> in the CPU. While there is <a href="https://emscripten.org/docs/porting/setjmp-longjmp.html#webassembly-exception-handling-based-setjmp-longjmp-support">an option</a> to use native Wasm exceptions, it’s only supported by relatively modern browsers, and I try be pretty conservative about which browsers are supported (the non-SharedArrayBuffer <a href="https://blog.persistent.info/2021/08/worker-loop.html">fallback mode</a> is still being used by 5% of users). </p>
<p>I made an attempt at retrofitting DPPC to not use <code>setjmp</code>/<code>longjmp</code>, but it became extremely invasive due to the need to abort execution in all of the codepaths that exceptions could happen. I then took a closer look at the stacks that that the profiler showed and noticed that <a href="https://github.com/dingusdev/dingusppc/blob/0e0de638d4f493d7ecfea6976f0b709d32b94673/cpu/ppc/ppcexec.cpp#L322-L324">ppc_exec_inner</a> was not in them — it was getting inlined into the <a href="https://github.com/dingusdev/dingusppc/blob/0e0de638d4f493d7ecfea6976f0b709d32b94673/cpu/ppc/ppcexec.cpp#L384-L396">ppc_exec outer loop</a>, which is also where the <code>setjmp</code> call was. I had a hunch that forcing the function to not be inlined would change when the JS ↔ WASM hops would happen, and I was right — they only happened once when starting the main loop, and <a href="https://github.com/mihaip/dingusppc/commit/c1e06dc31e70a96239c875b5a55e9b0ecfd38ab1">boot times became 2.5 times faster</a> with this 1-line change.</p>
<p>All my work until this point had been from floppy or CD-ROM images, but I was hoping to make hard drive images work too. It turned out that DPPC expects full device images (with a partition map), whereas I had previously operated only on bare HFS partitions/disk images. Coincidentally around this time the creator of <a href="https://bluescsi.com/">BlueSCSI</a> filed an issue <a href="https://github.com/mihaip/infinite-mac/issues/234">asking for device image support</a>, and had <a href="https://github.com/mihaip/infinite-mac/issues/234#issuecomment-1733682362">some technical pointers</a>. I was able to <a href="https://github.com/mihaip/infinite-mac/commit/bfc5b1920a6c11e200ab43a6fd4c81c147a72ab0">automatically wrap</a> my existing images and have them work for both BlueSCI uses and as boot devices for DPPC. Unfortunately DPPC’s SCSI hard drive support is still nascent, so while the hard drive boot process begins, it’s not able to complete.</p>
<p>The current status is that there are a few experimental DPPC-based machines that can be run, currently only accessible via the <a href="https://infinitemac.org/run">custom instance</a> option. Some notable combinations:</p>
<ul>
<li><a href="https://infinitemac.org/run?disk=System+7.1.2+Disk+Tools">Power Macintosh 6100 with 7.1.2 Disk Tools floppy</a></li>
<li><a href="https://infinitemac.org/run?cdrom=https%3A%2F%2Farchive.org%2Fdownload%2FPowerMac_System_Disc_MacOS_7.1.2%2FPowerMac_System_Disc_MacOS_7.1.2.iso&machine=Power+Macintosh+6100">Power Macintosh 6100 with 7.1.2 CD</a></li>
<li><a href="https://infinitemac.org/run?disk=Mac+OS+8.1+Disk+Tools+%28PPC%29&machine=Power+Macintosh+G3+%28Beige%29">Power Macintosh G3 (Beige) with 8.1 Disk Tools floppy</a></li>
<li><a href="https://infinitemac.org/run?cdrom=https%3A%2F%2Farchive.org%2Fdownload%2Fmac-os-8.1-b97050-315a-1998%2FMac%2520OS%25208.1.bin&machine=Power+Macintosh+G3+%28Beige%29">Power Macintosh G3 (Beige) with 8.1. install CD</a></li>
<li><a href="https://infinitemac.org/run?cdrom=https%3A%2F%2Farchive.org%2Fdownload%2Fosx_100_4k78_install%2Fosx_100_4k78_install.iso&machine=Power+Macintosh+G3+%28Beige%29">Power Macintosh G3 (Beige) with 10.0 install CD</a></li>
<li><a href="https://infinitemac.org/run?cdrom=https%3A%2F%2Farchive.org%2Fdownload%2Fapple-mac-os-x-10.2.0-build-6c115%2Fdisk1.iso&machine=Power+Macintosh+G3+%28Beige%29">Power Macintosh G3 (Beige) with 10.2 install CD</a></li>
</ul>
<p id="dppc-cpu-bugs">Only the first three boot successfully, but it’s still pretty satisfying to to have gotten this far. Code-wide, I still have <a href="https://github.com/mihaip/dingusppc">my own fork</a>, but I’ve been able to upstream <a href="https://github.com/dingusdev/dingusppc/commits/master?author=mihaip">20 or so changes</a>, including quite a few that help the native build too. I hope to blog about the CPU emulation ones soon — they do scratch my itech of learning more about computer internals. The <a href="https://github.com/mihaip/infinite-mac/issues/219">DPPC tracking issue</a> has some notes on remaining work, and I’ll be rebasing my fork as the project makes progress.</p>
<p>On the non-DingusPPC front, I’ve made a few small improvements to Infinite Mac:</p>
<ul>
<li>The aforementioned device image support means that it’s possible to import and export .hda images for use with BlueSCSI and other physical devices, here’s <a href="https://www.youtube.com/watch?v=2KCb7tuE9Io">a demo</a> from a user that tried it.</li>
<li>I added custom display size support (for Basilisk II and SheepShaver-based machines), so that you can <a href="https://infinitemac.org/1991/System%207.0?screenSize=640x870">pretend to have Portrait Display</a> or have the Mac take up your <a href="https://infinitemac.org/1996/System%207.5.3?screenSize=window">entire window</a> or <a href="https://infinitemac.org/1998/Mac%20OS%208.1?screenSize=fullscreen">screen</a>.</li>
<li>I kept running into <a href="https://github.com/mihaip/infinite-mac/issues/247">instability</a> with the Infinite HD drive under System 6, seemingly due to a corrupt desktop database. The simplest solution turned out to be to <a href="https://github.com/mihaip/infinite-mac/commit/60f23d373a512fcb85dead5d391389c33db746c9">generate</a> a smaller disk image for older OSes (since a lot of 90s-era software didn’t run on them anyway). Chances are the Finder from this era was not really set up for a 2GB disk with tens of thousands of files.</li>
</ul>
<p>Finally, I would remiss if I didn’t mention <a href="https://software.inc/">Software Applications Incorporated’s website</a>, which takes Infinite Mac in a very interesting direction.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com2tag:blogger.com,1999:blog-6525469191850690957.post-55080273077029044262023-09-14T08:33:00.001-07:002023-11-12T22:09:08.899-08:00Infinite Mac: Improved Persistence<p><a href="https://infinitemac.org/">Infinite Mac</a> has supported a limited form of persistence since <a href="https://blog.persistent.info/2022/03/blog-post.html#infinite-mac-persistence">its initial launch</a>. This was <a href="https://github.com/mihaip/infinite-mac/commit/0899205dba8fc3104f6a4f65a1714816720fe67b">done</a> by exporting the contents of the “Saved” folder in “The Outside World” to IndexedDB when the emulator was shut down. While this worked, it had several limitations:</p>
<ul>
<li>Saving was best-effort during the page unload process. The <a href="https://developer.chrome.com/blog/deprecating-unload/#:~:text=The%20beforeunload%20event,most%20unload%20replacements."><code>beforeunload</code> event</a> that it relied on does not always fire, and it required an asynchronous <code>postMessage</code> step by which point the process for the page may have been terminated.</li>
<li>It relied on Basilisk II and SheepShaver’s ExtFS support (which creates a virtual drive using the <a href="https://www.macintoshrepository.org/2070-file-system-manager-1-2-sdk">File System Manager</a>). This meant that it was not available in Mini vMac-based emulators (i.e. anything for System 6 or earlier).</li>
<li>Even when ExtFS/File System Manager is available, the virtual drive has several limitations: some software thinks it’s a networked drive and refuses to run on it, and it’s not bootable.</li>
</ul>
<p>With the <a href="https://blog.persistent.info/2023/08/infinite-mac-cd-roms.html">improved disk abstractions</a> that enabled remote CD-ROM support, I <a href="https://github.com/mihaip/infinite-mac/issues/152">realized</a> that that I could mount another hard disk image in the emulated Mac, intercept <code>read</code> and <code>write</code> calls to it, and handle them from a persisted store. This was made possible by the new-ish <a href="https://developer.mozilla.org/en-US/docs/Web/API/File_System_API/Origin_private_file_system">origin private file system</a> (OPFS) API, and specifically the ability to do <a href="https://developer.mozilla.org/en-US/docs/Web/API/FileSystemFileHandle/createSyncAccessHandle">synchronous file operations</a> from a worker (without <a href="https://jlongster.com/future-sql-web">heroics</a>). It’s supported in all modern browsers, including <a href="https://webkit.org/blog/12257/the-file-system-access-api-with-origin-private-file-system/">WebKit-based ones</a>. The synchronous part is key because it allows all changes to be persisted as they happen, instead of during page unload.</p>
<p>The specific approach <a href="https://github.com/mihaip/infinite-mac/commit/717e26ceb83f5f1b4886521ab1cc1ca9466eaec9">that I settled on</a> was to have a mostly empty 1GB HFS disk image that serves as a “Saved HD”. Any modifications to it (tracked using the same 256KB chunk granularity that’s used for disk image streaming) are persisted in an OPFS file (a second file maintains the indexes of all modified chunks). The “mostly” empty part is because there is some metadata (desktop database, custom icon) as well as a Read Me file that are in the initial state of the disk image. This system makes the actual OPFS space that’s consumed to be proportional to the amount of data written, instead of ballooning to 1GB from the get go. It could also be extended to support a snapshotting/copy-on-write system down the line.</p>
<p>While OPFS is pretty broadly available, Infinite Mac still has enough visitors from older browsers that I wanted to detect support for it (the API is also not available in private Safari windows). One annoying gotcha is that the synchronous file API is only available in workers, and there’s no way to detect its presence from the main browser process. I therefore have to create <a href="https://github.com/mihaip/infinite-mac/blob/941b7e9cc81067929e3e0e880f54588eb839d3a2/src/canSaveDisks-worker.ts">a temporary worker</a> to check for the existence of the <code>createSyncAccessHandle</code> and related functionality. The check is done <a href="https://github.com/mihaip/infinite-mac/blob/941b7e9cc81067929e3e0e880f54588eb839d3a2/src/index.tsx#L6-L12">as early as possible</a> and the results are cached, so that most of the time we can synchronously determine the support level.</p>
<br/>
<p style="text-align: center">
<img src="https://persistent.info/images/infinite-mac-saved-hd-settings.png" width="636" height="350" alt="Settings dialog showing options to import and export the Saved HD">
</p>
<p>With all that in place, it’s possible to use the Saved HD as a “real” disk in the emulated Mac. This enables quite a few capabilities:</p>
<ul>
<li>You can install system software on Saved HD (either by copying it from an existing disk or via a System Software CD-ROM from the <a href="https://blog.persistent.info/2023/08/infinite-mac-cd-roms.html#infinite-mac-cd-rom-library">library</a>). It can then be used as a startup disk (if running <a href="https://infinitemac.org/run">a custom instance</a> with no other disks), allowing custom extensions and control panels to be installed and persisted across sessions.</li>
<li>The contents can be exported and imported (from the emulator settings dialog), allowing backups and sharing across browsers and machines.</li>
<li>The contents can also be exported to a <code>.dsk</code> disk image file, so that they can be used in Basilisk II, SheepShaver and other native emulators.</li>
<li>The same disk is mounted in all instances, so it's also a way to move data from one to system disk to another.</li>
</ul>
<p style="text-align: center">
<img src="https://persistent.info/images/infinite-mac-saved-hd.png" width="968" height="768" alt="System 7.5 install with a Kaleidoscope scheme and ResEdit editing an icon">
<br>
<i>Your own private System 7 install</i>
</p>
<p>The end result is a way to make Infinite Mac feel like “your Mac” while still keeping everything local and fast. That also lines up well with my goal of keeping the site a fun hobby project — while persisting disks on a server would be neat, it would also be a more expensive and complex operation.</p>
<p><b>Update on October 1, 2023:</b> This has been <a href="https://github.com/mihaip/infinite-mac/issues/234">extended</a> to also support generating of disk device images (<code>.hda</code> files), which allows Infinite Mac prepare disks for <a href="https://bluescsi.com/">BlueSCSI</a> devices. Ron's Computer Videos has <a href="https://www.youtube.com/watch?v=2KCb7tuE9Io">a walkthrough of the process</a>.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com1tag:blogger.com,1999:blog-6525469191850690957.post-14855764985000997862023-08-23T07:45:00.001-07:002023-09-08T09:09:53.408-07:0020 Years of Blogging<p>The first proper blog post on this site was <a href="https://blog.persistent.info/2003/08/hello-world.html">20 years ago today</a>. It lived <a href="https://mscape.com/vidboi">under a different URL</a>, I didn’t register <a href="http://persistent.info/">persistent.info</a> until <a href="https://blog.persistent.info/2004/01/moving-house.html">later</a> (when <a href="https://en.wikipedia.org/wiki/.info">.info</a> was a new-and-shiny top-level domain). It also looked different (I don’t have a screenshot of the first version, but I switched themes <a href="https://blog.persistent.info/2003/12/new-look.html">a few months later</a>) and was published by very different software (<a href="https://web.archive.org/web/20071025125058/http://www.movabletype.com/blog/2003/05/version-264-released.html">Movable Type 2.64</a>), but there is continuity from then to today.</p>
<p style="text-align: center">
<img src="https://persistent.info/images/twenty-years.jpeg" width="640" height="360" alt=""10 years, man!" scene from Gross Pointe Blank with the 10 crossed out and replaced by 20" title="I am now (much) older than the characters from Grosse Pointe Blank">
</p>
<p><a href="https://blog.persistent.info/2003/08/temporal-discontinuity.html">Later that day</a> I imported a bunch of older programming journals, thus the entries go back <a href="https://blog.persistent.info/1998/09/and-so-it-begins.html">almost 25 years</a>. The first few years are not really blog posts, but it’s been nice to <a href="https://blog.persistent.info/1999/10/descriptions-of-all-current-projects.html">have</a> <a href="https://blog.persistent.info/2004/07/bus-times-on-your-cellphone.html">snapshots</a> of early work, even if they’re not all <a href="https://blog.persistent.info/2005/07/impromptu-market-research.html">gems</a>. I’ve been backfilling a <a href="https://persistent.info/projects">projects page</a>, and the posts also serve as tangible artifacts to link to, even if the project itself is long-dead.</p>
<table style="border-collapse: collapse; margin: 0 auto" border="1" cellspacing="0" cellpadding="2">
<tbody>
<tr>
<th>Year</th>
<td>2014</td>
<td>2015</td>
<td>2016</td>
<td>2017</td>
<td>2018</td>
<td>2019</td>
<td>2020</td>
<td>2021</td>
<td>2022</td>
<td>2023</td>
</tr>
<tr>
<th>Posts</th>
<td>13</td>
<td>3</td>
<td>4</td>
<td>2</td>
<td>4</td>
<td>1</td>
<td>2</td>
<td>5</td>
<td>7</td>
<td>12</td>
</tr>
</tbody>
</table>
<p>The trend for posts per year is looking encouraging when looking at the past decade, here’s to a few more (while computers <a href="https://en.wikipedia.org/wiki/Year_2038_problem">still work</a>).</p>
Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com1tag:blogger.com,1999:blog-6525469191850690957.post-29249638598940050672023-08-01T23:15:00.003-07:002023-09-13T22:10:01.089-07:00Infinite Mac: Disks, CD-ROMs and Custom Instances<p>Here are a few highlights of what’s been happening with <a href="https://infinitemac.org/">Infinite Mac</a> since <a href="https://blog.persistent.info/2023/03/infinitemac-dot-org.html">my last update</a>.</p>
<h3>Improving Disk Abstractions</h3>
<p>My broad goal this time around was to <a href="https://blog.persistent.info/2023/03/infinitemac-dot-org.html#:~:text=Though%20making%20it%20even%20easier%20to%20load%20software%20(even%20if%20I%20don%E2%80%99t%20include%20it%20directly)%20would%20be%20even%20better%20%E2%80%94%20I%20recently%20came%20across%20Discmaster%20and%20something%20that%20lets%20you%20load%20anything%20from%20the%20Internet%20Archive%20with%20one%20click%20would%20be%20great.">make it easier to load external software</a>. To lay the groundwork for this, I first needed to improve some of the disk abstractions that the project used. One of the things that makes the emulated Macs relatively fast to boot is that they stream in their disk images, since much of it is not needed at startup (e.g. <a href="https://macos9.app/">Mac OS 9</a> only reads 75MB out of a 150MB install). I had implemented this streamed/chunk approach<a href="https://github.com/mihaip/infinite-mac/commit/2992609ac3e86f595796759fb8aec6c1c354b298"> pretty early in the project</a>, when I understood the emulator internals (and Emscripten) less well. It worked by monkey-patching the read/write operations in Emscripten’s <a href="https://emscripten.org/docs/api_reference/Filesystem-API.html#memfs">MEMFS filesystem</a> (the default when doing an Emscripten build).</p>
<p>Besides being somewhat hacky, this had the downside of requiring that all mounted disks be loaded into memory as <code>ArrayBuffer</code>s (hence the name MEMFS). While the system software disks are relatively small, the Infinite HD disk with pre-loaded software is 1 GB, thus this was leading to significant memory use. I had added some <a href="https://github.com/mihaip/infinite-mac/commit/89cb33397881391d5ca0f31552cdd19ae97d24f8">basic instrumentation of emulator errors</a>, and running out of memory was surprisingly common (6.5% of emulator starts), presumably due to low-memory iOS devices or 32-bit Chrome OS devices. If I was going to add the ability to load additional (CD-ROM-sized) disk images, the memory use would increase even more.</p>
<p>I briefly explored Emscripten’s <a href="https://emscripten.org/docs/api_reference/Filesystem-API.html">other file system options</a>, but they all didn’t seem like quite the right fit. Fundamentally the issue was that all disk access was going through too many layers of abstraction, and the intent was lost. The emulator was operating against the POSIX file system API (<code>open</code>, <code>read</code>, on a file descriptor etc.) and by the time Infinite Mac’s code ran, it required extra bookkeeping to know which file was being requested (it could be a non-disk file), and the choice for the backing store for that file (as an <code>ArrayBuffer</code>) had already been made by Emscripten.</p>
<table style="margin: 0 auto; table-layout: fixed">
<tr>
<td style="width: 50%; padding: 4px; vertical-align: bottom; text-align: center; font-style: italic">
<img src="https://persistent.info/images/infinite-mac-disk-abstractions-before.png" width="288" height="320" alt="5 layers of abstraction"><br>
Before
</td>
<td style="width: 50%; padding: 4px; vertical-align: bottom; text-align: center; font-style: italic">
<img src="https://persistent.info/images/infinite-mac-disk-abstractions-after.png" width="288" height="192" alt="3 layers of abstraction"><br>
After
</td>
</tr>
</table>
<p>While many problems in computer science can be <a href="https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering">solved with a layer of indirection</a>, in this case it was a matter of removing one or two. The emulators are cross-platform systems themselves, and they have a way of swapping out the POSIX file backing for disks with others for other platforms. By adding more direct “JS” disk implementations in both <a href="https://github.com/mihaip/macemu/commit/8561af3f38af97892878511a9ef39b947a1debd3">Basilisk II/SheepShaver</a> (hereafter “Basilisk”) and <a href="https://github.com/mihaip/minivmac/commit/e351655612ee4b3d697b213765396b91f0bd9c7c">Mini vMac</a> and then implementing the JS APIs <a href="https://github.com/mihaip/infinite-mac/commit/9e361b9679e63b5c30f9bd61d2da983b69638dc3">on the Infinite Mac side</a>, the entire POSIX and Emscripten layers could be bypassed. We now only need to use as much RAM as the number of 256K chunks that have been accessed (and there is enough flexibility in the system that a LRU system could also be implemented to keep the working set fixed in size). After this change the out-of-emory error rate went to 0.3%.</p>
<p>The next thing to tackle was avoiding the need to restart the emulators when mounting additional disks. This only affected the Basilisk version, since Mini vMac already had built-in support for <a href="https://github.com/mihaip/infinite-mac/commit/6cc34a97dc87a82ee2a0c39059f240530975f40c">mounting disks on demand</a>. I assumed that this was not a fundamental limitation, all the emulators supporting mounting (real) removal media on demand already. It turned out to be <a href="https://github.com/mihaip/macemu/commit/91f06fdea8d2cc2e921ba9c482a765c70da94e4d">a missing feature in Basilisk’s disk abstraction</a>, and once I added that it was possible to <a href="https://github.com/mihaip/infinite-mac/commit/401ec13fafe35372c674a38a2e16e873ba53c528">mount disks immediately</a> there too.</p>
<h3 id="infinite-mac-cd-rom-library">CD-ROM Library</h3>
<p>With this in place, I was able to start working on the <a href="https://github.com/mihaip/infinite-mac/issues/166">CD-ROM “library” feature</a> I had envisioned. This was based on the observation that the <a href="https://archive.org/">Internet Archive</a> has a large collection of disk images in the <code>.iso</code> format, include <a href="https://archive.org/details/cdromsoftware?tab=collection&query=mac">many</a> <a href="https://archive.org/details/cdromsoftware?tab=collection&query=macintosh">for</a> <a href="https://archive.org/details/cdromimages?tab=collection&query=mac">the</a> <a href="https://archive.org/details/cdromimages?tab=collection&query=macintosh">Mac</a>. Furthermore, the files are <a href="https://blog.archive.org/developers/">accessible</a> without any kind of authentication, and support <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests">HTTP range requests</a>. This meant that it’s possible to take the same chunked/streaming approach I was using for my disk images and use it for these files without any kind of pre-processing. It is rather amusing that download speeds and latencies over the Internet are better than those of 1-2x CD-ROM drives from the early 90s (150-300 KB/sec and 100-200ms seek times).</p>
<p>I do these range requests in <a href="https://github.com/mihaip/infinite-mac/blob/main/workers-site/cd-rom.ts">simple handler</a> in a Cloudflare Worker, mostly so that they end up being same-origin requests. I later found that there is a <a href="https://git.archive.org/www/dweb-cors">CORS gateway</a> for the Internet Archive, but it appears to be a secondary service, and doing the fetching myself means that I can benefit from Cloudlare’s edge caching. However, switching to doing direct client-side loading is something to consider if the bandwidth costs become prohibitive (though for now <a href="https://github.com/sponsors/mihaip">donations</a> are more than covering the costs).</p>
<p>One gotcha that I ran into this that some disk images <a href="https://ia802305.us.archive.org/view_archive.php?archive=/13/items/the-manhole/Manhole%2C%20The%20%28USA%29.7z">are</a> <a href="https://ia801705.us.archive.org/view_archive.php?archive=/27/items/bungiemarathontrilogy/CD%20Images/Marathon%20Trilogy.zip">compressed</a>. While there ways to <a href="https://github.com/sozip/sozip-spec">make .zip files seekable</a>, they require re-compressing, which would defeat the purpose of making this be a minimal setup/low overhead feature. For now I’ve chosen to skip over compressed files (or support other disk image formats that support compression, such as .dmg).</p>
<p>Continuing with the project’s <a href="https://blog.persistent.info/2023/03/infinitemac-dot-org.html#:~:text=I%20used%20the,images)%20becomes%20overwhelming.">curation philosophy</a>, I wanted to have a nice browsable UI of CD-ROMs, ready to be mounted in the emulated Mac. I ended with a <a href="https://github.com/mihaip/infinite-mac/tree/main/CD-ROMs">collection of metadata files</a> that specify disk image URLs and cover art that is processed by <a href="https://github.com/mihaip/infinite-mac/blob/main/scripts/import-cd-roms.py">a simple importing script</a>. The generated catalog is rendered in a <a href="https://web.archive.org/web/20070406172644/http://arstechnica.com/paedia/f/finder/images/popup-folders.jpg">pop-up folder</a>-inspired UI:</p>
<p style="text-align: center;"><img alt="Infinite Mac with CD-ROM library pop-up window open" height="843" src="https://persistent.info/images/infinite-mac-cdrom-library.png" width="1037" /></p>
<p>I added <a href="https://github.com/mihaip/infinite-mac/blob/main/CD-ROMs/Apple%20Developer%20CDs/1993-01%20-%20The%20Postman%20Always%20Clicks%20Twice.json">some</a> <a href="https://github.com/mihaip/infinite-mac/blob/main/CD-ROMs/Compilations/Power%20Computing%EF%B9%95%20The%20Disc.json">of</a> <a href="https://github.com/mihaip/infinite-mac/blob/main/CD-ROMs/Reference/Grolier%20Multimedia%20Encyclopedia%20(1995).json">my</a> <a href="https://github.com/mihaip/infinite-mac/blob/main/CD-ROMs/Games/Myth%EF%B9%95%20The%20Fallen%20Lords.json">favorites</a> and <a href="https://hachyderm.io/@mihaip/110529659997491091">got a few more suggestions</a> via Mastodon. Around this same time <a href="http://cdrom.ca/games/2023/06/29/first-cdrom-games.html">a pretty throughly researched article</a> on the first CD-ROMs appeared, and I hoped to include the ones referenced too. However, most of the early CD-ROMs were actually just taking advantage of <a href="http://cdrom.ca/games/2023/06/29/first-cdrom-games.html#:~:text=The%20Manhole%20was,floppy%20game%20first.">the ability to</a> <a href="http://cdrom.ca/games/2023/06/29/first-cdrom-games.html#:~:text=The%20Voyager%20Company,using%20Apple%27s%20Hypercard">include audio CD tracks,</a> which is something that I have not gotten to work yet, so I was not able to add them.</p>
<p>I did also add the ability to mount arbitrary CD-ROMs via a dialog, so if you come across something on the Internet Archive that catches your eye, you can give it a try (you can even <a href="https://github.com/mihaip/infinite-mac/commit/a372439a15914f55ed77504c27c00e59bcb8701c">drag-and-drop the link</a> from another tab). I also made the Infinite HD disk image 2GB (i.e. with 1GB of free space), so that it’s available as a destination for any software that requires installation to a local hard drive. This is another way in which the change to not require disk images to be loaded into RAM paid off.</p>
<h3>Custom Instances</h3>
<p>Around the time that I was wrapping up CD-ROM support, I came across the recently-launched <a href="https://classicmacdemos.com/">Classic Macintosh Game Demos</a> site. It’s a “kindred spirit”, in that it’s trying to make things easily accessible and has a curatorial bent. <a href="https://jcgraybill.github.io/">Jules Graybill</a> (the author) was sourcing the game demos from CD-ROMs that came with magazines, images of which were also uploaded to the Internet Archive. I added the ability to specify a CD-ROM URL <a href="https://github.com/mihaip/infinite-mac/commit/656d6825b34eab0b046d59ed36279d5744bd2ba3">via a query parameter</a> and <a href="https://hachyderm.io/@mihaip/110642977729716394">reached out to him</a> to see if he wanted to make use of it (to allow the games to be played in-browser). He quickly <a href="https://hachyderm.io/@jcgraybill@oldbytes.space/110674444983705093">implemented it</a>, which is a great example of the flexibility of web-based projects — they can be made to work together (some might even say <a href="https://en.wikipedia.org/wiki/Hyperlink">“linked”</a> or <a href="https://en.wikipedia.org/wiki/Mashup_(web_application_hybrid)">“mashed up”</a>) with minimal coordination.</p>
<p>Shortly after, Jules <a href="https://github.com/mihaip/infinite-mac/issues/193">filed an issue</a> asking about an additional system extension he needed included on the system disk. Extrapolating from the ability to load CD-ROMs from URLs, I wondered if the same could be done for the system disk, so that he could provide his own image with exactly the extensions that he needed. And while we’re at it, why not allow the type of machine to be chosen via a URL parameter too, and control over other Infinite Mac features (<a href="https://blog.persistent.info/2022/07/infinite-mac-networking.html">AppleTalk support</a> or the extra Infinite HD disk). Getting perhaps a bit carried away, I ended up building a <a href="https://www.folklore.org/StoryView.py?story=Calculator_Construction_Set.txt">Calculator Construction Set</a> equivalent for emulated Macs, available at <a href="https://infinitemac.org/run">infinitemac.org/run</a>.</p>
<p style="text-align: center;"><img alt="Custom configuration dialog, matching a System 7 startup disk" height="350" src="https://persistent.info/images/infinite-mac-custom-classic.png" width="636" /></p>
<p style="text-align: center;"><img alt="Custom configuration dialog, matching a Mac OS 8.1 startup disk" height="350" src="https://persistent.info/images/infinite-mac-custom-platinum.png" width="636" /></p>
<p>To reduce the cognitive dissonance (and to have a bit of fun), I made the UI resemble the look-and-feel of the OS that is being booted. I had already added some classic- and Platinum-style controls for other bits of configuration UI, but this dialog also required implementing popup menus and checkboxes (the Appearance Sample app from the <a href="https://macintoshgarden.org/apps/appearance-manager-mac-os-7x">Appearance SDK </a>was invaluable in getting a view of all “modern” Mac OS controls). There’s still a bit of a an <a href="https://en.wikipedia.org/wiki/Uncanny_valley">uncanny valley</a> feel, perhaps due to fonts not being the same and modern screens having higher logical DPI (thus everything feeling smaller than it used to), but hopefully it will get closer over time.</p>
<p>You can also use this customization mode to have “Frankenstein” instances. For example, you can load the <a href="https://infinitemac.org/run?disk=System+7.1&disk=System+1.0&infinite_hd=true&machine=Quadra+650">System 1.0 image while booting with System 7</a>, allowing you to use a modern version of ResEdit to poke around the initial versions of the System and Finder. Or you can use this with any system disks that you may happen to come across (e.g. if you wanted to see what the <a href="https://infinitemac.org/run?machine=Quadra%20650&cdrom=https%3A%2F%2Farchive.org%2Fdownload%2Fslo-75_202301%2FSLO75.img">Slovenian version of System 7.5</a> looked like).</p>
<h3>Odds and Ends</h3>
<p>While the goal of the site is to make as many versions of classic Mac OS available in the browser, I realized some time after launch that 40+ releases is a lot to scroll through, and that more notable ones might get lost among all of the point releases. I therefore <a href="https://github.com/mihaip/infinite-mac/commit/2af38be9e54c8e565d1d582cae47a389dcbcfd39">added a simple filter toggle</a> to make the list that’s initially shown more manageable.</p>
<p>A longstanding issue was that dissolve animations in HyperCard would <a href="https://github.com/mihaip/infinite-mac/issues/42">run very slowly</a>, something that <a href="https://github.com/kanjitalk755/macemu/issues/107">affected</a> the native build of Basilisk II too. It turned out to be due to a missing implementation of high-resolution timers, which was <a href="https://github.com/kanjitalk755/macemu/commit/1199f8115e3e8569a7551126f51182183cea5a75">recently fixed</a> on the native side. While simply updating <a href="https://github.com/mihaip/macemu">my Basilisk II fork</a> did not get the fix for free, I did now have enough clues to implement <a href="https://github.com/mihaip/macemu/commit/c9b422345112d01b2f221d452cef3cf46d676a45">my own version of it</a>.</p>
<p>There was the usual round of upgrades for the software stack: a new way to <a href="https://github.com/mihaip/infinite-mac/commit/dc4050150115cc7c6c8f65e20ae832ad18bf03a1">install Emscripten</a>, <a href="https://github.com/mihaip/infinite-mac/commit/b221259595d323d2c45cd2175fdc7ce9287c3097">transitioning</a> from create-react-app to Vite (that one was a <a href="https://github.com/mihaip/macemu/commit/49787f862b1941a32c0a80077e1ecededdce5b06">yak</a> <a href="https://github.com/mihaip/minivmac/commit/804e16bf3432579415537cd1bf1c05fa27d59b66">shave</a>), and switching to the latest version of <a href="https://github.com/mihaip/infinite-mac/commit/9f798a758b571ba6076acea2f1409302debde3ee">TypeScript</a>, <a href="https://github.com/mihaip/infinite-mac/commit/feb2371237913f02286bfab088cfd4277513a28e">React</a>, and other <a href="https://github.com/mihaip/infinite-mac/commit/8a5b3b9b841998e11215d8bbfa623de9acaa426f">dependencies</a>. Though site performance hasn’t really been an issue, the switch to Vite made the initial bundle size more visible, so I spent a bit of time adding <a href="https://github.com/mihaip/infinite-mac/commit/9eb9fc6438332a73506563b76934fef413047afc">moving some things to dynamic imports</a> and some <a href="https://github.com/mihaip/infinite-mac/commit/a99e9cf618e7b5632117f59b1875be9f5519aea0">preloading</a> to pick up some easy wins.</p>
<p>I haven’t entirely decided what I’m going to focus on next, but I’m leaning towards improving the ability to persist changes to disks — it’s a shame to spend time installing software from CD-ROMs and then lose it all when closing the tab. The improved file system abstractions should make it easier to implement some of <a href="https://github.com/mihaip/infinite-mac/issues/152">my</a> <a href="https://github.com/mihaip/infinite-mac/issues/164">ideas</a> <a href="https://github.com/mihaip/infinite-mac/issues/182">here</a>.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com6tag:blogger.com,1999:blog-6525469191850690957.post-47246365530599498762023-07-06T09:09:00.001-07:002023-07-08T10:47:56.199-07:00Slack Canvas In The Streets, Quip In The Sheets<p>I finally got access to the <a href="https://slack.com/blog/news/meet-slack-canvas">recently-launched</a> <a href="https://slack.com/features/canvas">Slack canvas</a> feature. This project was the last thing I worked on before I left Quip/Slack/Salesforce in April of 2022, and I was curious how it had evolved since then.</p>
<p>Canvas started as a prototype in mid-2021 to reuse <a href="https://quip.com">Quip</a> technology to power a collaborative editing surface inside of Slack. The initial phase involved <a href="https://www.linkedin.com/in/mwhahn/">Michael Hahn</a> and I doing unspeakable things with iframes<sup>1</sup> to get the two large codebases to work together<sup>2 </sup>with minimal changes. This allowed a relatively rich feature set to be explored quickly, but it was not something that was designed to be generally available. At the time of my departure the work on productionizing (<a href="https://en.wikipedia.org/wiki/Ninety%E2%80%93ninety_rule">the second 90%</a> of the project) had just begun.</p>
<p>The first thing that becomes apparent is that the roots of canvas in Quip are entirely hidden from the user — no Quip references in the <a href="https://slack.com/blog/news/meet-slack-canvas">announcement</a> or anywhere in the UI. This makes sense from a <a href="https://en.wikipedia.org/wiki/Conway%27s_law">“don’t ship your org chart”</a> perspective — no user should care how a feature is implemented. However, if you peek under the hood, you can start to see the some Quip tidbits. The most obvious place to start is to look for network requests with <code>quip</code> in them — a couple of which happen when loading a Slack canvas:</p>
<p style="text-align: center">
<img alt="Slack canvas network requests" src="https://persistent.info/images/slack-canvas-network-requests.png" width="981" height="220">
</p>
<p>The “controller” is the core logic of Slack canvas editor, and we if load one of those URLs, we see even more Quip references:</p>
<p style="text-align: center">
<img alt="Slack canvas Quip minified JavaScript" src="https://persistent.info/images/slack-canvas-js-source.png" width="611" height="161">
</p>
<p>The DOM also resembles Quip’s, down to the same CSS class names being used. The need to scope/namespace them to avoid colliding with Slack’s was one of the open questions when I left, but I guess Slack has a BEM-like structure which ensures that Quip’s simpler class names don’t collide (as long as they don’t integrate another similar un-prefixed codebase). There are also no iframes in sight, which is great.</p>
<p style="text-align: center">
<img alt="Slack canvas DOM structure" src="https://persistent.info/images/slack-canvas-dom.png" width="1054" height="312">
</p>
<p>Quip also had extensive <a href="https://quip.com/blog/how-quip-builds-inproduct-debugging-tools">in-product debugging tools</a>, and I was curious if they also made the transition to Slack canvas. They’re normally only enabled for employee accounts, but as <a href="https://quip.com/blog/how-quip-builds-inproduct-debugging-tools#:~:text=However%2C%20if%20you%27d%20like%20to%C2%A0%E2%80%9Ccheat%E2%80%9D%20there%20may%20be%20a%20way%20%E2%80%94%20keep%20in%20mind%20that%20Marathon%20didn%27t%20have%20any%20cheat%20codes%20but%20one%20of%20its%20contemporaries%20did.">hinted</a> there is a way to enable them as a “civilian” user too. A couple of commands in the dev tools, and I was greeted by the green UI that I had spent so many years in:</p>
<p style="text-align: center">
<img alt="Slack canvas showing Quip's debug tools" src="https://persistent.info/images/slack-canvas-debug-tools.png" width="1317" height="941">
</p>
<p>I was also hoping that copying/pasting content from Quip into Slack canvas was a backdoor way to get some of features that have not (yet?) made the transition (spreadsheets, date mentions, etc.), but it does not appear to work.</p>
<p>On the mobile side, I had explored reusing Quip’s hybrid editing approach in the Slack iOS app, including the <a href="https://medium.com/@btaylor/react-with-c-building-the-quip-mac-and-windows-apps-c63155c1531b">core Syncer library</a>. Hooking up <a href="http://Console.app">Console.app</a> to an iOS device shows that the Syncer (and thus Quip) are still involved whenever a canvas is loaded.</p>
<p style="text-align: center">
<img alt="Slack canvas iOS logging showing Quip references" src="https://persistent.info/images/slack-canvas-ios-logging.png" width="1119" height="577">
</p>
<p>One of the open questions on mobile at the time of my departure was how to render Slack content that’s embedded in a document. Quip’s mobile editor is a (<a href="https://blog.persistent.info/search/label/WebKit">heavily customized</a>) web view, so that we can reuse the same rendering logic on all platforms. It's possible to see that the canvas rendering is still web-based by inspecting the Slack app bundle (<a href="https://www.emergetools.com/">Emerge Tools</a> provides a <a href="https://www.emergetools.com/app/example/ios/slack?search=collab">nice online tree view</a>) – there is a <code>mobile_collab.js</code> file which implements document rendering:</p>
<p style="text-align: center">
<img alt="CollabSdk.framework embedded in the Slack iOS app (as of June 2022)" src="https://persistent.info/images/slack-canvas-ios-collabsdk.png" width="411" height="521">
</p>
<p>Slack on the other hand is an entirely native app. Porting Quip’s editor to native components didn’t seem feasible on any sort of reasonable timeframe. It was also not appealing to reuse Slack’s web components on mobile, since they weren’t designed for that (either from a UI or data loading perspective). I had speculated that we could leave a “placeholder” element in the web view for Slack-provided UI (like a message card), and then overlay the native component on top of it. But I wasn’t sure if it would be feasible, especially when the document is scrolled (and the native view overlay would have to be repositioned continuously).</p>
<p>It’s not as easy to inspect the view hierarchy of an iOS app (<a href="https://doist.dev/posts/debugging-the-ui-of-third-party-ios-apps">without jailbreaking</a>), so I can’t verify this directly, but it would appear that this placeholder/overlay approach was indeed chosen. Most of the time, the native Slack UI is positioned perfectly over the document. However, in some edge cases (e.g when a scroll is triggered because the keyboard is being brought up), things end up slightly out of sync, and the placeholder is visible:</p>
<table style="margin: 0 auto; table-layout: fixed">
<tr>
<td style="width: 50%">
<img src="https://persistent.info/images/slack-canvas-ios-embed.png" width="390" height="844" alt="Message embed in Slack canvas">
</td>
<td style="width: 50%">
<img src="https://persistent.info/images/slack-canvas-ios-embed-offset.png" width="390" height="844" alt="Offset message embed in Slack canvas, captured during keyboard scroll">
</td>
</tr>
</table>
<p>This is my first time being on the outside of a project while significant work on it continued (unlike <a href="https://blog.persistent.info/2023/07/10th-anniversary-of-google-reader.html">other</a> <a href="https://twitter.com/mihai/status/1007099329226539008">times</a>), and it’s been fascinating to observe. I congratulate all the people involved in shipping Slack canvas, and will cheer them on<sup>3</sup>.</p>
<ol style="border-top: solid 1px #aaa; padding-top: 3px;" class="footnotes">
<li>I later realized I had done the same thing 15 years earlier, <a href="https://blog.persistent.info/2006/10/google-reader-redux.html">getting Reader iframed into Gmail</a> as another proof-of-concept.</li>
<li>At one point we had <a href="http://slack.com">slack.com</a> iframing <a href="https://quip.com/">quip.com</a> which in turn was iframing <a href="http://slack.com">slack.com</a> again (so that we could show Slack UI components inside documents), an architecture we took to calling “turducken.”</li>
<li>Especially if they bring back syntax highlighting for code blocks.</li>
</ol>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com0tag:blogger.com,1999:blog-6525469191850690957.post-3661601346810589462023-07-01T10:27:00.002-07:002023-07-01T11:01:54.828-07:0010th Anniversary of Google Reader Shutdown<p>It doesn't feel like it's been 5 years since <a href="https://blog.persistent.info/2018/07/google-reader-time-capsule-from-5-years.html">my last post about Reader</a>, but I guess the past few years have suffered from time compression. For this anniversary I don't have any <a href="https://readerisdead.com/reader/">cool projects to unveil</a>, but that's OK, because <a href="https://mastodon.social/@davidpierce">David Pierce</a> wrote a great article – <a href="https://www.theverge.com/23778253/google-reader-death-2013-rss-social">Who killed Google Reader?</a> – that serves as a nice encapsulation of the entire saga.</p>
<p>Other Reader team members and I had a chance to talk to David, and the article captures all of the major moments. Some things ended up being dropped though; there's enough twists and turns (in the <a href="https://blog.persistent.info/2011/10/google-reader-social-retrospective.html">social strategy</a> alone) that a a whole book could be written. Here's some more "fun" tidbits from Reader's history:</p>
<p>The article talks about "Fusion" being Reader's original/internal name (and how the "Reader" changed how it was perceived and limited its scope). The reason why "Fusion" was not used was because Google "wanted the name [Fusion] for another product and demanded the team pick another one. That product never launched, and nobody I spoke to could even remember what it was.". Fusion was the initial name that iGoogle launched under, as can be seen <a href="https://www.nytimes.com/2005/05/20/technology/google-moves-to-challenge-web-portals.html">in this article from mid-2005</a> (iGoogle itself was went through some naming changes, changing from Fusion to Google Personalized Homepage before ending up as iGoogle (its codename) in 2007). Finding the breadcrumbs of this story was somewhat difficult because Google later launched a product called <a href="https://support.google.com/fusiontables/answer/9551050?hl=en&visit_id=638238287170394143-1709642333&rd=1">Google Fusion Tables</a> (not surprisingly, it was also shut down).</p>
<p>In terms of naming, these were other names that were considered, so "Reader" was as worst-except-for-all-the-rest sort of thing:</p>
<ul>
<li>Google Scoop (which team took to referring to as "Scooper", as in <a href="https://www.chewy.com/b/poop-scoopers-1534">pooper scooper</a>)</li>
<li>Google Viewer</li>
<li>Google Finder</li>
<li>Google Post</li>
</ul>
<p>At one point during the (re-)naming saga Chris put in "Transmogrifier" as a placeholder name, with the logo being one of Calvin's <a href="https://en.wikipedia.org/wiki/Calvin_and_Hobbes#Cardboard_boxes">cardboard boxes</a>. During the next <a href="https://www.gawker.com/5182552/the-unflinching-stare-of-marissa-mayer">UI review</a> Marissa Mayer was not amused (or perhaps it was hard to tell what the logo was in those pre-retina days), and the feedback that we got was "logo: no trash".</p>
<p>A low point in the interal dynamics was hit in 2011. I had made some small tweaks (in my 20% time) to fix an annoying usability regression where links were black (and thus not obviously clickable). Since we were getting a lot of flack for it on Twitter, I <a href="https://twitter.com/googlereader/status/132615145615667200">tweeted from the Reader account</a> saying that it was fixed. A few hours later, I got a friendly-but-not-really ping from a marketing person saying that I need to run all future tweets by them, since there was an express request from Vic Gundotra to limit all communication about Reader, lest users think that it's still being actively worked on. That was the second-to-last tweet from the official account, the <a href="https://twitter.com/googlereader/status/311982565059858432">next one</a> was the shutdown announcement.</p>
<p>After Twitter <a href="https://www.wired.com/2007/03/twitter-is-ruling-sxsw/">blew up at SXSW 2007</a> there was a start of a "I don't need Reader/RSS, Twitter does it for me" vibe amongst some of the "influencers" of the time. I posted <a href="https://twitter.com/mihai/status/120441982">a somewhat oblique tweet</a> <a href="https://trends.google.com/trends/explore?date=2007-01-01%202007-03-01&q=%22google%20reader%22,twitter,toenails&hl=en">comparing</a> the Google Trends rankings of "google reader" and "twitter" (with "toenails" being a neutral term to set a baseline), showing that Reader dwarfed them all (the graph looks very different <a href="https://trends.google.com/trends/explore?date=all&q=%22google%20reader%22,twitter,toenails&hl=en">nowadays</a>). I couldn't understand why someone would want to replace Reader with a product that had no read state, limited posts to 140 characters, and didn't even linkify URLs, let alone unfurl them. In retrospect this was a case of <a href="https://en.wikipedia.org/wiki/Disruptive_innovation#Low-end_disruption">low-end disruption</a>.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com3tag:blogger.com,1999:blog-6525469191850690957.post-84534898176033993642023-04-25T13:12:00.000-07:002023-04-25T13:12:09.126-07:00Tailscale battery life instrumentation<p>I wrote a valedictory <a href="https://tailscale.dev/blog/battery-life">blog post</a> about the instrumentation we're adding to the Tailscale client to get a handle on battery life issues. It's been an interesting "full-stack" project, involving thinking about cell phone internals, hacking on the Go standard library, and exploring visualization options.<p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com0tag:blogger.com,1999:blog-6525469191850690957.post-2526052498285225052023-04-13T10:31:00.004-07:002023-04-13T10:31:56.235-07:00You won't believe what this one weird ioctl will do<p>Darwin was one of the first things Apple open-sourced (<a href="https://web.archive.org/web/19991012053536/http://apple.com/pr/library/1999/mar/16opensource.html">24 years ago</a>). It's been mostly a "throw it over the wall" approach, but being able to peek under the hood has been very handy.</p>
<p>I wrote a <a href="https://tailscale.dev/blog/darwin-spelunking">short tailscale.dev blog post</a> showing how we've been able to improve Tailscale's macOS and iOS apps by seeing how things are implemented.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com0tag:blogger.com,1999:blog-6525469191850690957.post-43956474802639239332023-03-28T08:00:00.006-07:002023-04-16T21:28:38.314-07:00Infinite Mac: infinitemac.org<p><b>tl;dr:</b> Infinite Mac has a new home at <a href="https://infinitemac.org/">infinitemac.org</a>. Using a new Emscripten port of Mini vMac, it is now able to run almost every notable version of Mac OS, from 1984’s <a href="https://infinitemac.org/1984/System%201.0">System 1.0</a> to 2000’s <a href="https://infinitemac.org/2000/Mac%20OS%209.0.4">Mac OS 9.0.4</a>. The project is also now accepting donations (via <a href="https://github.com/sponsors/mihaip">GitHub Sponsors</a> or <a href="https://www.paypal.com/donate/?hosted_button_id=394METCPT6PYN">PayPal</a>).</p>
<figure style="text-align: center">
<img src="https://persistent.info/images/infinite-macs.jpeg" width="800" height="627" alt="3 emulated Macs and a real one on a desk" style="max-width: 100%; height: auto">
<figcaption style="font-style:italic">Spot the odd one out.</figcaption>
</figure>
<h3>Mini vMac</h3>
<p>As I mentioned in my <a href="https://blog.persistent.info/2023/01/infinite-mac-2022-in-review.html#whats-next">last post on the project</a>, my immediate plans were to make even older versions of System Software runnable in the browser. <a href="https://github.com/adespoton">Em Adespoton</a> maintains a <a href="https://docs.google.com/spreadsheets/d/1us6SCBgVs8NqbxofJXTmHDeK3nKQJpcgya2nWC9_t2w/edit">compatibility matrix spreadsheet</a> showing which emulators can run specific OS versions, and it looked like my best bet was <a href="https://www.gryphel.com/c/minivmac/index.html">Mini vMac</a>. There is an existing <a href="https://github.com/yksoft1/minivmac-em">Emscripten port</a>, but I decided to start from scratch for two reasons:</p>
<ol>
<li>It used <a href="https://emscripten.org/docs/compiling/Building-Projects.html?highlight=sdl2#emscripten-ports">Emscripten’s SDL compatibility layer</a>, presumably to minimize the work to get something up and running. However, that’s both less efficient (as far as additional layers of abstraction to go through) and would not benefit from the JavaScript/browser API wrappers that I’d already developed for Basilisk II and SheepShaver.</li>
<li>It first ran Mini vMac’s custom build system to <a href="https://github.com/yksoft1/minivmac-em/commit/4448ccfea424d8fe684696d1b50271023693a709">generate</a> the Makefile and other configuration data for Linux/SDL, and then <a href="https://github.com/yksoft1/minivmac-em/commits/master">modified</a> the generated code to get things working. This seemed like a not very maintainable approach for the long run, since any re-runs of the generation system (e.g. to generate with <a href="https://www.gryphel.com/c/minivmac/options.html#in">different options</a>) would require the modifications to be redone.</li>
</ol>
<p>The first step in doing it myself was to get Mini vMac building. It’s an “interesting” project for a few reasons:</p>
<ul>
<li>It doesn’t use version control (source code is available as <a href="https://www.gryphel.com/d/minivmac/minivmac-37.03/minivmac-37.03.src.tgz">tarballs</a> <a href="https://www.gryphel.com/d/minivmac/minivmac-36.04/minivmac-36.04.src.tgz">for</a> <a href="https://www.gryphel.com/d/minivmac/minivmac-3.5.8/minivmac-3.5.8.src.zip">major</a> <a href="https://web.archive.org/web/20170619143145/http://www.gryphel.com/d/minivmac/minivmac-3.4.1/minivmac-3.4.1.src.zip">versions</a>).</li>
<li>Rather than a traditional autoconf + make setup, it has its own build system generator (also written in C). </li>
<li>Almost all <a href="https://www.gryphel.com/c/minivmac/options.html#in">options</a> are specified at compile time (to minimize binary size and runtime cost presumably), so there's a lot of branching that relies on the C preprocessor.</li>
<li>It can be built by <a href="https://www.gryphel.com/c/minivmac/appc/index.html#in">very old compilers</a> for a broad range of platforms (even <a href="https://github.com/minivmac/minivmac/blob/master/src/OSGLUMAC.c">classic Mac OS</a>, <a href="https://github.com/minivmac/minivmac/blob/master/src/OSGLUOSX.c">Carbon</a>, <a href="https://github.com/minivmac/minivmac/blob/master/src/OSGLUNDS.c">Nintendo DS</a>). Therefore it targets C89 and makes heavy use of macros and typedefs to make things work in such a broad range of environments.</li>
</ul>
<p><a href="https://invisibleup.com/articles/30/#:~:text=I%20could%20steal.-,Mini%20vMac,-This%20is%20the">This blog post</a> has a rather negative take on the whole setup, but I have some empathy for Paul (Mini vMac’s author). This is clearly a passion project for him (dating back to <a href="https://www.gryphel.com/c/news/nva0.html#n010228">2001</a>), and he’s set things up in a way that suits him. Incidentally, Paul himself<a href="https://www.emaculation.com/forum/viewtopic.php?t=11570"> has not been heard from</a> for a couple of years, hence Mini vMac is in a bit of a limbo state. An import of the codebase and website <a href="https://github.com/minivmac">to a GitHub org</a> was done in 2022, in case Paul’s site does eventually go down.</p>
<p>With that context out of the way, getting it to build was actually not as painful as it sounds. While the build system is custom, it’s also arbitrary C code, so it’s very easy to hack on (vs. dealing with <a href="https://en.wikipedia.org/wiki/M4_(computer_language)">m4</a> for autoconf). Since it targets some very low-end platforms it’s also very fast to build, so the iteration speed was great. The <a href="https://github.com/mihaip/minivmac/commit/2e845b62d3634b0f60f3dc3ce5f183f960ebd2d6">initial bring-up</a> involved adding an alternate Makefile generation mode that used the Emscripten toolchain and <a href="https://github.com/mihaip/minivmac/blob/2e845b62d3634b0f60f3dc3ce5f183f960ebd2d6/src/OSGLUESC.c">a glue file</a> to get configuration data in and the framebuffer out. With that scaffolding in place, adding <a href="https://github.com/mihaip/minivmac/commit/2bd517950679aaa5b4423b79deed3698208bf6d9">input</a>, <a href="https://github.com/mihaip/minivmac/commit/be99085a99a12116ea33b00bb8a2997393c902ae">disk image</a> <a href="https://github.com/mihaip/infinite-mac/commit/6cc34a97dc87a82ee2a0c39059f240530975f40c">mounting</a>, <a href="https://github.com/mihaip/minivmac/commit/8bcc54cbacad4cbebb603de260f7610bfbe34401">color</a> and <a href="https://github.com/mihaip/minivmac/commit/2fe49d588a24e635b680253f4c2144b4b6a7efcf">sound</a> was pretty straightforward.</p>
<p>Since the emulator allows the speed to be controlled, I <a href="https://github.com/mihaip/minivmac/commit/cde47f0034d04a21d5266c16223c84fc6a3fb911">exposed</a> <a href="https://github.com/mihaip/infinite-mac/commit/7476945e362710d7d84f2b96285ee84e653dced3">that</a> as well. I knew that modern machines emulate classic Mac OS significantly faster than 68K-era Macs ran it (even when targeting WebAssembly and not being able to use JIT), but it was made truly apparent when running in 1x speed mode. I was a lot more patient back in the 90s, back when you had to be <a href="http://web.archive.org/web/20040211083851/http://www.redshed.net/enbiggenEventQueue/index.html">aware of how big the event queue was</a> and make sure the computer could keep up.</p>
<p>The framebuffer updating in Mini vMac keeps track of <a href="https://wiki.c2.com/?DirtyRectangles">dirty rectangles</a> (another example of its focus on low-end host machines), and it was a nice bit of API “mechanical sympathy” to be able to <a href="https://github.com/mihaip/minivmac/commit/9742b0e19d6e9e5fe69b22f62b0009d17c3edbfe">hook that</a> <a href="https://github.com/mihaip/infinite-mac/commit/fcb93e00e9f4b41ec3c12c9950a15cdb56842fa1">up to</a> the dirty region option of <a href="https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/putImageData">CanvasRenderingContext2D.putImageData()</a>. Though as it turns out that code path is perhaps not well-tested on some versions of Chrome — non-macOS users <a href="https://github.com/mihaip/infinite-mac/issues/95">reported glitches</a> and I had to add some <a href="https://github.com/mihaip/infinite-mac/commit/436eed6ba9efd412dbbba619c9ed8084097862b9">opt-outs</a>.</p>
<h3>System 6</h3>
<p>Once I had Mini vMac up and running in the browser, I booted a System 6.0.8 image. I only started using a Mac with System 7.1, so this was my first time spending any significant amount of time with this older version. I recall that System 7 had a reputation for being more RAM hungry and slower than its predecessor, but by modern standards they seem equally snappy and stingy with their memory use (with MultiFinder enabled System 6 uses the same amount of RAM as System 7).</p>
<p>What was more surprising was that things that I took from granted from System 7 (and assumed had been there for ages) were in fact recent additions. Two that I ran into very quickly were typing into Finder windows to select items with that prefix and being able to drag files onto application icons to open them. Around this same time <a href="https://toot.community/@justkwin/109851080677206288">Quinn had a thread</a> about how earlier versions of the Mac UI were not as refined as people remembered them, which was a nice confirmation of my experience.</p>
<p>One of the other surprises that I ran into was that the Infinite HD disk (with its 1GB of Mac software) was taking a very long time to mount. I eventually figured out that its desktop file was being rebuilt (having a visible progress bar for this is another System7 refinement). It turned out that the <a href="https://support.apple.com/kb/TA46427?locale=en_US">Desktop DB/Desktop DF files</a> that I was used to were another System 7 advance, while System 6 used a simpler (single) “Desktop” file, which causes <a href="https://68kmla.org/bb/index.php?threads/way-to-stop-6-0-8-7-1-desktop-rebuilding.40069/">problems when switching back and forth</a>. The <a href="https://github.com/mihaip/infinite-mac/commit/935ea5973a16f11dcc6f09c373368433302c7a1e">solution</a> for my situation turned out to rebuild the desktop files twice, first under System 6 and then under Mac OS 8, so that both sets would be included in the generated image.</p>
<p>Once I had it all working, it was a matter of registering <a href="https://system6.app/">system6.app</a> and configuring it.</p>
<figure style="text-align: center">
<img src="https://persistent.info/images/system6.png" width="680" height="510" alt="System 6 showing the “About the Finder…” window" style="max-width: 100%; height: auto" loading="lazy">
<figcaption style="font-style:italic">System 6.0.8 running on an emulated Mac Plus.</figcaption>
</figure>
<h3>RUN <u>ALL</u> THE VERSIONS!</h3>
<figure style="float: right; text-align: center">
<a href="https://persistent.info/images/infinite-mac-browser.png"><img src="https://persistent.info/images/infinite-mac-browser.png" width="297" height="1302" alt="System 6 showing the “About the Finder…” window" style="max-width: 100%; height: auto; border: 0" loading="lazy"></a>
<figcaption style="font-style:italic">The mid-90s were a tough time.</figcaption>
</figure>
<p>With Mini vMac running, running older OSes was mostly a matter of figuring out which <a href="https://www.gryphel.com/c/minivmac/options.html#option_m">machines</a> to build and tracking down install disks. <a href="https://www.marchintosh.com/">MARCHintosh</a> was coming up, and I figured it would be a worthwhile goal to get a gallery up and running in time for that.</p>
<p>But first, I needed a domain name. While <a href="https://system7.app/">system7.app</a>, <a href="https://macos8.app/">macos8.app</a>, etc. were all very catchy and memorable, <code>system2point1.app</code> was somehow less so (this was the first version to <a href="https://en.wikipedia.org/wiki/Hierarchical_File_System_(Apple)#History">add support for HFS</a>, and thus notable, even if it was a point release). I initially considered <code>system.software</code> (to further extend my use of more obscure TLDs), but it’s a premium domain name costing $2,700/year, and thus not really within my budget. <code>macos.museum</code> was a more affordable $84/year, but it was only one of the few <code>macos.<TLD></code> domains that was unclaimed, leading me to conclude that Apple’s legal department may have an interest in it (“Mac OS” is also technically an anachronistic way to refer to pre-System 7.5.1 releases). In the end the “obvious” <a href="https://infinitemac.org/">infinitemac.org</a> won out, both for its affordability ($12/year) and because it matched the project name. The <code>.app</code> domains names will remain as shortcuts/redirects.</p>
<p>I used the word “gallery” above, and that was the experience I was going for. I wanted a curated experience, with pristine, period-accurate installations. I appreciate what the <a href="https://archive.org/details/softwarelibrary_mac">Internet Archive has done</a> (and will often use it as a source) but at some point the amount of choice (400+ bootable images) becomes overwhelming. I wanted to have a place where you can experience first-hand what someone means by <a href="https://arstechnica.com/gadgets/2010/09/macos-x-beta/14/#b6">“spatial Finder”</a>. Or if you <a href="https://gascoigne.social/@Charles/109684287235699788">come across some trivia</a> about option-double clicking only getting its modern behavior (close windows behind you) in System 6.0.4, you can <a href="https://infinitemac.org/1989/System%206.0.4">open it</a> and <a href="https://infinitemac.org/1989/System%206.0.3">its predecessor</a> side-by-side to see for yourself.</p>
<p>A reference point I had was Stephen Hackett’s <a href="https://512pixels.net/projects/aqua-screenshot-library/">macOS screenshot gallery</a> (or his more recent <a href="https://weblog.rogueamoeba.com/2023/03/03/come-visit-the-rogue-amoeba-historic-screenshot-archive/">Rogue Amoeba one</a>), or <a href="http://thewessens.net/collection/apple/systems/EarlyMacSystems.html">this one of early Mac systems</a>. While they were certainly curated, even a great set of screenshots can’t capture what it’s like to use one of these systems (e.g. the feeling of <a href="http://www.quinn.echidna.id.au/Quinn/WWW/HISubtleties/ZoomRects.html">zoom rects</a> going across the screen).</p>
<p>My other inspiration was <a href="https://mac.getutm.app/gallery/">UTM’s gallery</a> (which does provide one-click access, but requires native software and waiting for download and installation) and <a href="https://www.pcjs.org/">PCjs Machines</a> (which run in the browser, but being about PCs is inherently inferior in the eyes of this 90s Mac fanboy).</p>
<p>For getting pristine versions of each OS, I tried to use “primary” sources as much as possible, i.e. directly from Apple. In some cases Apple still hosts the downloads (unclear why <a href="https://support.apple.com/kb/DL1259?locale=en_US&viewlocale=en_US">Mac OS 8.6</a> gets such treatment). In other cases, there were “archive” releases from the 90s that have multiple operating systems (e.g. the <a href="https://macintoshgarden.org/apps/apple-eto-essentials-tools-objects-1991">1991 E.T.O. Essentials CD</a> has System 7.0 and earlier, and the <a href="https://archive.org/details/AppleLegacyRecoveryOct1999">1999 Legacy Recovery CD</a> has some later releases).</p>
<p>The one release that was surprisingly hard to track down was <a href="https://en.wikipedia.org/wiki/System_1">System 0.97/1.0</a> that shipped with the original Macintosh 128K. I was <a href="https://www.betaarchive.com/forum/viewtopic.php?t=31561">not the first</a> to run into this, and it appears to boil down to it being a short-lived release (1.1 came out a few months later), relatively few 128K Macs being sold, and the system disk also being the primary data storage mechanism for users. <a href="https://github.com/mihaip/infinite-mac/issues/100#issuecomment-1445007822">I tracked down</a> all the disk images I could find, and put together a combined one that matched the original’s spirit.</p>
<p>The other thing that was surprisingly hard to find was a detailed changelog for each release. Perhaps there are <a href="https://groups.google.com/g/comp.sys.mac.system">comp.sys.mac.system</a> posts from the era or Apple press releases, but my archeology skills do not extend to Usenet or wire services. I mostly used <a href="http://www.toddp.com/classic/Software%20Install/Apple%20Support%20Documents/Macintosh%20Secrets/Mac%20Secrets%204th%20ed./ch06.pdf">Macworld’s Mac Secrets</a> (which being from 1996 is somewhat contemporary) and <a href="https://web.archive.org/web/20160810183725/http://www.mac512.com/macwebpages/system.htm">Mac 512’s System Showcase</a>.</p>
<p>As previously mentioned HFS was introduced with System 2.1, thus earlier versions (that only support <a href="https://en.wikipedia.org/wiki/Macintosh_File_System">MFS</a>) cannot mount the Infinite HD HFS image <a href="https://blog.persistent.info/2022/03/blog-post.html#infinite-hd">with the software library.</a> I briefly considered making an MFS generator too, but the only MFS libraries I could find were either <a href="https://github.com/zydeco/libmfs">read-only</a> or <a href="https://developer.apple.com/library/archive/samplecode/MFSLives/Introduction/Intro.html">embedded inside sample code</a> from 20 years ago, thus it seemed like a yak too far. I ended up <a href="https://github.com/mihaip/infinite-mac/commit/6ace3f333201d69d1fcf3fe56c3fcab8ddf89368">hand-creating</a> an image with some representative programs.</p>
<p><a href="https://infinitemac.org/">The final gallery</a> includes every notable system software release (I'm sorry, fans of System 7.0.1). Perhaps over time I'll go back and backfill truly everything, but for now generating 36 bootable images was plenty. I tried to make the browsing experience as pleasant as possible — each release has a permalink, and you can command-click on the “Run” button to open things in new windows. Some releases only work with specific Macs, and some can run on multiple (I used the <a href="https://docs.google.com/spreadsheets/d/1wB2HnysPp63fezUzfgpk0JX_b7bXvmAg6-Dk7QDyKPY/edit#gid=840977089">Apple ROMs spreadsheet</a> and <a href="https://github.com/sentient06/MacROMan">Mac ROMan</a> to build up <a href="https://github.com/mihaip/infinite-mac/blob/930880fb8041df938caab2aae45fc6adc899a890/src/machines.ts">a library </a>of machines) — which machine is used can be chosen via the “Customize” button.</p>
<h3>Other Improvements</h3>
<p>The focus on gathering together all of the OS releases meant that I didn’t have as much time to work on the emulation aspects. The one thing I spent some time on was<a href="https://github.com/mihaip/infinite-mac/commit/5a682c7f1b18c186c018696345e61dfa918d2306"> improving the audio playback system</a>. Audio is one of the areas that I know less about, and I had kept most of the structure from <a href="https://github.com/jsdf/macemu">James Friend’s original Basilisk II port</a> in place. However, as I was bringing up the audio subsystem for Mini vMac I remembered that <a href="https://developer.mozilla.org/en-US/docs/Web/API/AudioWorklet">audio worklets</a> are now widely supported, and they also have an API “mechanical sympathy” with what the emulator expects. I’d also already set up a ring buffer system for the <a href="https://blog.persistent.info/2022/07/infinite-mac-networking.html">networking support</a>, and in fact the <a href="https://github.com/padenot/ringbuf.js#examples-and-use-cases">library mentioned emulation</a> as a use-case. The <a href="https://github.com/mihaip/infinite-mac/commit/5a682c7f1b18c186c018696345e61dfa918d2306">current implementation</a> is a system that is quite a bit simpler, has lower latency, and I understand better, though <a href="https://github.com/mihaip/infinite-mac/issues/123">there are still bugs</a>. Between <a href="https://github.com/mihaip/infinite-mac/blob/930880fb8041df938caab2aae45fc6adc899a890/src/emulator/emulator-audio-worklet.ts">the audio worklet</a>, <a href="https://github.com/mihaip/infinite-mac/blob/main/src/emulator/emulator-service-worker.ts">the service worker</a>, and <a href="https://github.com/mihaip/infinite-mac/blob/main/src/emulator/emulator-worker.ts">the dedicated worker</a>, I’ve managed to use most of the worker types — I now just need an excuse to use a <a href="https://developer.mozilla.org/en-US/docs/Web/API/PaintWorklet">paint worklet</a> to catch them all.</p>
<p>One small tweak that I made was related to scaling. Nearest-neighbor is nice for preserving a crisp look and feel, but it’s not appropriate when trying to fill the screen in <a href="https://github.com/mihaip/infinite-mac/commit/a417957d5734189110f3902938f720600f319000">full-screen mode</a> or when the host screen itself is <a href="https://github.com/mihaip/infinite-mac/commit/dc744f3f88557a7ba2e70f82a89ce20159b94ab5">doing non-integer scaling</a>. I can see why emulators end up with <a href="https://corteximplant.com/@scummvm/109935928397178919">configuration settings</a> to allow for more historical visual accuracy, but for now I’ve managed to stay away from too many fiddly settings.</p>
<p>I made a few improvements to the experience on mobile/touch devices — there’s now <a href="https://github.com/mihaip/infinite-mac/issues/7">a way to bring up the keyboard</a> and <a href="https://github.com/mihaip/infinite-mac/issues/79">import files</a> (without drag and drop) and <a href="https://github.com/mihaip/infinite-mac/commit/21ec859b475166d096dbb3afeed15494131a477e">controls are visible</a> even when the screen chrome is in its smallest mode. There are still fundamental limitations, these are all desktop UIs that were designed for very precise pointing devices, and they rely on actions like double-clicking that are <a href="https://github.com/mihaip/infinite-mac/issues/8">hard to do</a> precisely on mobile. PCjs appears to have <a href="https://www.pcjs.org/blog/2016/03/06/">taken a slightly different approach</a> that may be worth considering: touches appear to drive a virtual trackpad that moves the cursor and allows more complex operation (like drag-and-drop) to be accomplished.</p>
<p>I also ran into a few cases where the emulator would not load, especially in-app web views. Those turned out to be <a href="https://github.com/mihaip/infinite-mac/commit/773071876fda5a4101e2cd2a570a39b9dcec78c3">caused</a> by the service worker trying to do caching (but not being allowed to), but in general some <a href="https://github.com/mihaip/infinite-mac/commit/89cb33397881391d5ca0f31552cdd19ae97d24f8">basic error</a> <a href="https://github.com/mihaip/infinite-mac/commit/b39ff3cad6233891c035e196e8f97870282b831d">reporting</a> seemed appropriate. Perhaps setting up <a href="https://sentry.io/">Sentry</a> or the like would be appropriate, though I would like to minimize the number of other services I depend on.</p>
<p>I had <a href="https://blog.persistent.info/2023/01/infinite-mac-2022-in-review.html#donations">mentioned</a> wanting to have a way to cover the infrastructure costs of the project ($332.48 so far), and I got around to setting up <a href="https://github.com/sponsors/mihaip">GitHub Sponsors</a> and <a href="https://www.paypal.com/donate/?hosted_button_id=394METCPT6PYN">PayPal</a> donation accounts. A few people have already donated, for which I am very grateful. I will try not to blow it all on domain names.</p>
<h3>What’s Next</h3>
<p>First, a break. I’ve been working <a href="https://github.com/mihaip/infinite-mac/commits?author=mihaip&since=2023-01-01&until=2023-03-31">almost every day</a> on the project to hit my (self-imposed) MARCHintosh deadline of having all OS releases, and it’s gotten to be a bit of a grind. </p>
<p>Beyond that, there are <a href="https://github.com/mihaip/infinite-mac/issues?q=is%3Aissue+label%3ALibrary">a bunch of software requests</a> that I could incorporate into the Infinite HD image. Though making it even easier to load software (even if I don’t include it directly) would be even better — I recently came across <a href="http://discmaster.textfiles.com/">Discmaster</a> and something that lets you load anything from the Internet Archive with one click would be great.</p>
<p>Improving the persistence support is another area I’d like to explore — the “Saved” folder in “The Outside World” is a bit too obscure, and because it uses the <a href="https://macintoshgarden.org/apps/file-system-manager-12">File System Manager</a> some programs don’t work well with it. Ideally there should be a whole HFS volume that can be persisted. That would also integrate nicel with the browser’s <a href="https://developer.mozilla.org/en-US/docs/Web/API/File_System_Access_API">file system APIs</a>.</p>
<p>Finally, though I said I would take a break from adding additional operating system images, there are <a href="https://bitbang.social/@SinclairSpeccy/109980552130864852">pre-release and obscure ones</a> that might be fun to get running. Or perhaps even <a href="https://github.com/mihaip/infinite-mac/issues/121">parallel universe alternatives</a>.</p>
<p><b>Update:</b> See also the discussion on <a href="https://news.ycombinator.com/item?id=35345117">Hacker News</a>.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com3tag:blogger.com,1999:blog-6525469191850690957.post-65223858346564884182023-02-16T09:40:00.002-08:002023-02-16T09:40:21.439-08:00Reducing Tailscale’s Binary Size on macOS<p>One of the less visible changes in <a href="https://tailscale.com/changelog/#2023-01-24-client">Tailscale v1.36</a> is that the macOS binary is 35MB smaller. I wrote <a href="https://tailscale.com/blog/macos-binary-size/">a post on the Tailscale blog</a> about the chance observation and investigations that led to this size win. If you're interested in this kind of low-level shennanigans, we are <a href="https://boards.greenhouse.io/tailscale/jobs/4197939005">hiring iOS and macOS engineers</a>.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com0tag:blogger.com,1999:blog-6525469191850690957.post-24267458885592101712023-02-05T22:37:00.002-08:002023-06-05T22:27:55.081-07:00How I Consume Mastodon<p><b>tl;dr:</b> I use <a href="https://www.streamspigot.com/masto-feeder/">Masto Feeder</a> to generate a nicely-formatted RSS feed of my timeline, and then I can read it in my preferred feed reader.</p>
<p>A bit more than 10 years ago I wrote a post called “<a href="https://blog.persistent.info/2012/08/how-i-consume-twitter.html">How I Consume Twitter</a>”. It described how I contort Twitter to my completionist tendencies by generating RSS¹ feeds for my timeline (for more fine-grained updates) and lists (to read some accounts in a digest). That allowed me to treat Twitter as just another set of feeds in my feed reader, have read state, not need another app, and all the other benefits that RSS provides.</p>
<p>A bunch of things have changed since then: (lots of) Twitter drama, <a href="https://blog.twitter.com/en_us/topics/product/2017/Giving-you-more-characters-to-express-yourself">product</a> <a href="https://techcrunch.com/2015/04/06/retweetception/">changes</a> and feed reader <a href="https://blog.persistent.info/2013/03/the-people-behind-google-reader.html">churn</a> (I’m currently using <a href="https://netnewswire.com/">NetNewsWire</a>), but the <a href="https://streamspigot.com/bird-feeder/">Bird Feeder</a> and <a href="https://streamspigot.com/bird-feeder/">Tweet Digest</a> tools in <a href="https://streamspigot.com/">Stream Spigot</a> have continued to serve me well, <a href="https://github.com/mihaip/streamspigot/commits/ceaec6a6d79e7bbeab7711f58f971d96d6ec4271">with the occasional tweaks</a>.</p>
<p>When the people I follow started their migration to Mastodon <a href="https://simonwillison.net/2022/Nov/5/mastodon/">in early November</a>, I initially relied on Mastodon’s built-in <a href="https://mstdn.social/@rysiek/109288881064018003">RSS feeds for any user</a>. While that worked as a stopgap solution until I decided what instance to use², it proved unsatisfying almost immediately. The feeds are very bare bones (no rendering of images, polls, or other fancier features) and do not include boosts. Additionally, having potentially hundreds of feeds to crawl independently seemed wasteful, and would require manual management to track people if they ever moved servers.
<p>I saw that there was a <a href="https://github.com/mastodon/mastodon/issues/18601">Mastodon feature request</a> for a single feed for the entire timeline, but it didn’t seem to getting any traction. After a couple weeks of waffling, I decided to <a href="https://github.com/mihaip/streamspigot/commit/5fdedecb9a3b02f66ad99106fe253f3be1a2beab">fix it for myself</a>. The <a href="https://streamspigot.com/">Stream Spigot</a> set of tools I had set up for Twitter feed generation were pretty easily adaptable³ to also handle Mastodon’s API. Over the next few weeks I added<a href="https://github.com/mihaip/streamspigot/commit/00213a9b862de6823f2161786bc4b846c26fd891"> boost</a>, <a href="https://github.com/mihaip/streamspigot/commit/4560552da11a094c36972b8b37b113bc0d5a5e5b">spoiler/CW</a>, and <a href="https://github.com/mihaip/streamspigot/commit/4fad51a91b63404e5fd0b2647ea80e7a7e4c7551">poll</a> rendering, as well as <a href="https://github.com/mihaip/streamspigot/commit/1c04f7bfe5e402cda1cc5086c747d91ee854b650">list</a> and <a href="https://github.com/mihaip/streamspigot/commit/41779cddf6ba30ca140c3419903da1ba4269e276">digest</a> support.<p>
<p>The end result is available at <a href="https://www.streamspigot.com/masto-feeder/">Masto Feeder</a>, a tool to generate RSS feeds from your Mastodon timeline and lists. It can generate both one-item-per-post as well as once-a-day digests. It’s been working well for me for the past few weeks, and should be usable with any Mastodon instance (or anything else that implements its API). The main thing that would make it better is <a href="https://github.com/mastodon/mastodon/issues/16556">exclusive lists</a>, but there’s some <a href="https://github.com/mastodon/mastodon/pull/22048">hope of that happening</a>.
<p style="text-align: center">
<img src="https://persistent.info/images/mastofeeder-nnw.png" width="800" height="541" alt="Mastodon viewed as a feed in NetNewsWire" style="max-width: 100%; height: auto" loading="lazy">
</p>
<p>As for Twitter, with the upcoming <a href="https://twitter.com/TwitterDev/status/1621026986784337922">removal of all free API access</a> it likely means the end for Bird Feeder and Tweet Digest. What this means for me is that I’ll most likely stop reading Twitter altogether — I certainly have no interest in using the first-party client with its algorithmic timeline. However, I’ve been enjoying Mastodon more lately anyway (<a href="https://persistent.info/@mihai">@mihai@persistent.info</a> should find me), and with being able to consume⁴ it in my feed reader, it’s truly more than a 1:1 replacement for Twitter.</p>
<p><b>Update on June 5, 2023:</b> Twitter API access for Bird Feeder and Tweet Digest was eventually revoked on May 22.</p>
<ol style="border-top: solid 1px #aaa; padding-top: 3px;" class="footnotes">
<li>Technically Atom feeds, but I’ve decided to use the term RSS generically since it’s not 2004 anymore.</li>
<li>My dilly-dallying until I settled on <a href="https://hachyderm.io/home">hachyderm.io</a> mean that someone else <a href="https://hachyderm.io/@mihai">got @mihai</a> a few days before <a href="https://hachyderm.io/@mihaip">I signed up</a>.</li>
<li>It did pain me to have to write a bunch more Python 2.7 code that will eventually <a href="https://cloud.google.com/appengine/docs/standard/lifecycle/support-schedule#python">be unsupported</a>, but that’s a future Mihai problem.</li>
<li>For posting I end up using the built-in web UI on desktop, or <a href="https://tapbots.com/ivory/">Ivory</a> on my iPhone and iPad. <a href="https://www.opener.link/">Opener</a> is also handy when I want to open a specific post from NetNewsWire in Ivory (e.g. to reply to it).</li>
</ol>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com1tag:blogger.com,1999:blog-6525469191850690957.post-88477860852721139482023-01-25T10:29:00.000-08:002023-01-25T10:29:13.868-08:00Tailscale iOS and macOS Shortcuts<p>I worked on adding iOS and macOS Shortcuts support for Tailscale's latest release. I wrote <a href="https://tailscale.com/blog/ios-macos-shortcuts/">a blog post</a> with examples of shortcuts and automations that the Tailscale actions could be combined with. One that didn't make the cut was using sound recognition, for things like “In case of an emergency, break glass to activate Tailscale”.</p>
<p style="text-align: center">
<img src="https://persistent.info/images/tailscale-shortcut-sound.png" width="375" height="812" alt="iOS Shortcuts automation that connects Tailscale when the sound of breaking glass is detected" style="padding-right: 1em"/>
<img src="https://persistent.info/images/tailscale-shortcut-sound-options.png" width="375" height="812" alt="Shortcut automation sound triggers: baby crying, shouting, kettle, and others"/>
</p>
<p>Other <a href="https://support.apple.com/en-gw/guide/iphone/iphf2dc33312/ios">possibilities</a> include "Cry/shout/scream to connect" and "Let your pet control your connection state".</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com0tag:blogger.com,1999:blog-6525469191850690957.post-72655229771943035832023-01-12T08:47:00.006-08:002023-01-12T08:47:52.436-08:00Places Mihai Has Committed Code From<p>I've been running <a href="https://lolcommits.github.io/">lolcommits</a> for 10 years, and it's captured some interesting moments from my time at <a href="http://quip.com">Quip</a> and <a href="https://tailscale.com">Tailscale</a>.</p>
<p>I've put together <a href="https://persistent.info/commits/">a gallery</a>, it was a fun nostalgia trip to revisit so many places and coworkers.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com0tag:blogger.com,1999:blog-6525469191850690957.post-72757024597398711162023-01-03T10:45:00.004-08:002023-12-09T22:17:36.829-08:00Infinite Mac: 2022 In Review<p>I've come to think of Infinite Mac as my <a href="https://heredragonsabound.blogspot.com/2020/02/the-forever-project.html">forever project</a>. There's always something to work on, whether it's <a href="https://github.com/mihaip/infinite-mac/commit/6658c878b8bf23edead2895dcb5f1b18c43edcb0">expanding the library</a>, <a href="https://github.com/mihaip/macemu/commit/3d111bce5aecf0c1d4cc38ba2525a4ad154ee3d7">improving compatibility</a>, <a href="https://github.com/mihaip/macemu/commit/b8b868044b353a441f0065aab112121acfab57d5">adding more platforms</a>, <a href="https://github.com/mihaip/macemu/commit/3738e1e2e2ed771a7683dd4c4491392736020330">improving</a> <a href="https://github.com/mihaip/macemu/commit/2dbd859a74da4e347503ebdc0d8067a47e549fd4">performance</a>, <a href="https://github.com/mihaip/machfs/commit/754b595ffd403534aa7af2fd01a30933ffcff3e5">debugging data structures</a>, <a href="https://github.com/mihaip/macemu/commit/6fcc39e6cdfad05ea4e30eeb2560189531a2b5d1">bridging APIs from 30 years ago</a> with <a href="https://github.com/mihaip/infinite-mac/commit/b905ced74b8ea236e498c87ca3ac7144dd3b9cea">modern web platform features</a>, or <a href="https://github.com/cloudflare/wrangler2/issues/1151">fighting with frontend tooling</a>. With that in mind, here's where things stand at the end of the year — there have been quite a few changes since <a href="https://blog.persistent.info/2022/07/infinite-mac-networking.html">my last post on the project</a>.</p>
<h3>Foundations</h3>
<p>Befitting a long-term endeavor, I invested some time into maintainability. This included small changes like <a href="https://github.com/mihaip/macemu/commit/061451ca794a8cf0b88e0c49008bbdd67989bf8e">setting up auto-formatting</a> and bigger ones around code organization. I moved all of the browser-specific <a href="https://github.com/mihaip/macemu/commit/4ffe81ed3db581304e34e62aa761d69ad18b4d48">audio</a>, <a href="https://github.com/mihaip/macemu/commit/6146625997021b658b9d3eb2ec5250af4f1802e1">video</a>, <a href="https://github.com/mihaip/macemu/commit/5540fae81f4c2da8afd7e330404c7513498e053f">clipboard</a> and other subsystem implementations into their own modules, instead of adding lots of branching to existing ones.</p>
<p>That cleaner separation, combined with changes to <a href="https://github.com/mihaip/macemu/commit/27a21059447ce83eb56cf1abee45afc909890741">reduce</a> <a href="https://github.com/mihaip/macemu/commit/d29a02611e97df774e615dba8dc5bc4bd2848bf2">diffs</a> <a href="https://github.com/mihaip/macemu/commit/cdafdf75870b48bb715618292c32842bea5c08b2">with</a> <a href="https://github.com/mihaip/macemu/commit/70b08af5637d8eb2103529c2523a5a2a1acd05d6">the</a> <a href="https://github.com/mihaip/macemu/commit/b39c5a5532d1fa9c7bd0252787b8fcd3e990fbeb">upstream</a>, made it possible to rebase the repo on a more recent version of Basilisk II — I had still been basing my work on <a href="https://github.com/jsdf/macemu">James Friend’s initial Emscripten port</a>, which was a snapshot as of 2017. Most Basilisk II development is happening in the <a href="https://github.com/kanjitalk755/macemu">kanjitalk755’s fork</a>, and I switched to building on top of that.</p>
<p>Finally, I <a href="https://github.com/mihaip/macemu/commit/fbc446d1a2a5fed0a384ac1cd639371ec8581d81">made it easier</a> to do native (macOS) builds of Basilisk II (and <a href="https://github.com/mihaip/macemu/commit/b9dd6b477fc09289c60c81db44b774ab9cdd8d33">SheepShaver</a>) from the same repo. This reduced the friction when tracking down behavioral differences between the native and web-based versions: I can instrument or modify shared code and then run it in both builds to see how it differs.<br /><h3>SheepShaver and Mac OS 9</h3>With things on a more maintainable path, I decided to tackle a bigger project: PowerPC support (which would allow Mac OS 8.5 and later to run). This involved porting <a href="https://www.emaculation.com/doku.php/sheepshaver">SheepShaver</a> to WebAssembly/Emscripten. Luckily, it shares a lot of code and architectural decisions with Basilisk II (not surprising, since they were both created by <a href="http://www.cebix.net/">Christian Bauer</a>). The <a href="https://github.com/mihaip/macemu/commit/b8b868044b353a441f0065aab112121acfab57d5">initial bringup</a> and <a href="https://github.com/mihaip/infinite-mac/commit/6dd40f7a9bc37d1cb5443a07421f22b33a6b6c65">integration</a> involved similar <a href="https://www.gnu.org/software/autoconf/">autoconf</a> tweaks and <code>#define</code> changes to get the Emscripten build on the right code path. After that it was a matter of <a href="https://github.com/mihaip/macemu/commit/437fc51d5828cfe03eb0657ec12ca1afe72a42ee">hooking up</a> <a href="https://github.com/mihaip/macemu/commit/0c0baefb4b1634d029bd1fdddfd8338bc63a9b8e">each</a> <a href="https://github.com/mihaip/macemu/commit/f92a35415c4210446e99c56c21b2abb0b06597e2">subsytem</a> to the existing implementations that bridged to the JavaScript/browser world. </p>
<p>The end result is running at <a href="http://macos9.app">macos9.app</a>. My main takeaway is that it feels more sluggish than System 7 or Mac OS 8. A lot of that appears to be due to bloat in Mac OS 9 itself, running <a href="https://system7.app/?domain=system7-ppc.app">a PowerPC version of System 7</a> feels snappier. There’s probably <a href="https://artemis.sh/2022/08/07/emulating-calculators-fast-in-js.html">low-hanging fruit</a> in the emulation itself when targeting WebAssembly, but I have not done any investigations in that area.</p>
<h3>Features</h3>
<p style="text-align: center; font-style: italic"><img src="https://persistent.info/images/infinite-mac-2022.png" alt="Infinite Mac Mac OS 9 Screenshot" width="975" height="780">Mac OS 9 in an <a href="https://everymac.com/monitors/apple/studio_cinema/specs/apple_cinema_display.html">Apple Cinema Display</a> bezel with dynamically-generated Stickies</p>
<p>A somewhat silly feature I wanted to implement was to show <a href="https://github.com/mihaip/infinite-mac/blob/main/CHANGELOG.md">the changelog</a> in the set of stickies that is shown at startup. I had been previously been embedding the data by hand (by booting each image and <a href="https://github.com/mihaip/infinite-mac/commit/ca2158a70ae5032bc860a8bd16f4c84b08c593cb">editing the Stickies file</a>), but this was becoming tedious now that there were four separate variants and more frequent edits. I therefore <a href="https://github.com/mihaip/infinite-mac/blob/8bcd6a8f3bca66e4173694a06262cae4fba7d958/scripts/stickies.py#L50-L105">reverse engineered</a> the Stickies data format and then switched to <a href="https://github.com/mihaip/infinite-mac/commit/5be16f62c095759e1907d9a68301e6bf095bcf6b">dynamically generating it</a>, including <a href="https://github.com/mihaip/infinite-mac/commit/6b047582a72dfd889910f7d3540f48a48b879c84">the changelog</a>. A bit over-engineered perhaps (see the caveat below), but it was fun to reconstruct what <a href="https://jens.mooseyard.com/1997/04/13/they-made-me-an-offer-i-couldnt-refuse/">Jens Alfke had implemented</a> almost 30 years ago.</p>
<p>Another “because I felt like it” feature was adding <a href="https://github.com/mihaip/infinite-mac/commit/91594685e14bc1ba6322a8cce16e1ef0212a504e">era-appropriate screen bezels</a> to the ersatz monitor that is shown around the screen. Technically beige was no longer in use by the time System 7 was released (the <a href="https://en.wikipedia.org/wiki/Snow_White_design_language">Snow White design language</a> was fully rolled out a few years prior), so this may be something to revisit if older OS versions are supported.</p>
<p>I also added a couple of useful features: the ability to <a href="https://github.com/mihaip/infinite-mac/commit/5aec32805fc90c09988ee8514902b1d91d3036d0">swap the control and command keys</a> (so that shortcuts like Command-Q and Command-W can be used even when not in full screen mode) and <a href="https://github.com/mihaip/infinite-mac/commit/b905ced74b8ea236e498c87ca3ac7144dd3b9cea">clipboard syncing support</a>. The latter has some interface impedance mismatch issues, since the <a href="https://developer.mozilla.org/en-US/docs/Web/API/Navigator/clipboard">clipboard API</a> is asynchronous and has gesture trigger requirements. However, it seems to work well enough to get text in and out of the emulator in a more natural fashion than files saved in the “Uploads” folder.</p>
<h3>Improved Compatibility</h3>
<p>For the emulators to be more than just a curiosity (that gets played with for a few minutes and then forgotten), having compatibility that’s at least as good as the native builds is important. I spent some time fixing small bugs, like missing <a href="https://github.com/mihaip/macemu/commit/929c0b8a2ff523d7e05548b4b9e59da07431b9f3">thousands/16-bit support</a>, handling <a href="https://github.com/mihaip/infinite-mac/commit/bdf2e86891b307a212e9f24cee874189ff18b682">screen resolution changes</a>, and making the <a href="https://github.com/mihaip/macemu/commit/611d48c3f76ec20020aee66c652119d296fbe9dd">audio less laggy</a>.</p>
<p id="basilisk-ii-fpu">A bigger task was improving the accuracy of FPU emulation. This manifested itself as two bugs that initially seemed unrelated - a calculator program <a href="https://github.com/mihaip/infinite-mac/issues/58">failed when doing any operation</a>, and scroll thumbs <a href="https://github.com/mihaip/infinite-mac/issues/6">did not move in Mac OS 8</a>. <a href="https://twitter.com/Snial/status/1590064782119677952">A tip from a user</a> pointed out that native builds of Basilisk II used to have the latter problem too, <a href="https://www.emaculation.com/forum/viewtopic.php?p=59945&sid=eecd9f6aa8b9e4880f23d0f8eb93eb84#p59945">back in 2018</a>. Running with a build from that era reproduced the scrollbar behavior, and the calculator issue too, when I tried it on a hunch (presumably the scroll thumb drawing routines do a floating point division to compute the offset).</p>
<p>I then did a bisect of changes to find where it got fixed, and ended up with the <a href="https://github.com/kanjitalk755/macemu/commit/977056a0750d46153dd1e75c7f4bed81d9c15933">switch in FPU implementations</a> to a standard <a href="https://en.wikipedia.org/wiki/IEEE_754">IEEE 754-based one</a> (instead of a custom implementation). However, the WebAssembly version was already using the IEEE 754 implementation, thus it should be on the same code path. I eventually realized that the native build (on an Intel Mac) was on the <code>USE_LONG_DOUBLE</code> <a href="https://github.com/mihaip/macemu/blob/fbc446d1a2a5fed0a384ac1cd639371ec8581d81/BasiliskII/src/uae_cpu/fpu/types.h#L131-L150">sub-path</a>, while the WebAssembly one ends up with vanilla 64-bit doubles. Both <a href="https://en.wikipedia.org/wiki/X87">x87</a> and <a href="https://en.wikipedia.org/wiki/Motorola_68881">the 68881</a> support extended precision (80-bit specifically), which makes IEEE 754 a good match for them, but that’s not the case for the WebAssemby virtual machine. I then checked to see what the <a href="https://github.com/kanjitalk755/macemu/commit/d9e0761bd939ee085aa93ad7e8a5062e2e544874">arm64 port of Basilisk II does</a> (as an example of another platform without extended precision support), and it uses a different FPU implementation, based on <a href="https://www.mpfr.org/">mpfr</a>. <a href="https://github.com/mihaip/macemu/commit/3d111bce5aecf0c1d4cc38ba2525a4ad154ee3d7">Switching to it</a> resolved the issue, albeit with a performance hit (hopefully no one is doing long Infini-D renders in a browser).</p>
<p>Another place where an external tip provided a key clue was in tracking down the case of <a href="https://github.com/mihaip/infinite-mac/issues/67">missing sound support in Mac OS 8 and 9</a>. A user that was hosting their own instance of the emulator <a href="https://github.com/mihaip/infinite-mac/issues/67#issuecomment-1341077927">reported that sound worked for them</a>, which was surprising. I initially thought it was due to a different system image, but I could still not get it to work even when I used theirs. I eventually realized that they had a simpler build process, and were not using <code><a href="https://github.com/mihaip/machfs">machfs</a></code> to do the dynamic Stickies insertion mentioned above. When I switched to passing through my system image unmodified, sound began to work. </p>
<p>There’s probably another subtle bug in the HFS data structures that <code>machfs</code> emits (I have already fixed <a href="https://github.com/mihaip/machfs/commit/85eeb0b7ea129674913c3bd51659acc534b25a11">a</a> <a href="https://github.com/mihaip/machfs/commit/754b595ffd403534aa7af2fd01a30933ffcff3e5">couple</a>), but I did not feel like diving into <a href="https://web.archive.org/web/20031025060005/http://developer.apple.com/documentation/mac/Files/Files-104.html#HEADING104-55">Inside Macintosh: Files</a> again just yet. Instead I <a href="https://github.com/mihaip/infinite-mac/commit/8be8bd28c1f50397d4ab86d0f26c7c5629e9ca04">switched to a lower-tech way</a> of inserting the dynamically-generated Stickies file into the disk image: a placeholder file whose contents can then replaced by operating at the raw bytes level.</p>
<h3>Coverage</h3>
<p>Somewhat surprisingly, the SheepShaver/Mac OS 9 work ended up <a href="https://news.ycombinator.com/item?id=33868197">on Hacker News</a> before it was fully ready. The discussion was nice, though mostly in a nostalgic/“they don’t make them like they used to” vein. Amusingly, the TL of the .app domain project <a href="https://news.ycombinator.com/item?id=33872831">noticed their use</a>. The discussion also <a href="https://news.ycombinator.com/item?id=33869436">inspired me</a> to change the <a href="https://github.com/mihaip/macemu/commit/d36fa5e435b4e0bde03225f5c7039890b5908ff9">default alert sound to Sosumi</a>.</p>
<p>The Register also had an article <a href="https://www.theregister.com/2022/12/09/macos9app_emulation/">about Mac OS 9</a> where the author actually reached out and got some quotes from me — I appreciated the effort. A Japanese site had a pretty in-depth <a href="https://gigazine.net/news/20221207-infinite-mac-kanjitalk/">article about kanjitalk7.app</a>, it’s nice that the localized version was noticed.</p>
<p>My favorite was a <a href="https://www.youtube.com/watch?v=bXBsJA8Wbj4">very thorough YouTube video</a> about the project, including a demo of the LAN/AppleTalk functionality that I <a href="https://blog.persistent.info/2022/07/infinite-mac-networking.html">cobbled together over the summer</a>.</p>
<div style="text-align: center">
<iframe width="800" height="450" src="https://www.youtube.com/embed/bXBsJA8Wbj4" title="YouTube video player" frameborder="0" allow="autoplay; clipboard-write; encrypted-media; picture-in-picture" allowfullscreen></iframe>
</div>
<h3 id="whats-next">What’s Next</h3>
<p>I’d like to broaden the versions of System Software/Mac OS that can be run in a browser. There is some amount of “gap filling” between the 7.5, 8.1 and 9.0.4 images (<a href="https://en.wikipedia.org/wiki/Mac_OS_8#Mac_OS_8.5">Mac OS 8.5</a> has a special place in my heart because <a href="https://blog.persistent.info/1998/09/and-so-it-begins.html">I got my start</a> doing real development for the 32-bit <code>icns</code> icons that it introduced). However, older versions (System 6 and earlier) are only supported in other emulators, as <a href="https://docs.google.com/spreadsheets/d/1us6SCBgVs8NqbxofJXTmHDeK3nKQJpcgya2nWC9_t2w/edit">this handy spreadsheet shows</a>. There is an existing <a href="https://github.com/yksoft1/minivmac-em">mini vMac Emscripten port</a> that could serve as a starting point.</p>
<p id="donations">There is some cost involved with all this. Currently this includes 4 <a href="https://domains.google/tld/app/">.app domain names</a> at $10/each per year, the <a href="https://developers.cloudflare.com/workers/platform/pricing/">Cloudflare Workers Paid</a> plan at $5/month (to get Durable Objects that are used for LAN) and a GitHub <a href="https://docs.github.com/en/billing/managing-billing-for-git-large-file-storage/about-billing-for-git-large-file-storage#purchasing-additional-storage-and-bandwidth">Git LFS data pack</a> at $5/month. This is still at a point where it’s a reasonable “hobby” budget (especially compared to woodworking or 3D printing or photography, as examples of hobbies where gear cost can escalate), but I’m considering setting up a Patreon or GitHub sponsorship in case others do want to support the work.</p>
<p>Though I’m enjoying the solo aspects of this work (as far as working on whatever I want with no coordination overhead), I’m not opposed to outside contributions — it was nice to <a href="https://github.com/mihaip/infinite-mac/pull/81">get a PR</a> to improve the scrolling behavior on small screens. Hopefully there’ll be more of that in 2023.</p>
<p>But really the main goal is to <a href="https://justforfunnoreally.dev/">continue to have fun</a>.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com6tag:blogger.com,1999:blog-6525469191850690957.post-11567869053738062282022-12-05T15:12:00.002-08:002022-12-05T15:12:19.026-08:00Tailscale Fast User Switching<p>Account switching is one of the less fun parts of modern computing -- for a while I had registered <a href="https://web.archive.org/web/20120911012906id_/http://giveupandusemultiplebrowsers.com/">giveupandusemultiplebrowsers.com</a>. Even when software tries to accomodate these scenarios, the heuristics for when to switch can be tricky, and in some cases require the user to maintain a complex mental model of what state they're in and where they're trying to get to.</p>
<p>Tailscale did not distinguish itself in this regard -- fast user switching was <a href="https://github.com/tailscale/tailscale/issues/713">one of our most requested features</a> for the last couple of years. Besides being generally a good thing to do, user switching also plays into <a href="https://tailscale.com/blog/free-plan/">the business model</a> – if enthusisast users bring Tailscale to work, they're going to want to be able to quickly switch between their personal and work accounts.</p>
<p><img src="https://persistent.info/images/tailscale-fus.png" alt="Tailscale fast user switching UI in the Mac app menu bar" width="554" height="320" style="float:right; padding-left: 0.5em">With version 1.34 (released today), we now have the first step: <a href="https://tailscale.com/blog/fast-user-switching/">fast-user switching support</a>. <a href="https://twitter.com/maisem_ali">Maisem</a> did most of the hard parts (making the Tailscale backend profile-aware, data migration, and untangling what should be global vs. per-account state). I worked mostly on the Mac side, implementing <a href="https://ale.fyi/">Alessandro's</a> designs but also <a href="https://developer.apple.com/documentation/technotes/tn3137-on-mac-keychains">modernizing our use of the Keychain</a> (where account data is persisted).</p>
<p>I spent a lot of quality time with <a href="https://web.archive.org/web/20150905210920/https://developer.apple.com/videos/wwdc/2010/#145-video:~:text=HD%20and%20SD.-,Key%20Event%20Handling%20in%20Cocoa%20Applications,-Frameworks">WWDC sessions from 2010</a>, trying to make custom <code>NSMenuItem</code> views behave just like the real thing, even when using keyboard navigation or VoiceOver. The ironic thing is that Apple itself appears to have given up on this. On modern macOS versions the Wi-Fi and related "menu" controls are actually custom (SwiftUI?) views that don't behave quite like real menus (e.g. they don't highlight items with menu color).</p>
<p>While I was in this part of the codebase, I also cleaned up how the Mac app handles <a href="https://github.com/juanfont/headscale">custom</a> <a href="https://tailscale.com/glossary/2020-04-02-coordination-server/">coordination servers</a>. Though not officially supported, it's now possible to have profiles on different servers by using the debug menu (finding out how to trigger that is left as an exercise for the reader).</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com0tag:blogger.com,1999:blog-6525469191850690957.post-53647197073038827752022-10-28T09:36:00.000-07:002022-10-28T09:36:18.034-07:00If you wish to make a web SSH client from scratch, you must first invent the network stack<p>It was a lot of fun turning Brad's <a href="https://twitter.com/bradfitz/status/1450916922288005122">Taiscale-on-Wasm</a> <a href="https://github.com/tailscale/tailscale/commit/cd54f07bd968655e9fc7490f90b497dae776c6e5">prototype</a> into <a href="https://tailscale.com/kb/1216/tailscale-ssh-console/">Tailscale SSH Console</a>. I wrote a <a href="https://tailscale.com/blog/ssh-console/">post for the Tailscale blog</a> with more details (you can tell it wasn't ghost-written because it has my link-heavy style).</p>
<p>Almost all of the development was in the open-source repo, and started to land <a href="https://github.com/tailscale/tailscale/commit/6f5096fa61f36e20db84fa7c9b8806016393778f">a few months ago</a>. It was rather funny to have WebVM <a href="https://leaningtech.com/webvm-virtual-machine-with-networking-via-tailscale/">notice and start using it</a> a few weeks ago, sort of scooping our own announcement. Their use-case (as a way to get a browser-based VM online) is something I've considered for <a href="https://github.com/mihaip/infinite-mac">Infinite Mac</a>.</p>
<p>Brad and I did sneak in an easter egg. I spent a lot of time studying <a href="https://www.youtube.com/watch?v=F5bAa6gFvLs&t=35s">the product spec</a>, and landed it with the commit message "add alternate progress display for SSH sessions". It was very satisfying to <a href="https://twitter.com/jtcressy/status/1585778853263601664">have it be discovered</a> without prompting.</p>
<p>There was reasonable <a href="https://news.ycombinator.com/item?id=33360776">discussion of the feature on Hacker News</a>. My favorite comment was <a href="https://news.ycombinator.com/item?id=33361206">"Having an SSH client in your browser join your VPN violates all the principles of modern computing."</a> While not intended as a compliment, it did remind me of the <a href="https://www.youtube.com/watch?v=x7cQ3mrcKaY">rethinking best practices</a> approach from the early days of React.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com0tag:blogger.com,1999:blog-6525469191850690957.post-87372335888539334532022-09-15T09:06:00.001-07:002022-09-15T09:06:20.506-07:00The Case of the Spiky File Descriptors<p>One of the things that attracted me to Tailscale was their in-depth technical blog posts, like <a href="https://tailscale.com/blog/go-linker/">Josh's hacking on Go internals</a> to get iOS memory use down, or <a href="https://tailscale.com/blog/how-nat-traversal-works/">Dave's epic treatise on NAT traversal</a>. It was therefore nice to be able to contribute a debugging story: <a href="https://tailscale.com/blog/case-of-spiky-file-descriptors/">The Case of the Spiky File Descriptors</a>. There's nothing epic about it, but it was still a satisfying problem to troubleshoot.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com0tag:blogger.com,1999:blog-6525469191850690957.post-1975567579190386422022-09-12T21:38:00.000-07:002022-09-12T21:38:44.126-07:00How Well Did the Original iPhone Sell?<p>At this year’s <a href="https://www.macrumors.com/2022/09/07/cook-ive-powell-jobs-code-conference-2022/">Code Conference</a> Tim Cook discussed disagreements that he and Steve Jobs had, including <a href="https://twitter.com/markgurman/status/1567709002792071168">whether to have a carrier subsidy for the iPhone</a>:</p>
<div style="display: flex; justify-content: center;">
<blockquote class="twitter-tweet" data-dnt="true"><p dir="ltr" lang="en">Cook reveals him and Jobs disagreed over launching the iPhone as subsidized by the carrier or via receiving a carrier revenue share. He says this was their biggest debate. Jobs, of course, won with the original iPhone launching at $599. Cook got his wish the following year</p>— Mark Gurman (@markgurman) <a href="https://twitter.com/markgurman/status/1567709002792071168?ref_src=twsrc%5Etfw">September 8, 2022</a></blockquote> <script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
</div>
<p>The iPhone <a href="https://www.apple.com/newsroom/2007/06/28iPhone-Premieres-This-Friday-Night-at-Apple-Retail-Stores/">launched in June 2007</a> with a $599 price at a time when most high-end phones sold for $199 (with a 2-year carrier contract). The iPhone got <a href="https://www.apple.com/newsroom/2007/09/05Apple-Sets-iPhone-Price-at-399-for-this-Holiday-Season/">a $200 price cut in September</a>, and the next year the iPhone 3G <a href="https://www.apple.com/newsroom/2008/07/10iPhone-3G-on-Sale-Tomorrow/">went on sale</a> with a subsidized price of $199. The subsidized model continued for many years, until <a href="https://fortune.com/2017/01/09/verizon-subsidized-phones/">the entire industry moved away from it</a>.</p>
<p>In retrospect it’s clear that Tim was right, but it would be interesting to know how the original iPhone sales were going — that (presumably) was one of the data points that Tim could use to get Jobs and Apple to change strategies. As it happens I accidentally stumbled on a proxy for such data in that time period, though I didn’t fully grasp its significance until later.</p><p>In early 2008 I was browsing around internal the set of internal Google Maps dashboards when I came across one for mobile clients. Google provided the maps data for the iPhone (through 2012’s <a href="https://www.apple.com/newsroom/2012/06/11Apple-Previews-iOS-6-With-All-New-Maps-Siri-Features-Facebook-Integration-Shared-Photo-Streams-New-Passbook-App/">iOS 6 release</a>), and presumably the iPhone accessed this via the same API that <a href="https://web.archive.org/web/20070202004005/http://google.com/gmm/index.html">Google's own apps</a> used. By configuring the dashboard to display just new device IDs, it was possible to approximate iPhone sales (assuming that most users tried out the Maps app soon after purchase). Here’s my recollection of what that dashboard showed:</p>
<p style="text-align: center"><img src="https://persistent.info/images/iphone-sales.png" alt="Original iPhone Sales" width="606" height="337"></p>
<p>The downward trend after the initial launch was only reversed with the price cut, but even that did not result in a significantly higher plateau. It was also surprising that Christmas was a bigger spike than the initial launch. I also would have thought that the iPod touch spike would be higher (as an iPod touch user myself at the time, I was very excited to get access to Mail and Maps), but perhaps <a href="https://www.apple.com/newsroom/2008/01/15Apple-Announces-Major-Software-Upgrade-for-iPod-touch/">the $20 fee</a> turned off more people.</p>
<p>The dashboard also shows that it was <a href="https://twitter.com/mihai/status/2483733">a different era</a> in the Apple/Google relationship — Google had basically real-time visibility into how the iPhone was selling (as I recall the dashboard data was later locked down).</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com0tag:blogger.com,1999:blog-6525469191850690957.post-36671113334376837972022-07-26T09:23:00.001-07:002023-06-09T09:03:14.741-07:00Infinite Mac on a (Virtual) LAN<h3>tl;dr</h3>
<p>I’ve added a <a href="https://developers.cloudflare.com/workers/learning/using-durable-objects/">Cloudflare Durable Object-based</a> LAN mode to <a href="https://blog.persistent.info/2022/03/blog-post.html">Infinite Mac</a>, allowing networked games of Marathon, Bolo and anything else that works over AppleTalk. To try it out, use <a href="https://demo.system7.app/">demo.system7.app</a> (or any other subdomain — it defines the “zone” where packets are broadcast and thus instances are visible to each other).</p>
<div style="text-align: center">
<iframe width="800" height="558" src="https://www.youtube.com/embed/cPu7o6F_lwk" title="YouTube video player" frameborder="0" allow="autoplay; clipboard-write; encrypted-media; picture-in-picture" allowfullscreen></iframe>
<br><i>Remember to aim for the ground when using rocket launchers</i>
</div>
<h3><a href="https://tailscale.com/blog/remembering-the-lan/">Remembering The LAN</a></h3>
<p>Though a computer without internet access can feel like a useless brick nowadays, in the 80s and early 90s that was the default state. But even without the internet, local networking was somewhat common, especially in offices (some solutions were <a href="https://en.wikipedia.org/wiki/Ethernet_over_twisted_pair">more successful</a> than <a href="https://en.wikipedia.org/wiki/Token_Ring">others</a>, and some were <a href="https://en.wikipedia.org/wiki/PhoneNET">elegant kludges</a>). Classic Macs were a part of this, with <a href="https://en.wikipedia.org/wiki/AppleTalk">AppleTalk</a> arriving only one year after the launch of the Mac. Beyond the office use-cases like file sharing and networked printers, this was used for games, with <a href="https://en.wikipedia.org/wiki/Bolo_(1987_video_game)">Bolo</a> and <a href="https://en.wikipedia.org/wiki/Minotaur:_The_Labyrinths_of_Crete">Minotaur</a> (Bungie’s first game) being early examples.</p>
<p>After Infinite Mac was announced it took <a href="https://twitter.com/Case/status/1509735009170591750">less than an hour</a> for someone to ask for (Marathon) network play — I was not the only one with fond memories of <a href="https://marathon.bungie.org/spoiler/m2/29.shtml">Thunderdome</a>. The Basilisk II architecture is quite modular, and adding networking support for a platform is mostly a matter of implementing <a href="https://github.com/mihaip/macemu/blob/bas-emscripten-release/BasiliskII/src/dummy/ether_dummy.cpp">a few functions</a> called by the virtual Ethernet driver. There was also an existing <a href="https://github.com/mihaip/macemu/commit/6c35c2a9e8eff972816fee23501f3a14c57edf17">option to relay them over UDP</a>, which had pointers for the special broadcast addresses that needed to be handled. <a href="https://github.com/oldweb-today/macemu/commits/js-network">oldweb.today had used this approach</a> to get the emulator on the internet.</p>
<p>I began by implementing an <a href="https://github.com/mihaip/macemu/commit/989dfd163654bd19d44a86ab543d25ace3a8b5f7"><code>ether_js.cpp</code> version</a> of the Ethernet functions that sent/received data from the JS side, and a basic framework for <a href="https://github.com/mihaip/infinite-mac/commit/e302338a18f63ddc00844aac8de2821696891cf0">passing the packets to/from the worker to the UI process</a>. Initially that used a <code>BroadcastChannel</code> for local testing, but then I added a <a href="https://github.com/mihaip/infinite-mac/commit/791621c89a43096153aabf2a53a8b1a5bbaf5fa2">Cloudflare Durable Object-based transport</a> (somewhat inspired by <a href="https://blog.cloudflare.com/doom-multiplayer-workers/">this Doom port</a>). That turned into a bit of a yak shave because the tooling and recommended setup for Cloudflare Workers <a href="https://github.com/mihaip/infinite-mac/commit/e756f573de4284af9d46b7646b60ab135ac9d2cc">had changed</a> since I last updated them. However, it worked! The appeal of this approach is that it should have reasonable latency since the durable object will be created close to the clients that end up using it.</p>
<p style="text-align: center"><img src="https://persistent.info/images/infinite-mac-networking.png" alt="Infinite Mac network architecture" width="678" height="498"></p>
<p>As I was developing this, I noticed that the emulated Mac would pause for 5 seconds during the boot whenever AppleTalk was enabled. I verified this on <a href="https://everymac.com/systems/apple/powerbook/specs/mac_powerbook550c.html">actual hardware</a> and confirmed that it was not an emulation glitch. I decided to <a href="https://github.com/mihaip/infinite-mac/commit/84417926fa592f56242342924ed612793a8248df">add some logging</a> to understand why this was happening, and was amused to see that GitHub Copilot was capable of generating suggestions for TypeScript code that parsed AARP packets, which is surely not a common combination:</p>
<p style="text-align: center"><img src="https://persistent.info/images/copilot-aarp.gif" alt="GitHub Copilot suggesting AARP packet logging code" width="738" height="317"></p>
<p>Unfortunately, after some quality time with <a href="https://archive.org/details/Inside_AppleTalk/">Inside AppleTalk</a> it <a href="https://archive.org/details/Inside_AppleTalk/page/n81/mode/2up">turned out</a> that this delay was by design. AppleTalk nodes will self-assign an address, send out broadcast packets with it, and then wait to see if any nodes report conflicts. While this might be fixable by patching the ROM (a technique that Basilisk II makes <a href="https://github.com/mihaip/macemu/blob/bas-emscripten-release/BasiliskII/src/rom_patches.cpp">heavy use of</a>), it’s not something that I’m doing at this time. Because of the delay in booting, AppleTalk is not enabled in the default Infinite Mac instance, only when using a subdomain (to choose a “zone”). This also has the benefit of ensuring that zones do not get too big. Coincidentally wildcard subdomains recently <a href="https://blog.cloudflare.com/wildcard-proxy-for-everyone/">became free</a> on Cloudflare, enabling this approach.</p>
<p>I added some basic <a href="https://github.com/mihaip/infinite-mac/commit/3f07177a5c756d67dd74a2daa3e44cf9692c7a06">end-to-end latency tracking</a> and was surprised to see that even when using the <code>BroadcastChannel</code>-based transport it was averaging around 8ms, and sometimes was approaching 16ms. These times were suspiciously close to the 60 Hz screen refresh interrupt, and it turned out to be due to my approach of <a href="https://github.com/mihaip/macemu/commit/3f7c52934801d4d93c5cbc09c014d17a81eb58e2">hooking</a> into the input loop — it was only triggered before refreshing the screen. I fixed this by <a href="https://github.com/mihaip/macemu/commit/df44e317809a87acf71da54a6f9f439fdd675748">moving input reading</a> to being done on a higher frequency (1,000 Hz) — this both helped with reduced network latency and made mouse input feel smoother as well.</p>
<p><p style="text-align: center"><img src="https://persistent.info/images/infinite-mac-ping-time.png" alt="Infinite Mac screenshot showing 1ms ping times" width="960" height="720"><br><i>More acceptable ping times</i></p>
<p>The actual experience when playing Marathon is mixed. The <a href="https://github.com/Aleph-One-Marathon/alephone/blob/master/Source_Files/Network/RingGameProtocol.cpp">network protocol</a> was designed for LAN play, and does not handle the increased latency of being played over the internet well. If playing for more than 15-20 minutes the game state gets out of sync between players. That being said, it’s satisfying that it works at all.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com10tag:blogger.com,1999:blog-6525469191850690957.post-33794771420417860122022-05-17T08:00:00.007-07:002022-12-31T21:06:29.472-08:00Infinite Mac 7.5 Weeks Later<p><a href="https://blog.persistent.info/2022/03/blog-post.html">Infinite Mac</a> has been quite a whirlwind. I wasn’t sure if it would reach the Mac (or retro computing) community, but <a href="https://mjtsai.com/blog/2022/04/26/infinite-mac/">it</a> <a href="https://512pixels.net/2022/04/infinite-mac/">did</a>, and went beyond that too, including to <a href="https://arstechnica.com/gadgets/2022/04/boot-up-classic-mac-os-in-your-browser-window-with-the-infinite-mac-project/">Ars Technica</a> and <a href="https://news.ycombinator.com/item?id=30875259">Hacker News</a> (<a href="https://news.ycombinator.com/item?id=31168646">twice</a>). The most gratifying thing was seeing (and in some cases hearing from) people who were active Mac developers and community members in the 90s, including <a href="https://twitter.com/gaijinity/status/1511411977884770309">Andrew Welch</a>, <a href="https://arstechnica.com/gadgets/2022/04/boot-up-classic-mac-os-in-your-browser-window-with-the-infinite-mac-project/?comments=1&post=40797467">the author of Shufflepuck Cafe</a>, <a href="https://twitter.com/jorgbrown/status/1509776387871105026">Jorg Brown</a>, <a href="https://github.com/mihaip/infinite-mac/issues/58">James Thompson</a>, and <a href="https://twitter.com/SeanParent/status/1509797010387341315">others</a>. It was also great to see people actually <a href="https://twitter.com/jamisonsf/status/1510073312877035523">using</a> <a href="https://twitter.com/Martin_Adams/status/1519330262265610241">the</a> <a href="https://twitter.com/3K1/status/1522853132962721792">apps</a> <a href="https://twitter.com/blogmywiki/status/1515588719495069698">to</a> <a href="https://twitter.com/shibalympics/status/1513674773989150731">make</a> <a href="https://twitter.com/alanwritescode/status/1513511623452934145">things</a> — there’s more to old OSes than just playing games (though those are great too).</p>
<p style="text-align: center"><img src="https://persistent.info/images/infinite-mac-worker.png" alt="Infinite Mac Requests Graph" width="936" height="630">Cloudflare Workers are pretty nice for handling traffic spikes</p>
<p>Besides the ego boost, I also got a lot of feedback, and have have made a bunch of changes since then. In addition to <a href="https://github.com/mihaip/infinite-mac/commit/580b1b734314f8509823fd4f082f387891d43385">adding</a> <a href="https://github.com/mihaip/infinite-mac/commit/32dea24d99273af6a948c187bdf350669fe234c3">a</a> <a href="https://github.com/mihaip/infinite-mac/commit/8d12820f2b02b7b716fa09e7cde8907be56a9958">few</a> <a href="https://github.com/mihaip/infinite-mac/commit/c411beb41c76aa072108a0b75607610a6c1dd044">more</a> <a href="https://github.com/mihaip/infinite-mac/commit/baf14e9a294df85cffcced5ba7289df421f02aa4">things</a> to the library and making <a href="https://github.com/mihaip/infinite-mac/commit/3788b5deee47aea588081a7e6b1e64e711e431f1">bug</a> <a href="https://github.com/mihaip/infinite-mac/commit/7994629dfe59d2e1a5b66cf4777a40e3b1db5fdc">fixes</a>, the notable changes are:</p>
<ul>
<li>I added <a href="https://kanjitalk7.app/">kanjitalk7.app</a> as a companion site to <a href="https://macos8.app/">macos8.app</a> and <a href="https://system7.app/">system7.app</a>. Besides <a href="https://github.com/mihaip/macemu/commit/ff88805eb9a72b22e5346039108e50ec62c5d7cf">making a new base system image</a>, I also <a href="https://github.com/mihaip/infinite-mac/commit/80510f3f8ae9798ee7e2cb64deb5b6a9427eea0a">changed</a> how the library is stored, keeping it in a separate disk from the OS. This makes uploading and hosting of alternate OSes much easier, since the same library disk image can be used, instead of duplicating ~1GB of data. I probably could have leaned in more on <a href="https://blog.persistent.info/#:~:text=I%20settled%20on%20an%20approach%20where%20the%20disk%20image%20is%20broken%20up%20into%20fixed%2Dsize%20content%2Daddressed%20256K%20chunks.">the content hashing</a> by forcibly aligning files to chunk offsets, but that would have been brittle.</li>
<li>The HFS file system that I was generating had some malformed data structures which became more apparent as the number of files grew (and various B*-tree structures overflowed). After spending a lot of quality time with <a href="https://web.archive.org/web/20031210140152/http://developer.apple.com/documentation/mac/Files/Files-2.html">Inside Macintosh: Files</a> I was able to make <a href="https://github.com/mihaip/machfs/commit/85eeb0b7ea129674913c3bd51659acc534b25a11">two</a> <a href="https://github.com/mihaip/machfs/commit/754b595ffd403534aa7af2fd01a30933ffcff3e5">fixes</a> to the <a href="https://github.com/elliotnunn/machfs"><code>machs</code></a> library and get things working.</li>
<li>The file system also lacked a populated desktop database, which mean that double-clicking on files from other apps did not consistently work. Unfortunately that file format was never reverse engineered, so the easiest way to get it created was to <a href="https://github.com/mihaip/infinite-mac/commit/e9f39384e50b7d3d6cc6cdaac030863f3f77a39d">temporarily boot the image</a>, force a desktop DB rebuild, and then persist the results of that.</li>
<li>A lot of CD-ROM games are archived as disk images (especially in the <a href="https://en.wikipedia.org/wiki/Roxio_Toast">Toast</a> format), so I <a href="https://github.com/mihaip/infinite-mac/commit/800b508a33fe0e7c56adc5e434f63573c99b0373">special-cased the dragging in</a> of those files to instead directly mount them as disks. This unblocks playing of games like <a href="https://macintoshgarden.org/games/myst">Myst</a>.</li>
</ul>
<p>At some point I was reminded of <a href="https://web.archive.org/web/19991013082004/http://gui-central.com">GUI Central</a>, which was my pre-Mscape Software hobby project (it has a brief mention in <a href="https://blog.persistent.info/1998/09/and-so-it-begins.html">my first post</a>). It was a site cataloging Mac customizations (think Kaleidoscope schemes, desktop patterns, etc.). Its main gimmick was that it replicated the 1997-era Mac OS UI in the browser (complete with theme/scheme changing). I have in some ways come full circle.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com1tag:blogger.com,1999:blog-6525469191850690957.post-52117050811479859972022-03-31T19:12:00.008-07:002023-09-11T22:06:10.857-07:00Infinite Mac: An Instant-Booting Quadra in Your Browser<h3>tl;dr</h3>
<p>I’ve extended James Friend’s in-browser Basilisk II port to create a full-featured classic 68K Mac in your browser. You can see it in action at <a href="https://system7.app/">system7.app</a> or <a href="https://macos8.app">macos8.app</a>. For a taste, see also this screencast:</p>
<div style="text-align: center">
<iframe width="800" height="558" src="https://www.youtube.com/embed/tljxs9zuaA8" title="YouTube video player" frameborder="0" allow="autoplay; clipboard-write; encrypted-media; picture-in-picture" allowfullscreen></iframe>
</div>
<h3>Backstory</h3>
<p>It’s a golden age of emulation. Between increasing CPU power, WebAssembly, and retrocomputing being so popular <a href="https://www.nytimes.com/2021/01/08/style/retrocomputing.html">The New York Times is covering it</a>, it’s never been easier to relive your 80s/90s/2000s nostalgia. Projects like <a href="https://github.com/copy/v86">v86</a> make it easy to run your chosen old operating system in the browser. My heritage being of the <a href="https://blog.persistent.info/2021/04/git-resource-fork-hooks.html">classic</a> <a href="https://blog.persistent.info/2021/06/archiving-mscape-software.html">Mac</a> line, I was curious what the easiest to use emulation option was in the modern era. I had earlier <a href="https://blog.persistent.info/2011/10/adventures-in-retro-computing.html">experimented</a> with Basilisk II, which worked well enough, but it was rather annoying to set up, as far as gathering a ROM, a boot image, messing with configuration files, etc. As far as I could tell, that was <a href="https://www.emaculation.com/doku.php/mac_emulation">still the state of the art</a>, at least if you were targeting late era 68K Mac emulation.</p>
<p>Some research into browser-based alternatives uncovered a few options:</p>
<ul>
<li><a href="https://jamesfriend.com.au/">James Friend</a>’s in-browser ports of the <a href="https://jamesfriend.com.au/projects/basiliskii/BasiliskII-worker.html">Basilisk II</a> and <a href="https://jamesfriend.com.au/pce-js/">PCE.js</a> emulators</li>
<li>The <a href="https://blog.archive.org/2017/04/16/early-macintosh-emulation-comes-to-the-archive/">Internet Archive’s Mac Emulation</a>, which is partly based on James’s PCE.js work</li>
<li><a href="https://blog.archive.org/2017/04/16/early-macintosh-emulation-comes-to-the-archive/">OldWeb.Today</a>, which extends James’s Basilisk II work to support a working networking stack</li>
<li><a href="http://retroweb.maclab.org/">RetroWeb Vintage Computer Musem</a>, also based on James’s PCE.js work</li>
<li>The Macintosh Repository is <a href="https://www.emaculation.com/forum/viewtopic.php?t=10763">experimenting with</a> in-browser emulation via either Basilisk II or vMac</li>
</ul>
<p>However, none of these setups replicated the true feel of using a computer in the 90s. They’re great for quickly launching a single program and playing around with it, but they don’t have any persistence, way of getting data in or out of it, or running multiple programs at once. <a href="https://github.com/felixrieseberg/macintosh.js">macintosh.js</a> comes closest to that — it packages James’s Basilisk II port with a large (~600MB) disk image and provides a way of sharing files with the host. However, it’s an Electron app, and it feels wrong to download a ~250MB binary and <a href="https://persistent.info/images/macintosh.js.png">dedicate 1 CPU core</a> to running something that was meant to be in a browser.</p>
<p>I wondered what it would take to extend the Basilisk II support to have a macintosh.js-like experience in the browser, and ideally go beyond it.</p>
<h3>Streaming Storage and Startup Time</h3>The first thing that I looked into was reducing the time spent downloading the disk image that the emulator uses. There was some low-hanging fruit, like actually compressing it (ideally with Brotli), and dropping some <a href="https://github.com/mihaip/macemu/commit/4fd49f0cdf2f7ac914f0ec8273fc6ca331f6cca9">unused data</a> from it. However, it seemed like this goal was fundamentally incompatible with the other goal of putting as much software as possible onto it — the more software there was, the bigger the required disk image.</p>
<p>At this point I switched my approach to downloading pieces of the disk image on demand, instead of all upfront. After some <a href="https://github.com/mihaip/infinite-mac/commit/7ea4f6e6bf762ec59f53ff76b239d5adb0ae3cbd">false</a> <a href="https://github.com/mihaip/infinite-mac/commit/a304dab9bb330d84d56c7b9f595fa5b962dceff2">starts</a>, I settled on <a href="https://github.com/mihaip/infinite-mac/commit/2992609ac3e86f595796759fb8aec6c1c354b298">an approach</a> where the disk image is broken up into fixed-size content-addressed 256K chunks. Filesystem requests from Emscripten are intercepted, and when they involve a chunk that has not been loaded yet, they are sent off to a service worker who will load the chunk over the network. Manually chunking (as opposed to <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests">HTTP range requests</a>) allows each chunk to be Brotli-compressed (ranges technically support compression too, but it’s <a href="https://stackoverflow.com/a/53135659/343108">lacking</a> in the real world). Using content addressing makes the large number of identical chunks from the empty portion of the disk map to the same URL. There is also basic <a href="https://github.com/mihaip/infinite-mac/commit/d537786c9916182b7bd578448a061674ae9ba84c">prefetching support</a>, so that sequential reads are less likely to be blocked on the network.</p>
<p>Along with some <a href="https://github.com/mihaip/infinite-mac/commit/01a78bf70b6a14d969036f0a72bb67289a88bf80">old fashioned web optimizations</a>, this makes the emulator show the Mac’s boot screen in a second, and be fully booted in 3 seconds, even with a cold HTTP cache.</p>
<h3 id="infinite-hd">Building Disk Images, or Docker 1995-style</h3>
<p>I wanted to have a sustainable and repeatable way of building a disk image with lots of Mac software installed. While I could just boot the native version of Basilisk II and manually copy things over, if I made any mistakes, or wanted to repeat the process with a different base OS, I would have to repeat everything, which would be tedious and error-prone. What I effectively wanted was a <a href="https://docs.docker.com/engine/reference/builder/">Dockerfile</a> I could use to build a disk image out of a base OS and a set of programs. Though I didn’t go quite that far, I did end up something that is quite flexible:</p>
<ol>
<li>A bare OS image is <a href="https://github.com/mihaip/infinite-mac/commit/82a8a099116970336791163b14f1983b630ab9d1">parsed</a> using <a href="https://github.com/mihaip/machfs">machfs</a> (which can read and write the HFS disk format)</li>
<li>Software that’s been preserved by the Internet Archive <a href="https://archive.org/details/softwarelibrary_mac">as disk images</a> can be copied into it, by <a href="https://github.com/mihaip/infinite-mac/commit/a06a6df0d63139428d9e5959c47d9dca9cb6d159">reading those images</a> with <code>machfs</code> and merging them in</li>
<li>Software that’s available as <a href="https://en.wikipedia.org/wiki/StuffIt">Stuffit archives</a> or similar is decompressed with the <code>unar</code> and <code>lsar</code> utilities from <a href="https://github.com/mihaip/XADMaster">XADMaster</a> and copied into the image (the <a href="https://macintoshgarden.org/">Macintosh Garden</a> is a good source for these archives).</li>
<li>Software that’s only available as installers is installed by hand, and then the results of that are <a href="https://github.com/mihaip/infinite-mac/blob/main/src/BasiliskII/emulator-ui-extractor.ts">extracted</a> into a zip file that can be also copied into the image.</li>
</ol>
<p>(I later discovered <a href="https://github.com/sfuller/pimpmyplus">Pimp My Plus</a>, which uses a similar approach, including the use of the <code>machfs</code> library)</p>
<p>I wanted to have a full-fidelity approach to the disk image creation, so I had to extend both <a href="https://github.com/mihaip/machfs/commit/2dde60a4032019fc86a00a51ce69db9702fcd5b4">machfs</a> and <a href="https://github.com/mihaip/XADMaster/commit/cf6767ff49a0707d2618dc748ff9c0baf33172e2">XADMaster</a> to preserve and copy Finder metadata like icon positions and timestamps. There was definitely some cognitive dissonance in dealing with late 80s structures in <a href="https://github.com/mihaip/infinite-mac/blob/997ed60a76e148eceab77aff0f8d67f69503a1a6/scripts/build-disk-image.py">Python 3</a> and <a href="https://github.com/mihaip/infinite-mac/blob/a304dab9bb330d84d56c7b9f595fa5b962dceff2/src/BasiliskII/emulator-finder.ts">TypeScript</a>.</p>
<h3>Interacting With The Outside World</h3>
<p>Basilisk II supports mounting a directory from the “host” into the Mac (via the ExtFS module). In this case the host is the pseudo-POSIX file system that Emscripten creates, which has <a href="https://emscripten.org/docs/api_reference/Filesystem-API.html">an API</a>. It thus seemed possible to handle <a href="https://github.com/mihaip/infinite-mac/commit/ec058cea53a57b159cdd87c2631ed8d2bd72bca5">files being dragged</a> into the emulator by reading them on the browser side and sending the contents over to the worker where the emulator runs, and creating them in a “Downloads” folder. That worked out well, especially once I switched a <a href="https://github.com/mihaip/infinite-mac/commit/3cbaac0d49cca9af067cbd08dda07de44f14f3da">custom lazy file implementation</a> and <a href="https://github.com/mihaip/macemu/commit/80e1038734b1f78980b0315c2ca1483f3293561a">fixed encoding issues</a>.</p>
<p>To get files out, the reverse process can be used, where files in a special “Uploads” folder are <a href="https://github.com/mihaip/infinite-mac/commit/45a2c5bb275499d30e92354745fb74fb36155a66">watched</a>, and when new ones appear, the contents are sent to the browser (as a single zip file in the case of directories).</p>
<h3 id="infinite-mac-persistence">Persistence</h3>
<p>While Emscripten has an <a href="https://emscripten.org/docs/api_reference/Filesystem-API.html#filesystem-api-idbfs">IDBFS mode</a> where changes to the filesystem are persisted via IndexedDB, it’s not a good fit for the emulator, since it relies on there being an event loop, which is not the case in the emulator worker. Instead I used an approach <a href="https://github.com/mihaip/infinite-mac/commit/0899205dba8fc3104f6a4f65a1714816720fe67b">similar to uploading</a> to send the contents of a third ExtFS “Saved” directory, which can then be persisted using IndexedDB on the browser side.</p>
<h3>Performance</h3>
<p>The emulator using 100% of the CPU seems like a fundamental limitation — it’s simulating another CPU, and there’s always another instruction for it to run. However, Basilisk II is working at a slightly higher-level, and it <a href="https://github.com/mihaip/macemu/commit/adc4078914464794af33efa47c25455791aefb69">knows</a> when the Mac is idle (and waiting for the user input), and allows the host to intercept this and yield execution. I <a href="https://github.com/mihaip/macemu/commit/adc4078914464794af33efa47c25455791aefb69">made that work</a> in the browser-based version by using <code>Atomics</code> to wait until either there was user input or a screen refresh was required, which dropped CPU utilization significantly. A <a href="https://blog.persistent.info/2021/08/worker-loop.html">previous blog post</a> has more details, including the hoops required to get it working in Safari (which are thankfully not required with <a href="https://developer.apple.com/documentation/safari-release-notes/safari-15_2-release-notes#Security">Safari 15.2</a>).</p>
<p>The bulk of the remaining time was spent updating the screen, so I <a href="https://github.com/mihaip/macemu/commit/3738e1e2e2ed771a7683dd4c4491392736020330">made</a> <a href="https://github.com/mihaip/macemu/commit/973752a94ab4b1bc2ff75d9e66577330f677775c">some</a> <a href="https://github.com/mihaip/macemu/commit/2dbd859a74da4e347503ebdc0d8067a47e549fd4">optimizations</a> there to do less per-pixel manipulation, avoid some copies altogether, and not send the screen contents when they haven’t changed since the last frame.</p>
<p>The outcome of all this is that the emulator idles at ~13% of the CPU, which makes it much less disruptive to be left in the background.</p>
<h3>Odds and Ends</h3>
<p>There were a bunch more polish changes to improve the experience: making it responsive to <a href="https://github.com/mihaip/infinite-mac/commit/9081875a5f9d10c94e8fbada8771e6b1a5cb58a0">bigger</a> and <a href="https://github.com/mihaip/infinite-mac/commit/5e0dceb2ddd2d049afd412bd776ea445ef91c4a9">smaller</a> screens, <a href="https://github.com/mihaip/infinite-mac/commit/df9ecaef4d8db2a0f14199ec6bfc235d05ef98b5">handling touch events</a> so that it’s usable on an iPad (though double-taps are still tricky), <a href="https://github.com/mihaip/macemu/commit/5e822370c3b152822a56d18df6087011f8713d20">fixing the scaling</a> to preserve crispness, handling <a href="https://github.com/mihaip/macemu/commit/c5f7a41845274e43cfc59ce8556aeb290e682daf">other color modes</a>, <a href="https://github.com/mihaip/macemu/commit/25d10504e9e31ea41e1be2b4a9393bfc69ded100">better keyboard mapping</a>, and much more.</p>
<p>There is a ton <a href="https://github.com/mihaip/infinite-mac/issues">more work to be done</a>, but I figured <a href="https://www.marchintosh.com/">MARCHintosh</a> was as good a time at any to take a break and share this with the world. Enjoy!<br />
<p><b>Update:</b> See also the discussion on <a href="https://arstechnica.com/gadgets/2022/04/boot-up-classic-mac-os-in-your-browser-window-with-the-infinite-mac-project/?comments=1">Ars Technica</a> and <a href="https://news.ycombinator.com/item?id=30875259">Hacker News</a> (<a href="https://news.ycombinator.com/item?id=31168646">take 2</a>). There is also a <a href="https://blog.persistent.info/2022/05/infinite-mac-75-weeks-later.html">follow-up blog post</a> with some post-launch details, and another describing the <a href="https://blog.persistent.info/2022/07/infinite-mac-networking.html">implementation of networking</a>.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com66tag:blogger.com,1999:blog-6525469191850690957.post-70962173755145210722021-12-03T18:19:00.003-08:002021-12-03T18:19:35.742-08:00A-12 Software Development Parallels <p>I recently finished reading <a href="https://arc.aiaa.org/doi/book/10.2514/4.867316">From RAINBOW to GUSTO</a> which describes the development of the <a href="https://en.wikipedia.org/wiki/Lockheed_A-12">A-12</a> high-speed reconnaisance plane (the predecessor to/basis for the somewhat better known <a href="https://en.wikipedia.org/wiki/Lockheed_SR-71_Blackbird">SR-71 Blackbird</a>). Though a bit different from the <a href="https://press.stripe.com/the-making-of-prince-of-persia">software</a> <a href="https://www.amazon.com/Showstopper-Breakneck-Windows-Generation-Microsoft/dp/1497638836">history</a>/<a href="https://www.cs.princeton.edu/~bwk/memoir.html">memoirs</a> that I've also enjoyed, I did find some parallels.</p>
<p>Early on in the book, when Edwin Land (founder of Polaroid) is asked to put together a team to research ways of improving the US’s intelligence gathering capabilities, there's the mid-century analog of the <a href="https://docs.aws.amazon.com/whitepapers/latest/introduction-devops-aws/two-pizza-teams.html">two-pizza team</a>:
<blockquote>Following Land’s “taxicab rule” — that to be effective a working group had to be small enough to fit in a taxi — there were only five members.</blockquote>
<p>It turns out that cabs in the 1940s had to <a href="https://en.wikipedia.org/wiki/Checker_Taxi#:~:text=seating%20for%20five%20passengers%20in%20the%20rear%20compartment">seat 5 in the back seat</a> – I suppose the modern equivalent would be the "Uber XL rule".</p>
<p>Much later in the book, following the A-1 to A-11 design explorations, there was an excerpt from <a href="https://en.wikipedia.org/wiki/Kelly_Johnson_(engineer)">Kelly Johnson</a>’s diary when full A-12 development had started:</p>
<blockquote>Spending a great deal of time myself going over all aircraft systems, trying to add some simplicity and reliability.</blockquote>
<p>That reminded me of design, architecture and production reviews, and how the simplification of implementations is one of the more important pieces of feedback that can be given. Curious to find more of Johnson's log, I found that <a href="https://www.amazon.com/Lockheed-SR-71-Blackbird-Crowood-Aviation/dp/1861266979">another book</a> has an abridged copy. I've OCRed and cleaned it up and put it online: <a href="https://quip.com/Zb5ZAmKcTiSc/A-12-Log-by-Kelly-Johnson">A-12 Log by Kelly Johnson</a>.</p>
<p>It's a <a href="http://blog.idonethis.com/google-snippets-internal-tool/">snippets</a>-like approximation of the entire A-12 project, and chronicles the highs and lows of the project. I highlighted the parts that particularly resonated with me, whether it was Johnson's healthy ego, delays and complications generated by vendors, project cancelations, bureaucracy and process overhead, or customers changing their minds.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com0tag:blogger.com,1999:blog-6525469191850690957.post-37193454210434763492021-08-24T08:30:00.004-07:002022-03-31T19:14:25.054-07:00Communicating With a Web Worker Without Yielding To The Event Loop<p>I recently came across James Friends’s work on <a href="https://jamesfriend.com.au/">porting the Basilisk II classic Mac emulator to run in the browser</a>. One thing that I liked about his approach is that <a href="https://jamesfriend.com.au/basilisk-ii-classic-mac-emulator-in-the-browser#performance">it uses <code>SharedArrayBuffer</code></a> to allow the emulator to run in a worker with minimal modifications. This system can also be extended to use <code>Atomics.wait</code> and <code>Atomics.notify</code> to <a href="https://github.com/mihaip/macemu/commit/adc4078914464794af33efa47c25455791aefb69">implement idlewait support</a> in the emulator, significantly reducing its CPU use when the system is in the Finder or other applications that are mostly waiting for user input.</p>
<p>James’s work is from 2017, which is from before the Spectre/Meltdown era. Browsers have since then disabled <code>SharedArrayBuffer</code> and then <a href="https://developers.google.com/search/blog/2021/03/sharedarraybuffer-notes">brought it back with better safety/isolation.</a> The exception to this is (not surprisingly) Safari. Though there have been some <a href="https://trac.webkit.org/changeset/269531/webkit/">signs</a> <a href="https://trac.webkit.org/changeset/272341/webkit/">of</a> <a href="https://trac.webkit.org/changeset/274438/webkit">life</a> in the WebKit repository, it’s unclear when/if it will arrive.</p>
<p>I was hoping to resurrect James’s emulator to run in all modern browsers, but having to support an entirely different code path for Safari (e.g. using <a href="https://emscripten.org/docs/porting/asyncify.html">Asyncify</a>) did not seem appealing.</p>
<p>At a high level, <a href="https://excalidraw.com/#json=5297893569200128,4Nx6zcjEKHupR3IkBML7VA">this diagram</a> shows what the communication paths between the page and the emulator worker are:</p>
<p style="text-align: center">
<img alt="Page and worker communication" src="https://persistent.info/images/worker-loop-sab.png" width="792" height="317">
</p>
<p>Sending the output is possible even without <code>SharedArrayBuffer</code> — <code>postMessage</code> can be used even though the worker never yields to the event loop (because the receiving page does). The problem is going in the other direction — how can the worker know about user input (or other commands) if it can’t receive a <code>message</code> event.</p>
<p>I was going through the list of <a href="https://developer.mozilla.org/en-US/docs/Web/API/WorkerGlobalScope">functions available to a worker</a> when I was reminded of <a href="https://developer.mozilla.org/en-US/docs/Web/API/WorkerGlobalScope/importScripts"><code>importScripts</code></a><sup>1</sup>. As its documentation says, this <b>synchronously</b> imports (and executes) scripts, thus it does not require yielding to the event loop. The problem then becomes: how can the page generate a script URL that encodes the commands that it wishes to send? My first thought was to have the page construct a <code>Blob</code> and then use <code>URL.createObjectURL</code> to load the script. However, blobs are immutable and the contents (passed into the <a href="https://developer.mozilla.org/en-US/docs/Web/API/Blob/Blob">constructor</a>) are read in eagerly. This means that while it’s possible to send one blob URL to the worker (by telling it what the URL is before it starts its <code>while (true) {...}</code> loop), it’s not possible to tell it about any more (or somehow “chain” scripts together).</p>
<p>After thinking about it more, I wondered if it’s possible to use a <a href="https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API">service worker</a> to handle the <code>importScripts</code> request. The (emulator) worker could then repeatedly fetch the same URL, and rely on the service worker to populate it with commands (if any). The service worker has a normal event loop, thus it can receive <code>message</code> events without any trouble. <a href="https://excalidraw.com/#json=6720884081426432,bOa9fBqEfwI5659O1eT4VA">This diagram</a> shows how the various pieces are connected:</p>
<p style="text-align: center">
<img alt="Page, worker and service worker communication" src="https://persistent.info/images/worker-loop-service-worker.png" width="843" height="670">
</p>
<p><a href="https://persistent.info/web-experiments/worker-loop/">This demo</a> (<a href="https://github.com/mihaip/web-experiments/commit/b6642748d6ff8c7d9a72db4369583ae596f68bba">commit</a>) shows it in action. As you can see, end-to-end latency is not great (1-2ms, depending on how frequently the worker polls for commands), but it does work in all browsers.</p>
<p>I then implemented this approach as a fallback mode for the emulator (<a href="https://github.com/mihaip/transparent-mac/commit/68730802cb05f76945a626c5eaf9cdf4311e49e4">commit</a>), and it appears to work surprisingly well (the 1-2ms of latency is OK for keyboard and mouse input). As a bonus, it’s even possible (<a href="https://github.com/mihaip/transparent-mac/commit/3d6ef22ecd305396c8bc89e6a37c8ac0268bc96e">commit</a>) to use a variant of this approach to implement idlewait support without <code>Atomics</code>, thus reducing the CPU usage even in this fallback mode.</p>
<p>You can see the emulator at <a href="https://mac.persistent.info/">mac.persistent.info</a> (you can force the non-<code>SharedArrayBuffer</code> implementation with the <a href="https://mac.persistent.info/?use_shared_memory=false"><code>use_shared_memory=false</code></a> query parameter). Input responsiveness is still pretty good, compared with <a href="https://jamesfriend.com.au/projects/basiliskii/BasiliskII-mainthread.html">the version</a> (<a href="https://github.com/jsdf/macemu/commit/0ca2e3ab9b2b1e30144cac30413e19d2e6819e17">commit</a>) that uses <code>emscripten_set_main_loop</code> and regularly yields to the browser. Of course, it would be ideal if none of these workarounds were necessary — perhaps WWDC 2022 will bring us cross-origin isolation to WebKit.
<p><b>Update on 2022-03-31:</b> <a href="https://developer.apple.com/documentation/safari-release-notes/safari-15_2-release-notes#Security">Safari 15.2</a> added support for <code>SharedArrayBuffer</code> and <code>Atomics</code>, thus removing the need for this workaround for recent versions. We didn't have to wait for WWDC 2022 after all.</p>
<ol style="border-top: solid 1px #aaa; padding-top: 3px;" class="footnotes">
<li>It later occurred to me that synchronous <code>XMLHttpRequest</code>s might be another communication mechanism, but the effect would most be the same (the only difference is more flexibility in the output format, e.g. the contents of an <code>ArrayBuffer</code> could be sent over, thus better replicating the <code>SharedArrayBuffer</code> experience)</li>
</ol>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com1tag:blogger.com,1999:blog-6525469191850690957.post-54278408710093207272021-06-03T09:51:00.002-07:002021-06-03T09:51:26.870-07:00Archiving Mscape Software on GitHub<p><img src="https://persistent.info/images/mscape-software-title.gif" alt="Mscape Software" width="374" height="58" style="float:right;padding-left:10px"><a href="https://mscape.com/">Mscape Software</a> was the “label” that I used in my late teenage years for Mac shareware programs. While having such a fake company was (is?) <a href="https://twitter.com/awesomekling/status/1332967857676820480">a surprisingly common thing</a>, it turned into a pretty real side-gig during 1999 to 2003. I spent a lot of my hobby programming time working on <a href="https://macintoshgarden.org/apps/iconographer-24">Iconographer</a>, an icon editor for the new-at-the-time 32-bit <code><a href="https://en.wikipedia.org/wiki/Apple_Icon_Image_format">icns</a></code> icon format introduced with MacOS 8.5 (and extended more with the initial release of Mac OS X). The <a href="https://blog.persistent.info/1998/">early</a> <a href="https://blog.persistent.info/1999/">entries</a> of this blog describe its initial development in pretty high detail — the deal that I had with my computer class teacher was that I wouldn’t have to do any of the normal coursework as long as I documented my progress.</p>
<p>All of that wound down as I was finishing up college, and I officially <a href="https://blog.persistent.info/2008/02/decommissioning-mscape-software.html">decommissioned the site</a> in 2008. I’ve been on a bit of a retro-computing kick lately, partially inspired by listening to some of the <a href="https://www.computerhistory.org/collections/oralhistories/">oral histories</a> compiled by the Computer History Museum, and I was reminded of this phase of my programming career. Over the years I’ve migrated everything to GitHub, which has turned it into an effective archive of everything open source that I’ve done (it also makes for some good <a href="https://retrogit.com/">RetroGit</a> emails), but this earliest period was missing.</p>
<p>I didn’t actually use version control at the time, but I did save periodic snapshots of my entire development directories, usually tied to public releases of the program. It’s possible to <a href="https://alexpeattie.com/blog/working-with-dates-in-git/#:~:text=Manually%20set%20GIT_AUTHOR_DATE%20and%20GIT_COMMITTER_DATE">backdate commits</a>, and thus with the help of <a href="https://github.com/mihaip/mscape/blob/main/tools/commit.py">a script</a> and some <a href="https://blog.persistent.info/2021/04/git-resource-fork-hooks.html">custom tooling to make Git understand resource forks</a> I set about recreating the history. The biggest time sink was coming up with reasonable commit messages — nothing like puzzling over diffs <a href="https://github.com/mihaip/mscape/commit/9a76511ed14ca93eeb55d206b15db00f68512f3f">from 23 years ago</a> to understand what the intent was. Luckily by the later stages I had started to keep <a href="https://github.com/mihaip/mscape/blob/main/software/Iconographer/Release%20Notes.txt">more detailed release notes</a>, which helped a lot.</p>
<p><a href="https://github.com/mihaip/mscape">github.com/mihaip/mscape</a> is the result of the archiving efforts, and it’s showing up as expected on my profile:</p>
<p style="text-align: center"><img src="https://persistent.info/images/mscape-1998-commits.png" alt="GitHub commits from 1998" with="753" height="166"></p>
<p>I tried to be comprehensive in what is committed, so there is a fair bit of noise with build output and intermediate files from <a href="https://en.wikipedia.org/wiki/CodeWarrior#History">CodeWarrior</a>, manual test data, and the like. The goal was that a determined enough person (perhaps me in a few more years) would have everything needed to recompile (there are still <a href="https://www.highcaffeinecontent.com/blog/20150124-MPW,-Carbon-and-building-Classic-Mac-OS-apps-in-OS-X">toolchains</a> for doing Classic mac development).</p>
<p>It’s been interesting to skim through some of this code with a more modern eye. Everything was much lower-level — the event loop was not something that you could only be <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/EventLoop#event_loop">vaguely aware of</a>, it was <a href="https://github.com/mihaip/mscape/blob/b6828765f74d1d1e4f9b37e068f5b5cca95723b2/software/MFrame/MStandard/MApplication/MApplication.cpp#L81-L107">literally a loop</a> in your program (and all other programs). Similarly, you had <a href="https://github.com/mihaip/mscape/blob/a3c057dd472fd999b055eefc67e620a293de5d12/software/MFrame/MStandard/MUtilities.cpp#L25-L60">initialize everything by hand</a>, do (seemingly magical) incantations to <a href="https://web.archive.org/web/20030831182636/http://developer.apple.com/documentation/mac/Memory/Memory-17.html">request more master pointers</a>, and make sure to <a href="https://github.com/mihaip/mscape/blob/a3c057dd472fd999b055eefc67e620a293de5d12/software/Iconographer/Source/icnsEditorClass/PreviewPalette/PreviewPalette.cpp#L625">lock</a> (and <a href="https://github.com/mihaip/mscape/blob/a3c057dd472fd999b055eefc67e620a293de5d12/software/Iconographer/Source/icnsEditorClass/PreviewPalette/PreviewPalette.cpp#L628">unlock</a>) your handles. If you want to learn more about Classic Mac Toolbox programming, this <a href="https://mikeash.com/pyblog/friday-qa-2012-01-13-the-mac-toolbox.html">pair of</a> <a href="https://www.mikeash.com/pyblog/the-mac-toolbox-followup.html">blog posts</a> provide more context. Had I been aware of patterns like <a href="https://en.wikipedia.org/wiki/Resource_acquisition_is_initialization">RAII</a>, there would have been a lot less boilerplate (and crashing).</p>
<p>Speaking of C++ patterns, there are a bunch of cringe-worthy things, especially abuse of (multiple) inheritance. Need to make a class that represents an icon editor? Have it subclass from both <a href="https://github.com/mihaip/mscape/blob/fc83fc85dc40e4f171d759a916073b299e360d87/software/Iconographer/Source/icnsEditorClass/icnsEditorClass.h#L371">an icon class and a document window class</a>. It was nice to see some progression over the years to <a href="https://github.com/mihaip/mscape/commit/dbdf37b8b141a05c03ca2a86cbfcea495f2abfa8">better encapsulation</a> and <a href="https://github.com/mihaip/mscape/commit/18ed532618919e273809949058f1356561669e13#diff-4bb791dda4f8cdb17cf6ade8d69849cb1695ef8d2c6a45ede9958c80b9e92501R2663">data-driven code instead of boilerplate</a>.</p>
<p>Another difference in approach was that there was a much bigger focus on backwards compatibility. clip2cicn and clip2icns both had 68K versions, despite it being 4-5 years since the transition to PowerPC machines begun. clip2icns and Iconographer both used home-grown icon manipulation routines (including ones that <a href="https://blog.persistent.info/1998/10/rle-decyphered.html">reverse-engineered</a> the compression format) so that they could run on MacOS 8.1 and earlier, despite the icon format they targeted being 8.5-only. Iconographer only dropped Classic Mac OS support in 2003, more than 2 years after the release of Mac OS X. If I had to guess, I would attribute that to at least my not making rational trade-offs: would people that were hanging on to 5-year-old hardware be spending money on an icon editor? But I would also assume that Mac users tended to hang on their hardware for quite a while, presumably due to the higher cost.</p>
<p>On the business side, Brent Simmons’s recent article on <a href="https://inessential.com/2021/05/07/what_it_was_like_to_sell_apps_online_in_2003">selling apps online in 2003</a> pretty much describes my approach. I too used Kagi for the storefront and credit card processing, and an automated system that would send out registration codes after purchase. Iconographer ended selling 3,500 copies (the bulk in 2000-2003), which was pretty nice pocket change for a college student. On a lark I recreated the purchasing flow for 2021 <a href="https://mscape.com/stripe/checkout.html">using Stripe</a> and it appears to be even more painless now, so modulo <a href="https://en.wikipedia.org/wiki/Epic_Games_v._Apple">gatekeepers</a>, this would still be a feasible approach today.</p>Mihai Parparitahttp://www.blogger.com/profile/12343650264888591427noreply@blogger.com2