Background

Currently, Arcalibre has about 27k lines of code inherited from Calibre written in RapydScript-NG (ext *.pyj), a compile-to-JavaScript language intended to be close enough to Python to allow for copying and pasting some code between the two.

RapydScript-NG itself is Kovid Goyal’s fork of RapydScript, and as far as I am able to tell, is used exclusively in Calibre and its forks. The original Calibre build includes a snapshot of the RapydScript-NG compile as an xz-archive, itself minified from the compiler by the bin/web-repl-export script in the RapydScript-NG repo as called by calibre.utils.rapydscript.update_rapydscript.

Calls into RapydScript-NG language code currently use the QtWebEngine set of Qt components backed by the "headless" QPA plugin bundled with Calibre, which adds a significant additional chain of dependencies above and beyond the RapydScript-NG compiler itself. Notably, this chain of dependencies is responsible for some Arcalibre failures and other failures downstream of the original Calibre codebase wherein QtWebEngine fails to initialize a Vulkan context.

Moving Forward

Given that RapydScript-NG is a large additional dependency and is less needed given the state of modern JavaScript (it’s a much nicer language now than it was in 2012, when version 0.1.0 of RapydScript was first released!), if Arcalibre is going to be a maintainable fork of Calibre and not only an archival (see https://forums.rereading.space/post/6), we’ll likely need to rewrite RapydScript-NG code in pure JavaScript, or at most some well-established compile-to-JS language like TypeScript. That in turn raises a question: how do we do that? The contents of src/pyj currently contain a number of interdependent module imports, making it difficult to switch to more conventional JavaScript.

Footnote about TypeScript

While I don’t necessarily advocate for TypeScript, it’s worth noting that that would still reduce the weight of external dependencies, as it can be obtained and executed using npx rather than via a bespoke compiler and a toolchain that includes an implicit Vulkan dependency. Pure JavaScript requires even less in the way of dependencies, and may be better on that basis, to be sure.

Questions

  • Should we remove RapydScript-NG code?
  • What should replace it? Pure JavaScript, TypeScript, or something else?
  • If we do remove RapydScript-NG code, how should we go about doing so?
    • tempestEnglish
      1·
      13 days ago

      I did take a look at that, I don’t think that pytest specifically is going to be that helpful for this as ideally I would like to test these modules after being compiled to Javascript — so then the same tests can be run on each as they’re replaced

      With that in mind I think it might make sense to make a subdirectory inside the tests/ directory for JS tests, and then add a separate Justfile command for running jest or vitest or another Javascript test runner (I haven’t looked into what would be the best for this yet, those are just ones I have used previously)