Background
Currently, Arcalibre has about 27k lines of code inherited from Calibre written in RapydScript-NG (ext *.pyj), a compile-to-JavaScript language intended to be close enough to Python to allow for copying and pasting some code between the two.
RapydScript-NG itself is Kovid Goyal’s fork of RapydScript, and as far as I am able to tell, is used exclusively in Calibre and its forks. The original Calibre build includes a snapshot of the RapydScript-NG compile as an xz-archive, itself minified from the compiler by the bin/web-repl-export script in the RapydScript-NG repo as called by calibre.utils.rapydscript.update_rapydscript.
Calls into RapydScript-NG language code currently use the QtWebEngine set of Qt components backed by the "headless" QPA plugin bundled with Calibre, which adds a significant additional chain of dependencies above and beyond the RapydScript-NG compiler itself. Notably, this chain of dependencies is responsible for some Arcalibre failures and other failures downstream of the original Calibre codebase wherein QtWebEngine fails to initialize a Vulkan context.
Moving Forward
Given that RapydScript-NG is a large additional dependency and is less needed given the state of modern JavaScript (it’s a much nicer language now than it was in 2012, when version 0.1.0 of RapydScript was first released!), if Arcalibre is going to be a maintainable fork of Calibre and not only an archival (see https://forums.rereading.space/post/6), we’ll likely need to rewrite RapydScript-NG code in pure JavaScript, or at most some well-established compile-to-JS language like TypeScript. That in turn raises a question: how do we do that? The contents of src/pyj currently contain a number of interdependent module imports, making it difficult to switch to more conventional JavaScript.
Footnote about TypeScript
While I don’t necessarily advocate for TypeScript, it’s worth noting that that would still reduce the weight of external dependencies, as it can be obtained and executed using npx rather than via a bespoke compiler and a toolchain that includes an implicit Vulkan dependency. Pure JavaScript requires even less in the way of dependencies, and may be better on that basis, to be sure.
Questions
- Should we remove RapydScript-NG code?
- What should replace it? Pure JavaScript, TypeScript, or something else?
- If we do remove RapydScript-NG code, how should we go about doing so?
I’ve looked into Python/WASM as a possible plugin mechanism to allow either non-Python plugins, or decoupling plugins from the venv used by Calibre itself, and it definitely seems feasible? That said, some of the RS code seems to go into web contexts that may or may not yet support WASM, I don’t know how well QtWebEngine plays with WASM at moment, most of what I’ve seen goes (weirdly enough) the other way, embedding Qt in WASM contexts.
I really like the idea, but I think I fall more towards what you note, that it’s more risky at this point than TypeScript, and a higher bar to contribution.