[sword-devel] JavaScript & SWORD modules

[Greg Hellings]

On Sun, Jan 19, 2014 at 5:19 PM, Stephan <info at tetzels.de> wrote:

> Hi,
>
>  ...very immature or focused heavily on pure JS work that producing Sword
>>
>> bindings ranges from impossible (in the browser) to questionable
>> (browser plugins) to an unexplored unknown (server-side plugins).
>>
>
> Last year I've started serveral efforts to access SWORD modules in
> Javascript. Since Javascript supports manipulating and handling binary data
> is no problem to access sword modules in Javascript if you know how to read
> them.
>

It's that last phrase, "...if you know how to read them" that I wanted to
obviate the need for.


>
> First I've tried to compile sword to JS with Emscripten, but without real
> success. After that I've started to work on a serverside nodejs
> implementation (pure JS and a wrapper around the native C++ lib). You can
> find the sources here:
>
> * https://github.com/zefanja/node-swordjs
> * https://github.com/zefanja/sword-addon


Drawing emphasis to my contention that server-side plugins are still a very
immature option at this point.


>
>
> Because every serverside solution has the problem to solve licence and
> sword module distribution issues I've ported my efforts to browser-side JS.
>
> The project is still a very early stage (and my free time is limited), but
> you are able to install and access sword modules in your browser. You can
> take a look at http://zefanjas.de/biblezng to see a working demo.
> Currently it supports Firefox only, because Chrome/IE don't support file
> blobs in indexedDB.
>
> You'll find the sources on Github, too.
> * the library: https://github.com/zefanja/swordjs
> * the reference app: https://github.com/zefanja/biblez-ng
>
> Perfomance is good in a the browser and acceptable on a low-end FirefoxOS
> device. But there is much room for optimizations (web workers, ...).
>

This exactly exposes the problem: you now have 5 attempts to work on this
spanning the course of a year. I'm sure in large part some of your
difficulty stems from the fact that the format of the Sword module files is
undocumented. There are multiple file formats you need to support, there's
three levels of encryption possible on the content files, and there's two
different encryption algorithms supported (I'd be curious how the
performance of a book-level LZSS encrypted module shows up in FirefoxOS.

The difficulty with the approaches you've taken - except for your NodeJS
plugin which doesn't help front-end JavaScript programmers like me - is
that you've exposed yourself to a very high amount of effort if and when
the format of a Sword module file changes. Currently we support zlib and
LZSS compression of modules as they excel at speed (zlib) and maximal space
savings (LZSS) with a preference for the zlib option because we don't feel
that storage space is a major concern. But what if, in the future, Sword
transitions to preferring some sort of middle ground and now you'll need to
implement bzip2 compression? Or what if the structure of modules changes to
depend on per-module defined versification schemes and leverages the
"VerseTreeKey" that has been stubbed out and toyed with in the past? Or
what if, in the far distant future, a new format of module is added that
offers the possibility of storing data in a SQLite database? Now you need
to go an implement your own code to interact with any of these
possibilities. There are features in the C++ library that took a long time
even to trickle down into JSword - like support for alternate
versifications.

CrossWire only has two implementations of record for Sword: the C++ Sword
engine and JSword. This is not to say we don't permit other people to
access the modules or fight against people trying to import the data. There
have been implementations that have read Sword modules into pre-rendered
HTML files, into MySQL databases, and more. However, we have shied away
from re-implementing the engine for those reasons above. Even when giving
access in a language that supports binary file IO even better than
JavaScript does (Objective-C, Python, Perl, C, and Java through the JNI
bindings to name only a few) we have always encouraged people to tap into
the existing module system. Even the Sword study tool (
http://www.crosswire.org/study/) runs pretty much right out of the C++
library bindings implemented through CORBA. This prevents each
implementation from needing to re-implement and re-maintain the same code,
over and over again. A number of other efforts have leveraged JSword in
similar ways.

All I wished to do was demonstrate that the same could be accessed by
JavaScript programmers with only a minor amount of server-side work.

--Greg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20140119/4b26c832/attachment.html>