[sword-devel] HTML5 File API and SWORD modules

[Greg Hellings]

On Tue, Jul 31, 2012 at 2:08 AM, Stephan <info at tetzels.de> wrote:
> Hi,
>
> I'm thinking about to build a HTML5 frontend that works offline in a
> browser. I know that there were some discussions before about this topic,
> but I want to know if it is possible to read the sword modules with the
> HTML5 File API (http://www.w3.org/TR/FileAPI/)?

The SWORD module files are pretty heavily composed of a proprietary
binary format which is not documented anywhere outside of the C++ and
Java source code for SWORD and JSword. It has been a little while
since I looked into the limitations of browsers and their file APIs.
As long as it can read and write binary files, it should have no
problem with the files themselves. I have always found manipulating
binary data in JavaScript to be a little bit of a dark art because it
is so infrequently attempted as to be little documentation of it.

>
> Converting the modules to JSON format may be also an option, although there
> is the 2.5/5MB size limit for localStorage / IndexDB, but it would be easier
> to read such files.

Work has been undertaken by some people to accomplish this. Purists
will tell you it's a bad idea because it risks losing some amount of
this, that, or the other thing. In reality it is not much different
SWORD's binary format where verses have well defined start and end
boundaries and the rest of the contents is simply a block of text with
some attached notes/attributes/etc.

If converting the modules in JavaScript is not much help (you run into
the file read/parse issue) you could do the conversion on the server
side. There are working Perl and Python bindings, so you could write a
web service in your favorite framework there. You can always tackle
whatever it is that swordweb uses to do conversion of the data to
JSON. Likewise you can use the JSword port or the engine's JNI
bindings to write an app in Java that will output the JSON data.

Also, storing as JSON and storing in IndexDB are hardly dissimilar.
Storing as IndexDB would just give you faster retrieval times and
provide you the benefits of indexing and searching on arbitrary fields
in a faster, simpler way than loading an parsing an entire JSON file
for each module every time you load it. But fundamentally the IndexDB
storage system, as I have read it, just stores a JSON-like object for
retrieval. The only problem becomes requesting the browser for
additional storage when you reach the 5MB or 25MB limit or wherever it
is set. You also have the advantage that IndexDB is supported in all
major desktop browsers either natively (all the good ones) or through
a plugin developed by MS as their sort of alpha/beta testing of the
technology for IE.

>
> I already build an SWORD NPAPI plugin (32bit, linux only,
> http://www.mail-archive.com/sword-devel@crosswire.org/msg24393.html) to get
> my Bible App working in Chrome/Chromium, but I'm looking for a full offline
> web app and not a hybrid solution.

Offline storage is going to be your biggest issue. Web SQL database
was supposed to be held to, I believe, a 25MB limit and IndexDB is
probably similar. You can't fit very many modules into that space. For
Chrome, your app could always be bundled as a plugin which requests
the unlimited storage permission. Firefox can also handle pure-HTML/JS
plugins and probably has a similar permission request flag. IE, and
mobile browsers (and possibly Opera) will probably be your ares where
this comes into play. Then again, on mobile you're going to need an
entirely different UI already and it probably doesn't support the File
APIs.

My own forays into this area have taught me that it's best to write
using a framework where you can easily abstract your application away
from the particular storage mechanism you put into place. Using a big
JavaScript framework like ExtJS (GPLv3 or commercial) or Qooxdoo (LGPL
or Eclipse PL) or similar allows you to get a traditional class
inheritance structure and hide details like the storage implementation
from your application. You might need to write a few different
versions of that class to handle the different browser configurations
around, but in the end it is probably worth it to help keep your
sanity and provide increased portability in the future to your app.

--Greg

>
> Best Regards,
> Stephan
>
>
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page