[sword-devel] Remote Module Repository Wiki

[Troy A. Griffitts]

There are a number of finely delineated topics here we should separate.

1) Data format for our "module installation source".

  i.e, is it a list of zips, is it an existing install set, etc.

  Simply to be explicit, we have obvious use cases which come into play:

  a) Installing our software -- all implementations will be different
because we have a variety of packages running on a variety of platforms.
  b) Installing modules from a module installation source
  c) Running our software

The final item (c) seems out of place but really is important.  We have
successfully achieved a common data storage format for an installation
set of books (i.e. "library/mods.d/..., library/modules/...) across a
vast number of platforms.  Sure this hierarchy of files moves around to
different locations on different platforms, but once the location is
pinpointed, everything under that location remains the same for any of
our platforms.

The 'holy grail' for implementation of the (1) above is for both (b) and
(c) use cases to be handled by the SAME dataset, and is what we are
working towards.  In other words, if an installed module library can,
itself, also be an installation source for other instances of any SWORD
software, then distribution of Bibles becomes viral. This brings us to
topic 2.

2) The viral ability END USERS to share modules.

If I can, for example,

o	use the builtin installer of my SWORD software on my Android device to
install my important modules
o	then plug it into my friends computer and install modules FROM my
android device WITHOUT ANY PREPARATION
o	then from his computer right-click on the top module library folder
and "Share as... 'SWORD Modules'",
o	then go to any computer on the network and install modules from that
"SWORD Modules" share,
o	then install FileZilla on one of those computers and let anyone over
the Internet install Bibles from that location,
o	then point IIS or Apache to that location (http transport needs work
but is on the way-- more talk below)

... distribution of Bibles then becomes viral.  It is spontaneous.  No
preparation.  I have my library of books I use regularly with me and if
someone wants one of my books, then they can install straight from my
library.


3) The TRANSPORT TECHNOLOGY used to enable this over the Internet.

We chose FTP because it was more practical.  It is very difficult to
implement the programmatic traversal of a directory and file hierarchy
over HTTP.  Apache and IIS all spew out different html for directory
listings.

Now having said this, Nic Carter submitted the first pass at this which
works for our apache server (and likely many apache servers which spew
out the vanilla html index page like here:
http://crosswire.org/ftpmirror/pub/

So, HTTP is currently possible in the engine and if we get push back
from publishers or find users not able to use FTP due to firewall
restrictions, we plan to fully support HTTP moving forward.  We simply
need to break out our directory parsing code and make it work well with
a few of the more mainstream httpd services like Apache and IIS.


4) A single zip file for installation of a module (or modules) is simply
a zip of (1c) above.

Of the 3 'zip' formats we support for download from our server, only the
'rawzip' format should survive.  I believe I can safely say we all agree
on that.  This is in essence still the same thing.  It is a module
repository of exactly 1 module in a difference storage medium.  Heck it
could be 5 installed modules all zipped up and it would still be the
same thing.  As I see it, it is an end user preference left to the
exercise of frontend developers.  "Can my users operate their file
browser and drag and drop to my app easier than they can navigate their
"Browse to folder" dialog box in "Install SWORD Modules from path..."
option in my installer?  If so, then I should support drag and drop on
my frontend-- after all it is simply an unzip.  But the user gets no
benefit of our full installer: not description, to versioning upgrade
information if they already have an instance of that module installed,
to category, no language, etc.

5) If we find we cannot achieve this 'no prep' installation utopia in
the real world and it is necessary to have an intermediate format.
Matthew Talbert had a great suggestion on IRC last night to simply add a
'prep for publishing' (or in his words simply "publish") option to the
InstallMgr interface which would create (1) "module installation source"
data format if indeed it technically must be different.  Great idea if
we need to concede to 2 disparate data formats, but I am still hopeful
we can avoid this extra step for would-be module distributors-- end
users and publishers alike.


Hope this separates out thoughts a little better.


Troy



On 11/05/2010 01:15 PM, Greg Hellings wrote:
> On Fri, Nov 5, 2010 at 8:02 AM, Peter von Kaehne <refdoc at gmx.net> wrote:
>> I think Troy, the concern is correct.
>>
>> For the publisher with some decent IT muscle and budget a proper repo must be better, but for the small town church with a website and a couple of modules to share - zip and http is a must.
>>
>> Having a multiplicity of methods of getting modules into the system would certainly be easier.
>>
>> My preference:
>>
>> 1) keep current methods - it is best for huge numbers of modules and it is probably also best for anyone with enough money to have a fixed ip and a server, able to run anonymous ftp
> 
> While I very strongly agree that FTP should not be held up as the best
> solution for everyone (I have worked with and for several
> organizations who flat out refused to ever allow FTP of any type to be
> a part of their systems and who primarily ran IIS for their web
> content), I'm not sure the argument from excessive cost is accurate.
> 
> Annual DNS leasing from my provider is less than $40/year (I want to
> say it's only about $17, but I might be remembering incorrectly)
> Dedicated VPS hosting with dedicated IP address, about 20 GB of drive
> space and full root access to my system is $19.95/mo.
> 
> If $23 per month is excessive for anyone who really intends to
> distribute modules, they can almost certainly find someone else in the
> SWORD community who will be willing to host it for them at very little
> to no cost.  Additionally I can host any low-volume traffic I want
> from home by registering with a place like dyndns.com and setting up a
> server on any connection that is moderately reliable (my home). The
> option is not viable for people in developing countries but is
> certainly more than viable for any first world country I'm aware of.
> 
> --Greg
> 
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page