[sword-devel] sword module repository

[DM Smith]

Hi all,
	Time for me to chime in. Since Troy threw down the gauntlet. (BTW,  
we are good friends and I know he is teasing me for the differences)  
Yeah, we use a "custom" HTTP method rather than a "custom" FTP method.

JSword uses a plug-in model for an installer and as such can accept  
plugins for different sites and different methodologies for  
downloading. We started with a plugin for FTP that worked just like  
the SWORD C++ API. However, we kept getting problem reports that it  
would not work at all in some situations in others it left an  
incomplete download.

The number of reports increased significantly around the time that MS  
put out SP2 for WinXP. It turned out that MS had tightened security  
and was blocking ftp access in the manner that JSword use Apache  
Commons Net package for FTP.

We were also getting reports that it would not work through a  
corporate firewall.

And that it would not work on a Mac.

The problems of an incomplete download was that if the module  
consisted of 6 files as a compressed Bible has then it would download  
6 separate files. If the FTP failed during one or more of these, then  
the download left the module in an inconsistent state. Explaining to  
the users how to cleanup so that it could re-execute was a problem.  
So we were left with figuring out whether the download was complete  
or not and cleaning up afterwards.

One of the JSword devel subscribers suggested doing http tunneling to  
get to the files. We tried this using the same set of files to  
download, but the failed download was still a problem. And it was  
difficult to download a directory of files or get a listing of files.

So we decided to go after the zip file for the module. This solved  
the incomplete download problem, significantly improved download  
performance and greatly minimized failures in the first place.

We were able to further change JSword so that it can do downloads in  
separate threads, something that was not really practical with FTP.

We also added support for proxies.

While SWORD does not need the mods.d.tar.gz file and can get the list  
of modules directly from the mods.d directory on the server, this is  
not possible with the JSword HTTP tunneling method.

Since this was released the only complaint that we have received has  
been from Troy that it does not do it the same. *wink* :)

When we first released this mechanism we made it a user choice, with  
FTP being the default. That did not reduce the rate of reported  
problems. At the next release we made HTTP the default and problem  
reports stopped. Then we hid the FTP choice from the user in the  
following release and then in the release after that we retired the  
code to jsword-limbo.

Even if we were to restore the FTP methodology and get it to work  
from all installations of JSword, we would apply all that we learned  
and go after the zips.

I'd like to suggest that SWORD C++ api be changed to download the  
module's zip file if present.

-- DM



On Nov 6, 2006, at 5:26 PM, Troy A. Griffitts wrote:

> Hey Peter,
> 	Officially, a SWORD module repository merely requires FTP access to a
> standard install of SWORD modules.
>
> ftp://crosswire.org/pub/sword/raw
>
> 	This should allow all frontends which use the SWORD engine's install
> classes to use your repository (BibleCS, GnomeSword, Bibletime, maybe
> others).
>
> 	This WILL NOT allow JSword to use your repository (AFAIK), and I  
> don't
> believe there is a GUI installer for MacSword yet (however the
> commandline utilities/installmgr application should work fine on  
> the Mac
> and should work against your repository, but it's not likely to get a
> Mac user very interested).  JSword currently use their own custom HTML
> methods for doing module installation.  You can chide them publicly
> (please) for that.
>
> 	Also, if you would like to speed up the download experience for your
> users, you can create a cache of the mods.d directory with a simple:
>
> tar czfv mods.d.tar.gz mods.d
>
> 	You will see this at our FTP link posted above.  It is not necessary,
> but if the installer doesn't find this, it will download each .conf
> file, one at a time.  If you only have a few modules available, it
> shouldn't make too much difference.
>
> 	Hope this helps,
>
> 		-Troy.
>
>
>
> Peter von Kaehne wrote:
>> I have created a collection of sword modules for which i have only  
>> limited rights of distribution - i.e. our church can distribute  
>> them without any complaint, but I think I would end up in trouble  
>> if I uploaded them into the sword repositories. I do presume this  
>> will be resolved at some stage, but I am at the same time eager to  
>> go ahead and put a repository onto our church website until things  
>> are sorted.
>>
>> What exactly is needed in a repository to make it work with the  
>> module manager in BibleCS or in JSword's Bibledesktop
>>
>> How do I need to package modules? Zip or self extracting exe-files?
>>
>> Thanks
>>
>> Peter
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page