[sword-devel] sword module repository

[Troy A. Griffitts]

Just a few of the usual, cold, chiding remarks scattered below...

> Yeah, we use a "custom" HTTP method rather than a "custom" FTP method.

Actually the 'standard' SWORD mechanism for remote module installation 
is defined by the C++ SWORD engine.  If the purpose of JSword is to be a 
Java alternative to the C++ SWORD engine then you are failing in this 
regard.  The term "custom" was intended to convey, not compatible with 
SWORD.  Your application of the term "custom" to SWORD actually has no 
meaning in any sense, as the C++ engine uses "standard" FTP protocols, 
and SETS the standard for SWORD protocols.

Besides, in all actuality and seriousness, I have no intention of 
keeping the current web layout and http URLs as they are.  They were not 
thought out to be such.  I have no intention of asking bible.org, 
sil.org, or any other remote module provider to force generation of .zip 
files, and URL layouts similar to our current crosswire layout.  We have 
custom (Java, mind you) code on our server which automatically builds 
those zip files when any of the module files are touched.  It works out 
quite nicely for us, but I wouldn't want to force another organization 
to run tomcat or create these by hand or some other way.

It is very nice to say:

All it takes is to make FTP access available to a standard installed 
SWORD module set.

We actually support file:// access as well.  This is how SWORD makes CD 
installation possible for all frontends which use the engine.

On another note, I would not be amiss to ADDING support for installing 
from a standard SWORD module set over HTTP.

And of course, your other arguments are just silly, as usual.  You 
cannot conclude that just because you did not achieve your goal of 
matching the SWORD C++ methodology, that the SWORD C++ methodology is bad.

We've not had any of the same complaints that you mention... at least 
not for longer than I can remember specifically (not even for any XP SP2 
firewall issue you mention).  I thought Java was supposed to be 
cross-platform ;)  But I guess I can sympathize with Sun if Microsoft 
decided to screw up Java-only programs in regard to FTP access.

On another note, I do know we have an outstanding bug in our installer 
in 1.5.9, which I wish I, or someone else, had time to fix, dealing with 
module upgrades.  I think this broke 1.5.9, as I have done upgrades 
previously with no problems.

As usual.  I mean most of this, but it's fun being extra brutal with 
you, DM.  It's how I show my love...

	;)

		-Troy.





> JSword uses a plug-in model for an installer and as such can accept  
> plugins for different sites and different methodologies for  
> downloading. We started with a plugin for FTP that worked just like  
> the SWORD C++ API. However, we kept getting problem reports that it  
> would not work at all in some situations in others it left an  
> incomplete download.
> 
> The number of reports increased significantly around the time that MS  
> put out SP2 for WinXP. It turned out that MS had tightened security  
> and was blocking ftp access in the manner that JSword use Apache  
> Commons Net package for FTP.
> 
> We were also getting reports that it would not work through a  
> corporate firewall.
> 
> And that it would not work on a Mac.
> 
> The problems of an incomplete download was that if the module  
> consisted of 6 files as a compressed Bible has then it would download  
> 6 separate files. If the FTP failed during one or more of these, then  
> the download left the module in an inconsistent state. Explaining to  
> the users how to cleanup so that it could re-execute was a problem.  
> So we were left with figuring out whether the download was complete  
> or not and cleaning up afterwards.
> 
> One of the JSword devel subscribers suggested doing http tunneling to  
> get to the files. We tried this using the same set of files to  
> download, but the failed download was still a problem. And it was  
> difficult to download a directory of files or get a listing of files.
> 
> So we decided to go after the zip file for the module. This solved  
> the incomplete download problem, significantly improved download  
> performance and greatly minimized failures in the first place.
> 
> We were able to further change JSword so that it can do downloads in  
> separate threads, something that was not really practical with FTP.
> 
> We also added support for proxies.
> 
> While SWORD does not need the mods.d.tar.gz file and can get the list  
> of modules directly from the mods.d directory on the server, this is  
> not possible with the JSword HTTP tunneling method.
> 
> Since this was released the only complaint that we have received has  
> been from Troy that it does not do it the same. *wink* :)
> 
> When we first released this mechanism we made it a user choice, with  
> FTP being the default. That did not reduce the rate of reported  
> problems. At the next release we made HTTP the default and problem  
> reports stopped. Then we hid the FTP choice from the user in the  
> following release and then in the release after that we retired the  
> code to jsword-limbo.
> 
> Even if we were to restore the FTP methodology and get it to work  
> from all installations of JSword, we would apply all that we learned  
> and go after the zips.
> 
> I'd like to suggest that SWORD C++ api be changed to download the  
> module's zip file if present.
> 
> -- DM
> 
> 
> 
> On Nov 6, 2006, at 5:26 PM, Troy A. Griffitts wrote:
> 
>> Hey Peter,
>> 	Officially, a SWORD module repository merely requires FTP access to a
>> standard install of SWORD modules.
>>
>> ftp://crosswire.org/pub/sword/raw
>>
>> 	This should allow all frontends which use the SWORD engine's install
>> classes to use your repository (BibleCS, GnomeSword, Bibletime, maybe
>> others).
>>
>> 	This WILL NOT allow JSword to use your repository (AFAIK), and I  
>> don't
>> believe there is a GUI installer for MacSword yet (however the
>> commandline utilities/installmgr application should work fine on  
>> the Mac
>> and should work against your repository, but it's not likely to get a
>> Mac user very interested).  JSword currently use their own custom HTML
>> methods for doing module installation.  You can chide them publicly
>> (please) for that.
>>
>> 	Also, if you would like to speed up the download experience for your
>> users, you can create a cache of the mods.d directory with a simple:
>>
>> tar czfv mods.d.tar.gz mods.d
>>
>> 	You will see this at our FTP link posted above.  It is not necessary,
>> but if the installer doesn't find this, it will download each .conf
>> file, one at a time.  If you only have a few modules available, it
>> shouldn't make too much difference.
>>
>> 	Hope this helps,
>>
>> 		-Troy.
>>
>>
>>
>> Peter von Kaehne wrote:
>>> I have created a collection of sword modules for which i have only  
>>> limited rights of distribution - i.e. our church can distribute  
>>> them without any complaint, but I think I would end up in trouble  
>>> if I uploaded them into the sword repositories. I do presume this  
>>> will be resolved at some stage, but I am at the same time eager to  
>>> go ahead and put a repository onto our church website until things  
>>> are sorted.
>>>
>>> What exactly is needed in a repository to make it work with the  
>>> module manager in BibleCS or in JSword's Bibledesktop
>>>
>>> How do I need to package modules? Zip or self extracting exe-files?
>>>
>>> Thanks
>>>
>>> Peter
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>> _______________________________________________
>> sword-devel mailing list: sword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page
> 
> 
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page