[sword-devel] Missing ZIP files
Martin Denham
mjdenham at gmail.com
Sat Oct 19 14:05:37 MST 2013
Thanks DM,
I didn't realise. For now I have just forced creation of the zip for OSHB.
Cheers
Martin
On 19 October 2013 16:02, DM Smith <dmsmith at crosswire.org> wrote:
> I've not gotten it into place. I think it needs to be reviewed first. I'll
> run it to make them current.
> -- DM
>
> On Oct 19, 2013, at 10:40 AM, Martin Denham <mjdenham at gmail.com> wrote:
>
> Hi Stephan,
>
> That is the way it used to work but a few months ago DM created a script
> to automatically build the AV zip files because if a zip file doesn't
> exist then JSword apps throw an error when a user tries to download the
> module.
>
> Cheers
> Martin
>
>
> On 19 October 2013 12:43, Stephan <info at tetzels.de> wrote:
>
>> The zip files are only created if you use the ModDisp.jsp script/applet. (
>> http://www.crosswire.org/**sword/modules/ModDisp.jsp?**
>> modType=Bibles&av=true<http://www.crosswire.org/sword/modules/ModDisp.jsp?modType=Bibles&av=true>
>> )
>>
>> Best Regards,
>> Stephan
>>
>> Am 19.10.2013 15:58, schrieb Martin Denham:
>>
>>> Hi,
>>>
>>> Could somebody check the script that automatically creates the AV .zip
>>> files is still running?
>>>
>>> I had problems downloading RusSynodalLIO and OSHB via JSword and found
>>> the zips were missing. I have just forced the creation of RusSynodalLIO
>>> by selecting the jar from ModDisp.jsp and that now exists. I have not
>>> forced creation of OSHB yet in case somebody wanted to verify the
>>> OSHB.zip file is not there first.
>>>
>>> Here is where I am looking:
>>> http://www.crosswire.org/**ftpmirror/pub/sword/**avpackages/rawzip/<http://www.crosswire.org/ftpmirror/pub/sword/avpackages/rawzip/>
>>> The RusSynodalLIO zip now exists because I just triggered creation:
>>> RusSynodalLIO.zip 19-Oct-2013 01:513.0M
>>>
>>> but there is no OSHB.zip
>>>
>>> Thanks
>>> Martin
>>>
>>>
>>>
>>>
>>> On 30 July 2013 22:43, DM Smith <dmsmith at crosswire.org
>>> <mailto:dmsmith at crosswire.org>**> wrote:
>>>
>>> Thanks for everyone's input.
>>>
>>> Attached is a shell script which is modelled after
>>> http://crosswire.org/svn/**crosswire-java/trunk/src/org/**
>>> crosswire/sword/web/**PackageMod.java<http://crosswire.org/svn/crosswire-java/trunk/src/org/crosswire/sword/web/PackageMod.java>
>>> .
>>> PackageMod.java does the following:
>>> Zipping is only done if at least one of the module's files or conf
>>> has a more recent modification time than the zip.
>>> The name of the zip is taken from the [NAME] in the conf.
>>> The zip overwrites any existing zip.
>>>
>>> My script differs slightly:
>>> It zips to a temporary file in the same directory and then copies it
>>> over top of the existing zip.
>>> It manages the mods.d.tar.gz file when a zip is created, again by
>>> creating a temporary and moving that over top of the existing.
>>> It assumes that there are no spaces in any path and that the
>>> repository is on UNIX.
>>>
>>> I've tried this on all the repositories I could find.
>>>
>>> I did find that there were a few conf files that did not have module
>>> files. Should these confs be renamed, deleted, or otherwise moved
>>> out of the way.
>>>
>>> I did not update/manage InstallSize, which could be easily added.
>>>
>>> Please review and comment.
>>>
>>> BTW, it is very verbose.
>>>
>>> In Him,
>>> DM
>>>
>>>
>>>
>>> On Jul 21, 2013, at 5:00 AM, "Troy A. Griffitts"
>>> <scribe at crosswire.org <mailto:scribe at crosswire.org>> wrote:
>>>
>>> A few quick comments.
>>>>
>>>> If you're using http to grab the zips, we already have an http URL
>>>> which will be sure to give you the latest zip-- the packager.
>>>>
>>>> If I implement the cache mechanism in the engine, I will use the
>>>> same logic as what is in the packager to determine if any found
>>>> zip is current.
>>>>
>>>> The code for the packager is in the crosswire-java svn repo. You
>>>> should be able to find the servlet with a grep. Old sucky code I'm
>>>> not proud of but works.
>>>>
>>>> Chris Little <chrislit at crosswire.org
>>>> <mailto:chrislit at crosswire.org**>> wrote:
>>>>
>>>> On 07/20/2013 03:05 PM, DM Smith wrote:
>>>>
>>>> On Jul 20, 2013, at 5:42 PM, Chris Little
>>>> <chrislit at crosswire.org <mailto:chrislit at crosswire.org**>>
>>>>
>>>> wrote:
>>>>
>>>> On 07/20/2013 07:14 AM, Martin Denham wrote:
>>>>
>>>> They should be there again now. And Bible depends
>>>> on the av repository.
>>>>
>>>> See this thread for more info:
>>>> http://svn.crosswire.org/**
>>>> pipermail/jsword-devel/2013-**April/004639.html<http://svn.crosswire.org/pipermail/jsword-devel/2013-April/004639.html>
>>>>
>>>> It is a real shame the av zips are not created
>>>> automatically.
>>>>
>>>>
>>>>
>>>> Or...
>>>> It's a real shame that JSword doesn't use the
>>>> repositories correctly, according to their intended
>>>> usage.
>>>>
>>>>
>>>> Yes it is a shame. I'm almost done w the code to use the
>>>> expanded repository via ftp. This will be with the release
>>>> after next.
>>>>
>>>>
>>>> Awesome.
>>>>
>>>> Or...
>>>> It's a real shame you don't write a cron job that can
>>>> update the ZIPs nightly, if necessary. (Maybe Karl can
>>>> help you, if he has a script to update ZIPs in his
>>>> repository.)
>>>>
>>>>
>>>> Can anyone point me to the code that does the zipping?
>>>> While coding it up is trivial, there's no point to
>>>> re-inventing the wheel. Just port the code to a loop.
>>>>
>>>> The latter is probably more useful at this point and
>>>> more urgent since I am going to write a cron job to
>>>> delete the ZIPs on occasion.
>>>>
>>>>
>>>> Please don't do this until we have had a chance to write
>>>> the script.
>>>>
>>>>
>>>>
>>>> Yeah, it's not urgent. It can wait a while. And I only intend
>>>> to have it
>>>> run once a week or so.
>>>>
>>>> What would the purpose of doing the mass deletes?
>>>> Currently JSword uses the file time to determine whether
>>>> the module has changed. Since our frontends do not
>>>> advertise updated modules, this is no big deal. We've got
>>>> the mechanism written to use the conf version field, but
>>>> it is not in the frontends yet.
>>>>
>>>>
>>>>
>>>> There are a couple of reasons to clear out the cache on
>>>> occasion. The
>>>> web download applet will generate ZIPs for updates and new
>>>> modules, but
>>>> don't remove pulled modules (and sometimes I forget to do it
>>>> manually).
>>>>
>>>> We also get people who think they're really smart and start
>>>> feeding the
>>>> download applet things like NIV and NKJV. And the applet will
>>>> happily
>>>> package those for them (except that they are empty of content).
>>>> So it's
>>>>
>>>> nice to clear out those sorts of files from the server.
>>>>
>>>> If the purpose is to ensure that old stuff is
>>>> appropriately deleted, then a script comparing module conf
>>>> to zips deleting those that don't match would be better.
>>>> This is even simpler to write than the zipper program.
>>>> This could be added to the zipper program.
>>>>
>>>>
>>>>
>>>> Sure, there are other ways of handling the issues.
>>>>
>>>> BTW, Troy's position has been that it is OK to go to the
>>>> zip cache first and failing that get the module by parts.
>>>> But that there should be no requirement of a frontend to
>>>> have the zips. Many have noted that getting the zip is
>>>> faster and more reliable than getting by parts.
>>>>
>>>>
>>>>
>>>> At present, I believe that is a bad position. Shortly after
>>>> updates,
>>>> ZIPs and the module parts can get out of sync so that, e.g.,
>>>> you think
>>>> you're downloading version 2.0 of a module, but you're actually
>>>>
>>>> downloading 1.0 since that's what the ZIP is because no one has
>>>> downloaded through the applet, causing the ZIP to update. And
>>>> no amount
>>>> to trying to update from the ZIP is going to update to 2.0
>>>> until someone
>>>>
>>>> else hits the download applet.
>>>>
>>>> A cron job running an update script would at least mean that
>>>> both the
>>>> mods.d.tar.gz and ZIPs get updated at approximately the same
>>>> time.
>>>>
>>>> --Chris
>>>>
>>>>
>>>>
>>>> ------------------------------**------------------------------*
>>>> *------------
>>>>
>>>> sword-devel mailing list:sword-devel at crosswire.org <mailto:
>>>> sword-devel at crosswire.**org <sword-devel at crosswire.org>>
>>>>
>>>>
>>>> http://www.crosswire.org/**mailman/listinfo/sword-devel<http://www.crosswire.org/mailman/listinfo/sword-devel>
>>>> Instructions to unsubscribe/change your settings at above page
>>>>
>>>>
>>>> --
>>>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>>>> ______________________________**_________________
>>>> sword-devel mailing list: sword-devel at crosswire.org
>>>> <mailto:sword-devel at crosswire.**org <sword-devel at crosswire.org>>
>>>>
>>>> http://www.crosswire.org/**mailman/listinfo/sword-devel<http://www.crosswire.org/mailman/listinfo/sword-devel>
>>>> Instructions to unsubscribe/change your settings at above page
>>>>
>>>
>>>
>>> ______________________________**_________________
>>> sword-devel mailing list: sword-devel at crosswire.org
>>> <mailto:sword-devel at crosswire.**org <sword-devel at crosswire.org>>
>>>
>>> http://www.crosswire.org/**mailman/listinfo/sword-devel<http://www.crosswire.org/mailman/listinfo/sword-devel>
>>> Instructions to unsubscribe/change your settings at above page
>>>
>>>
>>>
>>>
>>> ______________________________**_________________
>>> sword-devel mailing list: sword-devel at crosswire.org
>>> http://www.crosswire.org/**mailman/listinfo/sword-devel<http://www.crosswire.org/mailman/listinfo/sword-devel>
>>> Instructions to unsubscribe/change your settings at above page
>>>
>>>
>> ______________________________**_________________
>> sword-devel mailing list: sword-devel at crosswire.org
>> http://www.crosswire.org/**mailman/listinfo/sword-devel<http://www.crosswire.org/mailman/listinfo/sword-devel>
>> Instructions to unsubscribe/change your settings at above page
>>
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
>
>
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20131019/aac07d92/attachment-0001.html>
More information about the sword-devel
mailing list