[sword-devel] Module upload: BlaMat

Sun Dec 16 23:24:52 MST 2018

Hi all,

Yes, PocketSword does things the unadvised way because initially it took forever to create a search index on a device when PS was first created. I haven’t tested things recently (in the past 5 years?), but I’m guessing any device from the last 3 years should be more than up to the task, and older devices are probably ok.
However, I haven’t had time to investigate, and I’m in the process of moving countries, so it won’t happen any time soon.

PocketSword currently looks for a search index on the CW server. There are scripts that are run when Peter creates a new module that will create the index and upload it to the correct place. I used to manually run scripts to create them for the other official repos about once a month, but it has been a long time since that has happened. I investigated doing that for eBible, but due to the LARGE number of modules (yay!) and the rapid updating of them (also yay), creating an index for each version of each module would have not worked. You need a new index for a new version of a module, obviously, as the text _may_ have changed. (Sometimes a version number gets bumped because of a conf change.)

Currently, the CW server is being migrated. I believe that is causing some issues for some of Peter’s scripts, so hopefully once the migration is completed, everything will be working swimmingly again?

FYI: PS looks at the HTTP (well, HTTPS) mirror of the CW FTP server for all modules & downloads indices from https://www.crosswire.org/pocketsword/indices/v1/ <https://www.crosswire.org/pocketsword/indices/v1/>
If you follow that link, you actually see that BlaMat-1.0.zip AND BlaMat-1.0.1.zip are available there, so it appears the indices are working for that module?

I hope that helps? Gotta run.
Nic. :)

> On 17 Dec 2018, at 9:04 am, Troy A. Griffitts <scribe at crosswire.org> wrote:
> 
> To chime in about PocketSword indexes, CrossWire does not officially support downloading pre-generated search indexes for our modules. SWORD, and thus PocketSword has the ability to generate the search indexes on the client, like every other SWORD frontend. PocketSword asked for disk space because they were pre-generating these for some reason at the start of their frontends and we accommodated. I am not sure who is maintaining PocketSword these days, but rather than asking all of our module repos to pre-generated search indexes for one of our frontends, I would suggest someone just change the PocketSword Download function to instead call the SWORD makeSearchIndex function.
> 
> Troy
> 
> On December 16, 2018 2:53:57 PM MST, "refdoc at gmx.net <mailto:refdoc at gmx.net>" <refdoc at gmx.net <mailto:refdoc at gmx.net>> wrote:
> No one is fobbing you off, David. Not answering on the mailing list is normal if you are not able to.help. 
> 
> There are two separate issues here. For one reason the indices I produce for CrossWire are sometimes visible and sometimes not. I am clueless why, but presume the two servers are the problem. I have not looked into that further due to.travel. dM gave me some hints. 
> 
> EBible is a separate matter. In short , this is Michael's repo and he needs to produce indices or find a way to produce them automatically. 
> 
> Peter
> 
> Sent from my mobile. Please forgive shortness, typos and weird autocorrects.
> 
> 
> -------- Original Message --------
> Subject: Re: [sword-devel] Module upload: BlaMat
> From: David Haslam 
> To: SWORD Developers' Collaboration Forum 
> CC: 
> 
> 
> I keep pushing at this issue and I get fobbed off repeatedly. 
> 
> None of the eBible.org modules have an installable search index for PS. 
> 
> Why is nobody seeking to fix this?
> 
> Where does PS look?
> Which server?
> Which path ?
> What protocol?
> 
> Who maintains such a sync ?
> Has it ever been tested ?
> 
> The requirements are not transparent. 
> We ought to document them in our wiki. 
> 
> If modules on the Xiphos repo do have a search index, why not eBible.org too?
> 
> David
> 
> Sent from ProtonMail Mobile
> 
> 
> On Sun, Dec 16, 2018 at 21:32, refdoc at gmx.net <refdoc at gmx.net <mailto:refdoc at gmx.net>> wrote:
>> 
>> I think the sync script runs only once every so often.. 
>> 
>> Sent from my mobile. Please forgive shortness, typos and weird autocorrects.
>> 
>> 
>> -------- Original Message --------
>> Subject: Re: [sword-devel] Module upload: BlaMat
>> From: David Haslam 
>> To: sword-devel at crosswire.org
>> CC: 
>> 
>> 
>> Installed bla1890eb from eBible.org to PocketSword. 
>> 
>> No search index available!
>> 
>> Is anyone working on providing all these for this repository? What’s the main stumbling block?
>> Where does PS look?
>> 
>> Still no Blackfoot language or equivalent found in CrossWire Main. 
>> 
>> Peter - what’s making it invisible?
>> 
>> David
>> 
>> Sent from ProtonMail Mobile
>> 
>> 
>> On Sun, Dec 16, 2018 at 19:52, Kahunapule Michael Johnson <Kahunapule at eBible.org <mailto:Kahunapule at eBible.org>> wrote:
>>> 
>>> Thank you very much, David, for pointing out that there was a problem with the eBible.org repository. It turns out that the ftp server was failing to follow a symbolic link, probably due to a "security" update. (Yes, following symbolic links CAN be a security risk, but this was a breaking change that I wasn't aware of until just now.) Anyway, we are back in service. The Blackfoot Matthew module is listed under the language autonym (Siksika).
>>> 
>>> On 12/16/18 9:01 AM, Kahunapule Michael Johnson wrote:
>>> > Aloha!
>>> >
>>> > A Blackfoot Matthew module is in the eBible.org repository as bla1890eb, sourced from Jon Bitgood's digitization. (Duplication of Public Domain texts doesn't bother me, by the way, but it might be worth checking before working on a module if duplicated effort might bother you.)
>>> >
>>> > I am trying to figure out what is going on with the eBible.org repository, right now. It looks like it is visible via ftp, http, and https, but refreshes on some front ends come up empty. Investigating...
>>> >
>>> > On 12/16/18 7:28 AM, David Haslam wrote:
>>> >> Thanks Peter. 
>>> >>
>>> >> I was aware of Jon Bitgood having several projects on GitHub. I’ve even commented on some of them during 2017.
>>> >>
>>> >> Here’s my problem:
>>> >> The language Blackfoot does not appear, nor does any unknown language code, looking at the CrossWire Main repo from PocketSword. And that’s after a contents refresh. 
>>> >>
>>> >> NB. Michael Johnson works closely with the Digital Bible Society. He knows Jon Bitgood. Are we potentially duplicating content that would ordinarily be hosted on the eBible.org rep?
>>> >>
>>> >> Btw. It looks right now as though the eBible.org repo has become unavailable. Is this the case for all users? Hopefully, this is merely a temporary outage. 
>>> >>
>>> >> David
>>> >>
>>> >> Sent from ProtonMail Mobile
>>> >>
>>> >>
>>> >> On Sun, Dec 16, 2018 at 16:19, Peter von Kaehne <refdoc at gmx.net <mailto:refdoc at gmx.net>> wrote:
>>> >>> On Sun, 2018-12-16 at 14:28 +0000, David Haslam wrote:
>>> >>>> Please tell us additionally in these announcements
>>> >>>>
>>> >>>> Module type
>>> >>>> Language
>>> >>> Added to work list
>>> >>>
>>> >>>> I’ve not yet been able to find BlaMat.
>>> >>> It is Jon Bitgood's effort to digitise the Blackfoot Gospel of Mark.
>>> >>> Blackfeet are one of the First Nations of Northern America.
>>> >>>
>>> >>> I found his work on GitHub and started taking it on. There are some
>>> >>> problems with it still and i have opened issues but I think it is
>>> >>> worthwhile to start publishing.
>>> >>>
>>> >>> I am far from sure how many Blackfoot speakers still exist but how ever
>>> >>> few, I do think we owe them. Them and all other decimated First
>>> >>> Nations.
>>> >>>
>>> >>> Peter
>>> >>>
>>> >>>
>>> >>> _______________________________________________
>>> >>> sword-devel mailing list: sword-devel at crosswire.org
>>> >>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>> >>> Instructions to unsubscribe/change your settings at above page
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> sword-devel mailing list: sword-devel at crosswire.org
>>> >> http://www.crosswire.org/mailman/listinfo/sword-devel
>>> >> Instructions to unsubscribe/change your settings at above page
>>> >
>>> 
>>> --
>>> signature
>>> 
>>> Aloha,
>>> */Michael Johnson/**
>>> PO BOX 881143 • PUKALANI HI 96788-1143*• USA
>>> mljohnson.org <http://mljohnson.org> • Phone: +1 808-333-6921 • Skype: kahunapule
>>> 
>>> 
>>> _______________________________________________
>>> sword-devel mailing list: sword-devel at crosswire.org
>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>> Instructions to unsubscribe/change your settings at above page
>> 
>> 
> 
> 
> 
> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity._______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org <mailto:sword-devel at crosswire.org>
> http://www.crosswire.org/mailman/listinfo/sword-devel <http://www.crosswire.org/mailman/listinfo/sword-devel>
> Instructions to unsubscribe/change your settings at above page

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20181217/4ead3c39/attachment-0001.html>