[sword-devel] SWORD trunk / Regression when updating remote sources

[Tobias Klein]

I'll see whether I can collect a stack trace. It may take some time 
until I have it.

The multi-threaded "remote source refreshing" worked without issues 
until recently.

Here is the code of the function that does the actual work in a thread.
See 
https://github.com/tobias-klein/node-sword-interface/blob/787160ccb4b3bab2a762d22f74031c7237edc803/src/sword_backend/repository_interface.cpp#L105.

intRepositoryInterface::refreshIndividualRemoteSource(stringremoteSourceName, 
std::function<void(unsignedintprogress)>*progressCallback)
{
//cout << "Refreshing source " << remoteSourceName << endl << flush;
InstallSource* source= this->getRemoteSource(remoteSourceName);
intresult= this->_installMgr->refreshRemoteSource(source);
if(result!= 0) {
cerr<<"Failed to refresh source "<<remoteSourceName<<endl<<flush;
}
remoteSourceUpdateMutex.lock();
this->_remoteSourceUpdateCount++;
unsignedinttotalPercent= 
(unsignedint)calculateIntPercentage<double>(this->_remoteSourceUpdateCount,
this->_remoteSourceCount);
if(progressCallback!= 0) {
(*progressCallback)(totalPercent);
}
remoteSourceUpdateMutex.unlock();
returnresult;
}

Best regards,
Tobias

On 10/12/20 9:01 PM, Troy A. Griffitts wrote:
> Any luck getting a stack trace on crash?
>
> Regarding the "multitheaded mode", I'd have to get a bit more 
> information as to exactly how you are sharing SWORD objects across 
> your threads. Generally, as a rule, you shouldn't. We recommend a 
> separate instance of SWMgr per thread and that probably goes for 
> InstallMgr, as well.
>
> Troy
>
> On October 12, 2020 8:29:31 PM GMT+02:00, Tobias Klein 
> <contact at tklein.info> wrote:
>
>     Hi Troy,
>
>     I'm using curl on all three platforms.
>
>     Regarding the timeout configuration I have not changed anything
>     yet, to make this configurable in Ezra Project is still on my todo
>     list.
>
>     I just checked on Linux.
>     With the old version (May 18th 2020) no hanging or crash in 10 out
>     of 10 times.
>     WIth the new version (latest trunk / SWORD 1.9 RC3) I get 1 x
>     crash, 2 x hanging, 7 x working.
>
>     I'm running the InstallMgr::refreshRemoteSource "in a
>     multi-threaded mode".
>
>     Best regards,
>     Tobias
>
>     On 10/12/20 6:59 PM, Troy A. Griffitts wrote:
>>     Hi Tobias,
>>
>>     What transport library are you building with? ftplib or curl?
>>
>>     Have you changed the value of our new timeout from the default, I
>>     believe we decided on, 10 seconds?
>>
>>     Troy
>>
>>     On October 12, 2020 6:46:54 PM GMT+02:00, Tobias Klein
>>     <contact at tklein.info> wrote:
>>
>>         Hi Troy,
>>
>>         In my latest Ezra Project builds using SWORD trunk I’ve been noticing random „hangs“ and crashes related to "updating remote sources“. I suppose it must be around InstallMgr::refreshRemoteSource.
>>
>>         This was still rock solid when using SWORD trunk from May 18th 2020, but not so any more with the recent SWORD trunk.
>>
>>         Unfortunately I cannot pinpoint this more specifically. I just wanted to first share this observation, because it’s worrying me.
>>
>>         I’ve been noticing this regression both on Windows and macOS. Need to check later whether this also happens on Linux, cannot recall it right now.
>>
>>         Best regards,
>>         Tobias
>>         ------------------------------------------------------------------------
>>         sword-devel mailing list:sword-devel at crosswire.org
>>         http://crosswire.org/mailman/listinfo/sword-devel
>>         Instructions to unsubscribe/change your settings at above page
>>
>>
>>     -- 
>>     Sent from my Android device with K-9 Mail. Please excuse my brevity. 
>
>
> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://crosswire.org/pipermail/sword-devel/attachments/20201012/b6c63cef/attachment-0001.html>