<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>I'll see whether I can collect a stack trace. It may take some
time until I have it.</p>
<p>The multi-threaded "remote source refreshing" worked without
issues until recently.<br>
</p>
<p>Here is the code of the function that does the actual work in a
thread.<br>
See
<a class="moz-txt-link-freetext" href="https://github.com/tobias-klein/node-sword-interface/blob/787160ccb4b3bab2a762d22f74031c7237edc803/src/sword_backend/repository_interface.cpp#L105">https://github.com/tobias-klein/node-sword-interface/blob/787160ccb4b3bab2a762d22f74031c7237edc803/src/sword_backend/repository_interface.cpp#L105</a>.</p>
<div style="color: #d4d4d4;background-color: #1e1e1e;font-family: 'Droid Sans Mono', 'monospace', monospace, 'Droid Sans Fallback';font-weight: normal;font-size: 14px;line-height: 19px;white-space: pre;"><div><span style="color: #569cd6;">int</span><span style="color: #d4d4d4;"> </span><span style="color: #4ec9b0;">RepositoryInterface</span><span style="color: #d4d4d4;">::</span><span style="color: #dcdcaa;">refreshIndividualRemoteSource</span><span style="color: #d4d4d4;">(</span><span style="color: #4ec9b0;">string</span><span style="color: #d4d4d4;"> </span><span style="color: #9cdcfe;">remoteSourceName</span><span style="color: #d4d4d4;">, </span><span style="color: #4ec9b0;">std</span><span style="color: #d4d4d4;">::</span><span style="color: #4ec9b0;">function</span><span style="color: #d4d4d4;"><</span><span style="color: #569cd6;">void</span><span style="color: #d4d4d4;">(</span><span style="color: #569cd6;">unsigned</span><span style="color: #d4d4d4;"> </span><span style="color: #569cd6;">int</span><span style="color: #d4d4d4;"> </span><span style="color: #4ec9b0;">progress</span><span style="color: #d4d4d4;">)></span><span style="color: #569cd6;">*</span><span style="color: #d4d4d4;"> </span><span style="color: #9cdcfe;">progressCallback</span><span style="color: #d4d4d4;">)</span></div><div><span style="color: #d4d4d4;">{</span></div><div><span style="color: #6a9955;"> //cout << "Refreshing source " << remoteSourceName << endl << flush;</span></div><div><span style="color: #d4d4d4;"> </span><span style="color: #4ec9b0;">InstallSource</span><span style="color: #d4d4d4;">* </span><span style="color: #9cdcfe;">source</span><span style="color: #d4d4d4;"> = </span><span style="color: #569cd6;">this</span><span style="color: #d4d4d4;">-></span><span style="color: #dcdcaa;">getRemoteSource</span><span style="color: #d4d4d4;">(</span><span style="color: #9cdcfe;">remoteSourceName</span><span style="color: #d4d4d4;">);</span></div><div><span style="color: #d4d4d4;"> </span><span style="color: #569cd6;">int</span><span style="color: #d4d4d4;"> </span><span style="color: #9cdcfe;">result</span><span style="color: #d4d4d4;"> = </span><span style="color: #569cd6;">this</span><span style="color: #d4d4d4;">-></span><span style="color: #9cdcfe;">_installMgr</span><span style="color: #d4d4d4;">-></span><span style="color: #dcdcaa;">refreshRemoteSource</span><span style="color: #d4d4d4;">(</span><span style="color: #9cdcfe;">source</span><span style="color: #d4d4d4;">);</span></div><div><span style="color: #d4d4d4;"> </span><span style="color: #c586c0;">if</span><span style="color: #d4d4d4;"> (</span><span style="color: #9cdcfe;">result</span><span style="color: #d4d4d4;"> != </span><span style="color: #b5cea8;">0</span><span style="color: #d4d4d4;">) {</span></div><div><span style="color: #d4d4d4;"> </span><span style="color: #9cdcfe;">cerr</span><span style="color: #d4d4d4;"> </span><span style="color: #dcdcaa;"><<</span><span style="color: #d4d4d4;"> </span><span style="color: #ce9178;">"Failed to refresh source "</span><span style="color: #d4d4d4;"> </span><span style="color: #dcdcaa;"><<</span><span style="color: #d4d4d4;"> </span><span style="color: #9cdcfe;">remoteSourceName</span><span style="color: #d4d4d4;"> </span><span style="color: #dcdcaa;"><<</span><span style="color: #d4d4d4;"> </span><span style="color: #dcdcaa;">endl</span><span style="color: #d4d4d4;"> </span><span style="color: #dcdcaa;"><<</span><span style="color: #d4d4d4;"> </span><span style="color: #dcdcaa;">flush</span><span style="color: #d4d4d4;">;</span></div><div><span style="color: #d4d4d4;"> }</span></div>
<div><span style="color: #d4d4d4;"> </span><span style="color: #9cdcfe;">remoteSourceUpdateMutex</span><span style="color: #d4d4d4;">.</span><span style="color: #dcdcaa;">lock</span><span style="color: #d4d4d4;">();</span></div><div><span style="color: #d4d4d4;"> </span><span style="color: #569cd6;">this</span><span style="color: #d4d4d4;">-></span><span style="color: #9cdcfe;">_remoteSourceUpdateCount</span><span style="color: #d4d4d4;">++;</span></div><div><span style="color: #d4d4d4;"> </span><span style="color: #569cd6;">unsigned</span><span style="color: #d4d4d4;"> </span><span style="color: #569cd6;">int</span><span style="color: #d4d4d4;"> </span><span style="color: #9cdcfe;">totalPercent</span><span style="color: #d4d4d4;"> = (</span><span style="color: #569cd6;">unsigned</span><span style="color: #d4d4d4;"> </span><span style="color: #569cd6;">int</span><span style="color: #d4d4d4;">)</span><span style="color: #dcdcaa;">calculateIntPercentage</span><span style="color: #d4d4d4;"><</span><span style="color: #569cd6;">double</span><span style="color: #d4d4d4;">>(</span><span style="color: #569cd6;">this</span><span style="color: #d4d4d4;">-></span><span style="color: #9cdcfe;">_remoteSourceUpdateCount</span><span style="color: #d4d4d4;">,</span></div><div><span style="color: #d4d4d4;"> </span><span style="color: #569cd6;">this</span><span style="color: #d4d4d4;">-></span><span style="color: #9cdcfe;">_remoteSourceCount</span><span style="color: #d4d4d4;">);</span></div><div><span style="color: #d4d4d4;"> </span></div><div><span style="color: #d4d4d4;"> </span><span style="color: #c586c0;">if</span><span style="color: #d4d4d4;"> (</span><span style="color: #9cdcfe;">progressCallback</span><span style="color: #d4d4d4;"> != </span><span style="color: #b5cea8;">0</span><span style="color: #d4d4d4;">) {</span></div><div><span style="color: #d4d4d4;"> (*</span><span style="color: #9cdcfe;">progressCallback</span><span style="color: #d4d4d4;">)</span><span style="color: #dcdcaa;">(</span><span style="color: #9cdcfe;">totalPercent</span><span style="color: #dcdcaa;">)</span><span style="color: #d4d4d4;">;</span></div><div><span style="color: #d4d4d4;"> }</span></div><div><span style="color: #d4d4d4;"> </span><span style="color: #9cdcfe;">remoteSourceUpdateMutex</span><span style="color: #d4d4d4;">.</span><span style="color: #dcdcaa;">unlock</span><span style="color: #d4d4d4;">();</span></div>
<div><span style="color: #d4d4d4;"> </span><span style="color: #c586c0;">return</span><span style="color: #d4d4d4;"> </span><span style="color: #9cdcfe;">result</span><span style="color: #d4d4d4;">;</span></div><div><span style="color: #d4d4d4;">}</span></div></div>
<p>Best regards,<br>
Tobias<br>
</p>
<div class="moz-cite-prefix">On 10/12/20 9:01 PM, Troy A. Griffitts
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:36F83DB3-F6C4-4D15-9146-40A6C5B762A0@crosswire.org">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
Any luck getting a stack trace on crash?<br>
<br>
Regarding the "multitheaded mode", I'd have to get a bit more
information as to exactly how you are sharing SWORD objects across
your threads. Generally, as a rule, you shouldn't. We recommend a
separate instance of SWMgr per thread and that probably goes for
InstallMgr, as well.<br>
<br>
Troy<br>
<br>
<div class="gmail_quote">On October 12, 2020 8:29:31 PM GMT+02:00,
Tobias Klein <a class="moz-txt-link-rfc2396E" href="mailto:contact@tklein.info"><contact@tklein.info></a> wrote:
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;">
<p>Hi Troy,</p>
<p>I'm using curl on all three platforms.</p>
<p>Regarding the timeout configuration I have not changed
anything yet, to make this configurable in Ezra Project is
still on my todo list.</p>
<p>I just checked on Linux.<br>
With the old version (May 18th 2020) no hanging or crash in
10 out of 10 times.<br>
WIth the new version (latest trunk / SWORD 1.9 RC3) I get 1
x crash, 2 x hanging, 7 x working.</p>
<p>I'm running the InstallMgr::refreshRemoteSource "in a
multi-threaded mode".<br>
</p>
<p>Best regards,<br>
Tobias<br>
</p>
<div class="moz-cite-prefix">On 10/12/20 6:59 PM, Troy A.
Griffitts wrote:<br>
</div>
<blockquote type="cite"
cite="mid:E1689B41-6534-4EE4-9267-22BE9F59941E@crosswire.org">
<meta http-equiv="content-type" content="text/html;
charset=UTF-8">
Hi Tobias,<br>
<br>
What transport library are you building with? ftplib or
curl?<br>
<br>
Have you changed the value of our new timeout from the
default, I believe we decided on, 10 seconds?<br>
<br>
Troy<br>
<br>
<div class="gmail_quote">On October 12, 2020 6:46:54 PM
GMT+02:00, Tobias Klein <a class="moz-txt-link-rfc2396E"
href="mailto:contact@tklein.info" moz-do-not-send="true"><contact@tklein.info></a>
wrote:
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;">
<pre class="k9mail">Hi Troy,
In my latest Ezra Project builds using SWORD trunk I’ve been noticing random „hangs“ and crashes related to "updating remote sources“. I suppose it must be around InstallMgr::refreshRemoteSource.
This was still rock solid when using SWORD trunk from May 18th 2020, but not so any more with the recent SWORD trunk.
Unfortunately I cannot pinpoint this more specifically. I just wanted to first share this observation, because it’s worrying me.
I’ve been noticing this regression both on Windows and macOS. Need to check later whether this also happens on Linux, cannot recall it right now.
Best regards,
Tobias<hr>sword-devel mailing list: <a class="moz-txt-link-abbreviated" href="mailto:sword-devel@crosswire.org" moz-do-not-send="true">sword-devel@crosswire.org</a>
<a href="http://crosswire.org/mailman/listinfo/sword-devel" moz-do-not-send="true">http://crosswire.org/mailman/listinfo/sword-devel</a>
Instructions to unsubscribe/change your settings at above page</pre>
</blockquote>
</div>
<br>
-- <br>
Sent from my Android device with K-9 Mail. Please excuse my
brevity. </blockquote>
</blockquote>
</div>
<br>
-- <br>
Sent from my Android device with K-9 Mail. Please excuse my
brevity.
</blockquote>
</body>
</html>