I stripped out all the progress and size stuff from WebResource.copy and the download time for Darby decreased to 20 secs - 3 times faster than I have ever seen before even with the previous code!!!<div><br></div><div>Have to go now but will get back to this later.<br>
<br><div class="gmail_quote">On 4 December 2010 17:42, Martin Denham <span dir="ltr"><<a href="mailto:mjdenham@gmail.com">mjdenham@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
I don't think so. The code I was using before, which was fast, updated the Progress after every block fetch in the inner loop in WebResource.copy().<div><div></div><div class="h5"><br><br><div class="gmail_quote">On 4 December 2010 17:24, DM Smith <span dir="ltr"><<a href="mailto:dmsmith@crosswire.org" target="_blank">dmsmith@crosswire.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF"><div>Wild guess: it updates progress too often. <br><br>Cent from my fone so theer mite be tipos. ;)</div>
<div><div></div><div><div><br>On Dec 4, 2010, at 12:17 PM, Martin Denham <<a href="mailto:mjdenham@gmail.com" target="_blank">mjdenham@gmail.com</a>> wrote:<br><br></div><div></div><blockquote type="cite">
<div>I have reverted to the code I had a couple of days ago and the speed is back to normal again i.e. ~1 min to download Darby bible compared to >10 min with the latest code.<div><br></div><div>It looks like a JSword issue but could be an Android environment issue because I did see diabolical download performance some months ago due to configuration.<br>
<div><br></div><div><div class="gmail_quote">On 4 December 2010 16:54, Martin Denham (JIRA) <span dir="ltr"><<a href="mailto:jira@crosswire.org" target="_blank"></a><a href="mailto:jira@crosswire.org" target="_blank">jira@crosswire.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
[ <a href="http://www.crosswire.org/bugs/browse/JS-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631#action_13631" target="_blank"></a><a href="http://www.crosswire.org/bugs/browse/JS-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631#action_13631" target="_blank">http://www.crosswire.org/bugs/browse/JS-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631#action_13631</a> ]<br>
<br>
Martin Denham commented on JS-133:<br>
----------------------------------<br>
<br>
Hi DM,<br>
<br>
I am getting horrendous download performance with the latest build compared to the version I got a few days ago after you had first updated to the latest httpclient and jdom. Downloads are not just a bit longer but take ~10 times longer.<br>
<br>
Any idea why? It isn't just the time of day because I did a fast test just 10 mins ago then upgraded my mobile and now the progress bar just crawls from left to right taking ~10 minutes when it used to take ~1 min.<br>
<br>
Also, the Progress message used to say something like "Installing book: <module name>" but now it just says "Install". Is this my error or JSword's?<br>
<br>
Regards<br>
Martin<br>
<div><div></div><div><br>
<br>
> Percentage complete during a download should reflect file size<br>
> --------------------------------------------------------------<br>
><br>
> Key: JS-133<br>
> URL: <a href="http://www.crosswire.org/bugs/browse/JS-133" target="_blank"></a><a href="http://www.crosswire.org/bugs/browse/JS-133" target="_blank">http://www.crosswire.org/bugs/browse/JS-133</a><br>
> Project: JSword<br>
> Issue Type: Improvement<br>
> Components: o.c.common.util, o.c.jsword.book.install<br>
> Environment: Android<br>
> Reporter: Martin Denham<br>
> Assignee: DM Smith<br>
><br>
> Downloads to a mobile are slower and more variable than to a pc and there can be quite a large difference between predicted download time and actual download time. Also, the first download does not have a predicted download time and so the first progress bar And Bible showed to users had no percentage complete (maybe this second problem could have been avoided by pre-loading a guesstimate) .<br>
> There is code in JSword to get the download file size (WebResource.getSize()) but the actual download loop does not have access to the Progress object in order to update the percentage so I changed the code to pass the Progress object to the download method. And Bible has been using this for a few weeks now to allow actual download percent to be displayed with no known problems.<br>
> There were a few changes in different classes:<br>
> AbstractSwordInstaller.run() determines if the download times are predicted. I had to set the last parameter to false to turn predictions off:<br>
> //change final parameter to false to prevent guessing progress of file download because now actual file size is used<br>
> Progress job = JobManager.createJob(UserMsg.gettext("Installing book: {0}", sbmd.getName()), predictURI, this, false);<br>
> In HttpSwordInstaller.copy(..) pass the Progress object through to WebResource which actually does the download:<br>
> private void copy(Progress job, URI uri, URI dest) throws LucidException {<br>
> if (job != null) {<br>
> // TRANSLATOR: Progress label for downloading one or more files.<br>
> job.setSectionName(UserMsg.gettext("Downloading files"));<br>
> }<br>
> //MJD START - modified following line - pass job thro to WebResource to allow actual progress indicator<br>
> WebResource wr = new WebResource(uri, proxyHost, proxyPort, job);<br>
> wr.copy(dest);<br>
> }<br>
> The above implies a new constructor was also needed and created for WebResource.<br>
> The WebResource.copy method is where the percent complete is updated currently in And Bible. It could be argued that this creates inefficiency in a tight loop but the loop isn't that tight because it deals in blocks rather than bytes. Inefficiency of updating Progress too often was the reason I added progressUpdateCounter so the calls do not happen after every block is fetched, but in reality it did not really save much as anything more than every other time leads to a very jumpy progress bar so I think the progressUpdateCounter part of below could be removed so that the progress is updated every time.<br>
> 95.0% was used below, as in other places to allow 5% for post download operations but I think users expect a delay after download so it could be 100.0%.<br>
> My first attempt involved calling getSize() somewhere after GetMethod and it just hung, hence the comment below.<br>
> public void copy(URI dest) throws LucidException {<br>
> InputStream in = null;<br>
> OutputStream out = null;<br>
> //MJD START calculate progress as percent of total file size<br>
> //must call getSize before opening the connection or it hangs<br>
> int size = getSize();<br>
> HttpMethod method = new GetMethod(uri.getPath());<br>
> try {<br>
> // Execute the method.<br>
> int status = client.executeMethod(method);<br>
> if (status == HttpStatus.SC_OK) {<br>
> in = method.getResponseBodyAsStream();<br>
> // Download the index file<br>
> out = NetUtil.getOutputStream(dest);<br>
> int progressUpdateCounter = 0;<br>
> long totalRead = 0;<br>
> byte[] buf = new byte[4096];<br>
> int count = in.read(buf);<br>
> while (-1 != count) {<br>
> out.write(buf, 0, count);<br>
> count = in.read(buf);<br>
><br>
> totalRead += count;<br>
> //update progress occasionally - every other time<br>
> if (job!=null && size>0 && progressUpdateCounter++ % 2 == 0) {<br>
> int pctRead = (int)((95.0*totalRead)/size);<br>
> job.setWork(pctRead);<br>
> }<br>
> }<br>
> } else {<br>
> // rest is unchanged<br>
> My only reservation is that getSize() may sometimes not work but I have not found any cases where it failed although And Bible only currently downloads from the CrossWire repository.<br>
<br>
--<br>
This message is automatically generated by JIRA.<br>
-<br>
If you think it was sent incorrectly contact one of the administrators: <a href="http://www.crosswire.org/bugs/secure/Administrators.jspa" target="_blank"></a><a href="http://www.crosswire.org/bugs/secure/Administrators.jspa" target="_blank">http://www.crosswire.org/bugs/secure/Administrators.jspa</a><br>
-<br>
For more information on JIRA, see: <a href="http://www.atlassian.com/software/jira" target="_blank"></a><a href="http://www.atlassian.com/software/jira" target="_blank">http://www.atlassian.com/software/jira</a><br>
<br>
<br>
<br>
_______________________________________________<br>
jsword-devel mailing list<br>
<a href="mailto:jsword-devel@crosswire.org" target="_blank"></a><a href="mailto:jsword-devel@crosswire.org" target="_blank">jsword-devel@crosswire.org</a><br>
<a href="http://www.crosswire.org/mailman/listinfo/jsword-devel" target="_blank"></a><a href="http://www.crosswire.org/mailman/listinfo/jsword-devel" target="_blank">http://www.crosswire.org/mailman/listinfo/jsword-devel</a><br>
</div></div></blockquote></div><br></div></div>
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>jsword-devel mailing list</span><br><span><a href="mailto:jsword-devel@crosswire.org" target="_blank">jsword-devel@crosswire.org</a></span><br>
<span><a href="http://www.crosswire.org/mailman/listinfo/jsword-devel" target="_blank">http://www.crosswire.org/mailman/listinfo/jsword-devel</a></span><br></div></blockquote></div></div></div><br>_______________________________________________<br>
jsword-devel mailing list<br>
<a href="mailto:jsword-devel@crosswire.org" target="_blank">jsword-devel@crosswire.org</a><br>
<a href="http://www.crosswire.org/mailman/listinfo/jsword-devel" target="_blank">http://www.crosswire.org/mailman/listinfo/jsword-devel</a><br>
<br></blockquote></div><br>
</div></div></blockquote></div><br></div>