<p dir="ltr">The reason I went for passages is for less memory consumption. I also thought it would provide better parsing and we're parsing less. </p>
<p dir="ltr">But I think the way we store passages is to represent every verse by a bit. means that iterating through ranges, calculating cardinality or initialising them means is expensive. </p>
<p dir="ltr">I wonder if we could have a passage that just stores boundaries rather than all its content. That would be very fast to parse and very quick for operations on it. </p>
<p dir="ltr">Chris<br>
</p>
<div class="gmail_quote">On 18 Jan 2014 19:17, "Martin Denham" <<a href="mailto:mjdenham@gmail.com">mjdenham@gmail.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">I think I have found a workaround for this problem (using a background thread makes it run very quickly) but it seems to point toward a possible point of improvement in the new JSword mapping data structure.<div>
<br></div><div>In the old AB v11n mapping code I stored maps of Verses. However the new JSword code seems to store maps of RocketPassage/BitwisePassage. Can you verify this is the best data structure for what will normally be a single verse. I don't know an awful lot about this so I am just asking, but maybe a Passage will always consume more memory than a Verse.</div>
<div><br></div><div>Thanks</div><div>Martin</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 18 January 2014 13:30, Martin Denham <span dir="ltr"><<a href="mailto:mjdenham@gmail.com" target="_blank">mjdenham@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I tried your patch and it did noticeably improve things, so it is a worthwhile patch, but did not solve the actual problem. Still viewing the RusSynodal straight after download takes well over a minute, whereas to boot straight into it takes about 2 secs.<div>
<br></div><div>One or 2 things I have noticed:</div><div><ul><li>a huge amount of memory allocation and GC occurs when trying to view RusSynodal straight after download<br></li><li>I am not great with the profiler but here are my tentative observations</li>
<ul><li>Seems to be spending about half the time in java.util.BitSet.cardinality() </li><ul><li>called by BitwisePassage.countVerses()<-RocketPassage.countVerses()<-AbstractPassage.getCardinality()<-VersificationToKJVMapper.add...</li>
</ul><li>I tried caching AbstractPassage.getCardinality but that had no affect which would be the case if there were lots of different keys</li></ul><li>Could it be that a module is in an unusual state after download? I have noticed that a module cannot be deleted directly after download, until the app is restarted due to:</li>
<ul><li>sbmd.getConfigFile()==null when called by SwordBookDriver.isDeletable directly after module download</li></ul>
</ul><div>Any suggestions are welcome.</div></div><div><br></div><div>Cheers</div><span><font color="#888888"><div>Martin</div></font></span><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">
On 16 January 2014 23:13, Martin Denham <span dir="ltr"><<a href="mailto:mjdenham@gmail.com" target="_blank">mjdenham@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">I think I will use the old AB v11n mapping for the next release of AB and come back to this after the release which has quite a lot of changes in it already anyway.<div>
<br></div><div>The problem could be AB, Android, or JSword related and it is not trivial to reproduce involving repeated re-installs of RusSynodal. I had a similar problem a year or two ago, it took a long time to investigate and ended up being a flaw in the Android VM which required a change to JSword. </div>
<div><br></div><div>I keep trying to think what could cause these symptoms and drawing a blank.</div><div><br></div><div>Cheers</div><span><font color="#888888"><div>Martin</div></font></span></div><div>
<div><div class="gmail_extra"><br><br><div class="gmail_quote">On 16 January 2014 21:35, Chris Burrell <span dir="ltr"><<a href="mailto:chris@burrell.me.uk" target="_blank">chris@burrell.me.uk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><p dir="ltr">I'll try profiling it! Nothing comes to mind though. </p>
<div><div>
<div class="gmail_quote">On 16 Jan 2014 20:04, "Martin Denham" <<a href="mailto:mjdenham@gmail.com" target="_blank">mjdenham@gmail.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">There is something unusual happening with the v11n mapping creation straight after downloading a file requiring mapping e.g. It takes over 2 mins to load the mapping (a single call to mapVerse) for RusSynodal straight after download but a few seconds to load the file normally.<div>
<br></div><div>1. View ESV and anything else in split screen</div><div>2. Download RusSynodal</div><div>3. Exit download screen after download</div><div>4. select RusSynodal for display (with ESV) in 1 half of split screen</div>
<div>5. Mapping code now takes over 2 minutes to load mapping</div><div>6. But there are no errors and then everything seems fine </div><div><br></div><div>7. Exit and kill And Bible</div><div>8. Restart with ESV/RST in split screens</div>
<div>9. This time everything is initialised in a few seconds</div><div><br></div><div>Any ideas?</div><div><br></div><div>Martin</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 16 January 2014 19:06, Chris Burrell <span dir="ltr"><<a href="mailto:chris@burrell.me.uk" target="_blank">chris@burrell.me.uk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><p dir="ltr">Hi Martin</p>
<p dir="ltr">Fine by me. Think it should be opt in if possible. </p>
<p dir="ltr">A couple of thoughts on that note. Do you know of it s the io or the cpu time that's the issue? </p>
<p dir="ltr">We probably want to opt in with a list as well as all as there's no point in loading the v11n if not required. </p>
<p dir="ltr">I guess especially if we end up with mappings per module eventually. </p>
<p dir="ltr">On a separate note, I also found the books.installed call is very expensive. Thinking it may be worth partially loading these. With around 200 modules we spend almost 15 seconds loading them. </p>
<p dir="ltr">Finally you can apply for a open source license of jprofiler which helps massively to work out what's going on. Have got a couple of uncommitted fixes for books. Installed find with that. </p>
<p dir="ltr">Chris</p>
<div class="gmail_quote"><div><div>On 16 Jan 2014 18:10, "Martin Denham" <<a href="mailto:mjdenham@gmail.com" target="_blank">mjdenham@gmail.com</a>> wrote:<br type="attribution"></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><div>
<div dir="ltr">Hi,<div><br></div><div>I have integrated the new v11n mapping code and find I am getting a pause when doing the initial mapping between any 2 v11ns.</div><div><br></div><div>I originally had the same problem with the AB mapping code so pre-loaded all mapping required for the installed set of documents at the start in a background thread to prevent delays. The code I used is in <a href="https://github.com/mjdenham/and-bible/blob/master/AndBible/src/net/bible/android/control/versification/mapping/VersificationMappingFactory.java" target="_blank">VersificationMappingFactory.<span style="color:rgb(153,0,0);font-family:Consolas,'Liberation Mono',Courier,monospace;font-size:12px;font-weight:bold;line-height:18px;white-space:pre-wrap">initialiseRequiredMappings()</span></a>.</div>
<div><br></div><div>I am trying to think of the best way to do this in the new code. Either i) you could add a method to do it which could be called or ii) you could proeload all required mappings automatically or iii) add a new method to allow AB (or other frontend) to trigger preload of all required mappings by adding a public method a bit like the new VersificationsMapper.ensure(v11ntopreload).</div>
<div><br></div><div>This is quite an issue for mobile users. I have a fast mobile and loading a mapping causes a noticeable delay the first time a verse changes, but the preload fix is fairly simple and worked well.</div>
<div><br></div><div>Cheers</div><div>Martin</div><div><br></div></div>
<br></div></div>_______________________________________________<br>
jsword-devel mailing list<br>
<a href="mailto:jsword-devel@crosswire.org" target="_blank">jsword-devel@crosswire.org</a><br>
<a href="http://www.crosswire.org/mailman/listinfo/jsword-devel" target="_blank">http://www.crosswire.org/mailman/listinfo/jsword-devel</a><br>
<br></blockquote></div>
<br>_______________________________________________<br>
jsword-devel mailing list<br>
<a href="mailto:jsword-devel@crosswire.org" target="_blank">jsword-devel@crosswire.org</a><br>
<a href="http://www.crosswire.org/mailman/listinfo/jsword-devel" target="_blank">http://www.crosswire.org/mailman/listinfo/jsword-devel</a><br>
<br></blockquote></div><br></div>
</blockquote></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div></div></div>
</blockquote></div><br></div>
</blockquote></div>