[jsword-devel] New versification mapping - preload mappings to prevent pauses

[Martin Denham]

I tried your patch and it did noticeably improve things, so it is a
worthwhile patch, but did not solve the actual problem.  Still viewing the
RusSynodal straight after download takes well over a minute, whereas to
boot straight into it takes about 2 secs.

One or 2 things I have noticed:

   - a huge amount of memory allocation and GC occurs when trying to view
   RusSynodal straight after download
   - I am not great with the profiler but here are my tentative observations
      - Seems to be spending about half the time in
      java.util.BitSet.cardinality()
         - called by
         BitwisePassage.countVerses()<-RocketPassage.countVerses()<-AbstractPassage.getCardinality()<-VersificationToKJVMapper.add...
      - I tried caching AbstractPassage.getCardinality but that had no
      affect which would be the case if there were lots of different keys
   - Could it be that a module is in an unusual state after download?  I
   have noticed that a module cannot be deleted directly after download, until
   the app is restarted due to:
      - sbmd.getConfigFile()==null when called by
      SwordBookDriver.isDeletable directly after module download

Any suggestions are welcome.

Cheers
Martin


On 16 January 2014 23:13, Martin Denham <mjdenham at gmail.com> wrote:

> I think I will use the old AB v11n mapping for the next release of AB and
> come back to this after the release which has quite a lot of changes in it
> already anyway.
>
> The problem could be AB, Android, or JSword related and it is not trivial
> to reproduce involving repeated re-installs of RusSynodal.  I had a similar
> problem a year or two ago, it took a long time to investigate and ended up
> being a flaw in the Android VM which required a change to JSword.
>
> I keep trying to think what could cause these symptoms and drawing a blank.
>
> Cheers
> Martin
>
>
> On 16 January 2014 21:35, Chris Burrell <chris at burrell.me.uk> wrote:
>
>> I'll try profiling it! Nothing comes to mind though.
>> On 16 Jan 2014 20:04, "Martin Denham" <mjdenham at gmail.com> wrote:
>>
>>> There is something unusual happening with the v11n mapping creation
>>> straight after downloading a file requiring mapping e.g. It takes over 2
>>> mins to load the mapping (a single call to mapVerse) for RusSynodal
>>> straight after download but a few seconds to load the file normally.
>>>
>>> 1. View ESV and anything else in split screen
>>> 2. Download RusSynodal
>>> 3. Exit download screen after download
>>> 4. select RusSynodal for display (with ESV) in 1 half of split screen
>>> 5. Mapping code now takes over 2 minutes to load mapping
>>> 6. But there are no errors and then everything seems fine
>>>
>>> 7. Exit and kill And Bible
>>> 8. Restart with ESV/RST in split screens
>>> 9. This time everything is initialised in a few seconds
>>>
>>> Any ideas?
>>>
>>> Martin
>>>
>>>
>>> On 16 January 2014 19:06, Chris Burrell <chris at burrell.me.uk> wrote:
>>>
>>>> Hi Martin
>>>>
>>>> Fine by me. Think it should be opt in if possible.
>>>>
>>>> A couple of thoughts on that note. Do you know of it s the io or the
>>>> cpu time that's the issue?
>>>>
>>>> We probably want to opt in with a list as well as all as there's no
>>>> point in loading the v11n if not required.
>>>>
>>>> I guess especially if we end up with mappings per module eventually.
>>>>
>>>> On a separate note, I also found the books.installed call is very
>>>> expensive. Thinking it may be worth partially loading these. With around
>>>> 200 modules we spend almost 15 seconds loading them.
>>>>
>>>> Finally you can apply for a open source license of jprofiler which
>>>> helps massively to work out what's going on. Have got a couple of
>>>> uncommitted fixes for books. Installed find with that.
>>>>
>>>> Chris
>>>> On 16 Jan 2014 18:10, "Martin Denham" <mjdenham at gmail.com> wrote:
>>>>
>>>>>  Hi,
>>>>>
>>>>> I have integrated the new v11n mapping code and find I am getting a
>>>>> pause when doing the initial mapping between any 2 v11ns.
>>>>>
>>>>> I originally had the same problem with the AB mapping code so
>>>>> pre-loaded all mapping required for the installed set of documents at the
>>>>> start in a background thread to prevent delays.  The code I used is in
>>>>> VersificationMappingFactory.initialiseRequiredMappings()<https://github.com/mjdenham/and-bible/blob/master/AndBible/src/net/bible/android/control/versification/mapping/VersificationMappingFactory.java>
>>>>> .
>>>>>
>>>>> I am trying to think of the best way to do this in the new code.
>>>>>  Either i) you could add a method to do it which could be called or ii) you
>>>>> could proeload all required mappings automatically or iii) add a new method
>>>>> to allow AB (or other frontend) to trigger preload of all required mappings
>>>>> by adding a public method a bit like the new
>>>>> VersificationsMapper.ensure(v11ntopreload).
>>>>>
>>>>> This is quite an issue for mobile users.  I have a fast mobile and
>>>>> loading a mapping causes a noticeable delay the first time a verse changes,
>>>>> but the preload fix is fairly simple and worked well.
>>>>>
>>>>> Cheers
>>>>> Martin
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> jsword-devel mailing list
>>>>> jsword-devel at crosswire.org
>>>>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>>>>
>>>>>
>>>> _______________________________________________
>>>> jsword-devel mailing list
>>>> jsword-devel at crosswire.org
>>>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>>>
>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/jsword-devel/attachments/20140118/132ae2cd/attachment.html>