[jsword-devel] New versification mapping - preload mappings to prevent pauses

Martin Denham mjdenham at gmail.com
Sat Jan 18 12:17:54 MST 2014 

I think I have found a workaround for this problem (using a background
thread makes it run very quickly) but it seems to point toward a possible
point of improvement in the new JSword mapping data structure.

In the old AB v11n mapping code I stored maps of Verses.  However the new
JSword code seems to store maps of RocketPassage/BitwisePassage.  Can you
verify this is the best data structure for what will normally be a single
verse.  I don't know an awful lot about this so I am just asking, but maybe
a Passage will always consume more memory than a Verse.

Thanks
Martin




On 18 January 2014 13:30, Martin Denham <mjdenham at gmail.com> wrote:

> I tried your patch and it did noticeably improve things, so it is a
> worthwhile patch, but did not solve the actual problem.  Still viewing the
> RusSynodal straight after download takes well over a minute, whereas to
> boot straight into it takes about 2 secs.
>
> One or 2 things I have noticed:
>
>    - a huge amount of memory allocation and GC occurs when trying to view
>    RusSynodal straight after download
>    - I am not great with the profiler but here are my tentative
>    observations
>       - Seems to be spending about half the time in
>       java.util.BitSet.cardinality()
>          - called by
>          BitwisePassage.countVerses()<-RocketPassage.countVerses()<-AbstractPassage.getCardinality()<-VersificationToKJVMapper.add...
>       - I tried caching AbstractPassage.getCardinality but that had no
>       affect which would be the case if there were lots of different keys
>    - Could it be that a module is in an unusual state after download?  I
>    have noticed that a module cannot be deleted directly after download, until
>    the app is restarted due to:
>       - sbmd.getConfigFile()==null when called by
>       SwordBookDriver.isDeletable directly after module download
>
> Any suggestions are welcome.
>
> Cheers
> Martin
>
>
> On 16 January 2014 23:13, Martin Denham <mjdenham at gmail.com> wrote:
>
>> I think I will use the old AB v11n mapping for the next release of AB and
>> come back to this after the release which has quite a lot of changes in it
>> already anyway.
>>
>> The problem could be AB, Android, or JSword related and it is not trivial
>> to reproduce involving repeated re-installs of RusSynodal.  I had a similar
>> problem a year or two ago, it took a long time to investigate and ended up
>> being a flaw in the Android VM which required a change to JSword.
>>
>> I keep trying to think what could cause these symptoms and drawing a
>> blank.
>>
>> Cheers
>> Martin
>>
>>
>> On 16 January 2014 21:35, Chris Burrell <chris at burrell.me.uk> wrote:
>>
>>> I'll try profiling it! Nothing comes to mind though.
>>>  On 16 Jan 2014 20:04, "Martin Denham" <mjdenham at gmail.com> wrote:
>>>
>>>> There is something unusual happening with the v11n mapping creation
>>>> straight after downloading a file requiring mapping e.g. It takes over 2
>>>> mins to load the mapping (a single call to mapVerse) for RusSynodal
>>>> straight after download but a few seconds to load the file normally.
>>>>
>>>> 1. View ESV and anything else in split screen
>>>> 2. Download RusSynodal
>>>> 3. Exit download screen after download
>>>> 4. select RusSynodal for display (with ESV) in 1 half of split screen
>>>> 5. Mapping code now takes over 2 minutes to load mapping
>>>> 6. But there are no errors and then everything seems fine
>>>>
>>>> 7. Exit and kill And Bible
>>>> 8. Restart with ESV/RST in split screens
>>>> 9. This time everything is initialised in a few seconds
>>>>
>>>> Any ideas?
>>>>
>>>> Martin
>>>>
>>>>
>>>> On 16 January 2014 19:06, Chris Burrell <chris at burrell.me.uk> wrote:
>>>>
>>>>> Hi Martin
>>>>>
>>>>> Fine by me. Think it should be opt in if possible.
>>>>>
>>>>> A couple of thoughts on that note. Do you know of it s the io or the
>>>>> cpu time that's the issue?
>>>>>
>>>>> We probably want to opt in with a list as well as all as there's no
>>>>> point in loading the v11n if not required.
>>>>>
>>>>> I guess especially if we end up with mappings per module eventually.
>>>>>
>>>>> On a separate note, I also found the books.installed call is very
>>>>> expensive. Thinking it may be worth partially loading these. With around
>>>>> 200 modules we spend almost 15 seconds loading them.
>>>>>
>>>>> Finally you can apply for a open source license of jprofiler which
>>>>> helps massively to work out what's going on. Have got a couple of
>>>>> uncommitted fixes for books. Installed find with that.
>>>>>
>>>>> Chris
>>>>> On 16 Jan 2014 18:10, "Martin Denham" <mjdenham at gmail.com> wrote:
>>>>>
>>>>>>  Hi,
>>>>>>
>>>>>> I have integrated the new v11n mapping code and find I am getting a
>>>>>> pause when doing the initial mapping between any 2 v11ns.
>>>>>>
>>>>>> I originally had the same problem with the AB mapping code so
>>>>>> pre-loaded all mapping required for the installed set of documents at the
>>>>>> start in a background thread to prevent delays.  The code I used is in
>>>>>> VersificationMappingFactory.initialiseRequiredMappings()<https://github.com/mjdenham/and-bible/blob/master/AndBible/src/net/bible/android/control/versification/mapping/VersificationMappingFactory.java>
>>>>>> .
>>>>>>
>>>>>> I am trying to think of the best way to do this in the new code.
>>>>>>  Either i) you could add a method to do it which could be called or ii) you
>>>>>> could proeload all required mappings automatically or iii) add a new method
>>>>>> to allow AB (or other frontend) to trigger preload of all required mappings
>>>>>> by adding a public method a bit like the new
>>>>>> VersificationsMapper.ensure(v11ntopreload).
>>>>>>
>>>>>> This is quite an issue for mobile users.  I have a fast mobile and
>>>>>> loading a mapping causes a noticeable delay the first time a verse changes,
>>>>>> but the preload fix is fairly simple and worked well.
>>>>>>
>>>>>> Cheers
>>>>>> Martin
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> jsword-devel mailing list
>>>>>> jsword-devel at crosswire.org
>>>>>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> jsword-devel mailing list
>>>>> jsword-devel at crosswire.org
>>>>> http://www.crosswire.org/mailman/listinfo/jsword-devel
>>>>>
>>>>>
>>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/jsword-devel/attachments/20140118/6a5a7cd0/attachment.html>

More information about the jsword-devel mailing list