[sword-devel] InstallMgr::getModuleStatus / Performance optimization possible?

[Tobias Klein]

Hi Troy,

Thanks for looking into this!

Just running this on my Dev PC (Linux, SSD disk, Intel i7) I am actually 
noticing a degradation with this change ... but only at first sight and 
compared to the SWORD version I have been using before.

I am running this function within a test binary.
https://github.com/ezra-bible-app/node-sword-interface/blob/1.0.0/src/sword_backend/repository_interface.cpp#L300

I previously used SVN Rev. 3873 (which is from Nov. 2021).
With that revision my test binary takes 1.65s to execute.

After switching to SVN Rev. 3887 (HEAD) the test binary takes 1.95 
seconds to execute, so roughly 300ms slower.

To be sure, I also tested with the SVN Rev. preceding your latest 
changes (3883) and this was a surprise: With that the test binary takes 
2.9s to execute.

So ... compared to SVN Rev. 3883 your latest changes were actually a 
significant performance improvement ... but compared to the version from 
one year ago (which is my production version) it is slower.

So something else must have changed in the last 12 months that impacts 
performance?!

Best regards,
Tobias

On 11/4/22 12:13 AM, Troy A. Griffitts wrote:
>
> OK Tobias,
>
> I have a first cut of a caching mechanism for a mods.d/ folder coded 
> up and pushed to trunk.  I have attempted to remove the cache when the 
> folder is changed (= new module is installed or removed), but I 
> possibly have forgotten something.  Give it a go and let me know if it 
> improves things for you on your slower devices and we'll go from there.
>
> Troy
>
>
> On 11/3/22 12:05, Tobias Klein wrote:
>>
>> Thank you, Troy!
>>
>> I'd be glad to test any changes you come up with!
>>
>> Best regards,
>> Tobias
>>
>> On 11/3/22 5:57 PM, Troy A. Griffitts wrote:
>>>
>>> Dear Tobias,
>>>
>>> I can have a look at optimizing getModuleStatus.  I can certainly 
>>> see that only checking locally installed modules might be a speed 
>>> improvement, but I suspect most of the time is loading all the .conf 
>>> files for each module, and that is done when constructing the SWMgr 
>>> (remote and local).  SWORD previously worked with a single 
>>> modules.conf file where the section was appended each time a module 
>>> was loaded.  It still will work in this manner.  I might try dumping 
>>> all the .conf files into a single cache.conf file and using that for 
>>> reading the info of a remote repository-- which can include 
>>> thousands of individual .conf files.  We can see if that speeds up 
>>> this operation.
>>>
>>> On 10/30/22 08:49, Tobias Klein wrote:
>>>>
>>>> Hi Troy,
>>>>
>>>> When integrating the module update functionality in Ezra Bible App, 
>>>> I noticed a performance issue in the function 
>>>> InstallMgr::getModuleStatus.
>>>>
>>>> On my laptop, it takes almost two seconds to run this against all 
>>>> repositories from the master repo list. On my slower surface 
>>>> tablet, it takes even longer and I haven’t tested it on my even 
>>>> slower Android devices, yet. This generates a bit of an issue in my 
>>>> JavaScript based application (longer interrupts of the JavaScript 
>>>> event loop lead to some freezing in the UI).
>>>>
>>>> Considering the parameters constSWMgr &base, constSWMgr &other, I 
>>>> saw that the function loops through all modules within other. If 
>>>> one just wants to see which local modules are outdated, it would be 
>>>> enough to go through the ones that are also present within base, 
>>>> right? Could that be a way of optimizing the performance?
>>>>
>>>> Best regards,
>>>> Tobias
>>>>
>>>>
>>>> _______________________________________________
>>>> sword-devel mailing list:sword-devel at crosswire.org
>>>> http://crosswire.org/mailman/listinfo/sword-devel
>>>> Instructions to unsubscribe/change your settings at above page
>>>
>>> _______________________________________________
>>> sword-devel mailing list:sword-devel at crosswire.org
>>> http://crosswire.org/mailman/listinfo/sword-devel
>>> Instructions to unsubscribe/change your settings at above page
>>
>> _______________________________________________
>> sword-devel mailing list:sword-devel at crosswire.org
>> http://crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://crosswire.org/pipermail/sword-devel/attachments/20221104/6086aeb8/attachment.htm>