[sword-devel] Determining the repository a SWModule belongs to

[contact at tklein.info]

Hi,

in my Sword backend (node-sword-interface) I've now implemented a new  
method isModuleAvailableInRepo:

bool isModuleAvailableInRepo(std::string moduleName, std::string  
repoName="all");

This method avoids the creation of a SWMgr instance (which is slow for  
large repos), but just quickly goes through the .conf files of  
sword::InstallSource.localShadow. It then extracts the module ids  
(from the first line of each .conf file) and checks whether the given  
module id is found in the overall list.

On my Windows tablet this operation completes in 0.5s, whereas  
creation of SWMgr + going through the ModMap takes 5.8s.
When executed for a bunch of modules at start-up this change  
significantly reduces start-up time of Ezra Project (especially on  
slower computers).

The question remains whether a similar function would make sense  
directly in the SWORD engine.

Best regards,
Tobias

Some links for you to see how this was implemented:

bool isModuleAvailableInRepo(std::string moduleName, std::string  
repoName="all");
https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L483

std::vector<std::string> getRepoModuleIds(std::string repoName);
https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L301

std::string getModuleIdFromFile(std::string moduleFileName);
https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L276

Quoting contact at tklein.info:

> To correct what I wrote earlier ...
> It seems to be the construction of SWMgr that takes some time, in my
> case InstallSource::getMgr().
> The actual iteration over the ModMap is fast.
>
> Best regards,
> Tobias
>
> Quoting contact at tklein.info:
>
>> Hi,
>>
>> For now I'm thinking to store this information (Repository of a
>> module) in my own application-level persistence layer.
>>
>> However, I'm still wondering about the performance of
>> InstallMgr::getModuleStatus.
>> It's not a big issue on current desktop PCs with fast SSDs. On my
>> Windows tablet, though - it takes ~3s to call this function using
>> the "eBible.org" repository. When calling it for all repositories to
>> gather a complete list of modules, it takes ~6s on my Windows tablet.
>>
>> I guess the function walks through all the module files and a long
>> list of files just takes its toll ... Is there a possibility for a
>> performance improvement, though?
>>
>> Best regards,
>> Tobias
>>
>> Quoting contact at tklein.info:
>>
>>> Hi Troy!
>>>
>>>> I guess InstallMgr could add a line into the .conf when
>>>> installing: InstalledFrom, but that could be a side load, a local
>>>> repo, another SWORD application install on a local lan, or one of
>>>> our remote sources which might not still host the module.
>>>
>>> That would be helpful!
>>>
>>> I have 2 use cases:
>>>
>>> 1) Show the repository that a module was installed from as part of
>>> other module meta information ("Module About Dialog")
>>> 2) Check, which repository a certain module is available from in
>>> the context of a "sync feature". In Ezra Project, I'm storing a
>>> list of installed modules and this list can be synced between
>>> different computers. My expectation as a user is that I want to
>>> work with the same set of modules whichever of my computers I'm
>>> using. The "sync feature" in Ezra Project basically checks the
>>> "list of installed modules" (from Ezra Project synced
>>> configuration, synced via Dropbox or similar service) and then
>>> automatically installs missing modules if there are any modules in
>>> the list that have not been installed locally yet. For this to work
>>> efficiently it would be good to know the source repository of a
>>> module, because otherwise I have to search all repositories for it
>>> (That's what I'm currently doing and the performance is the issue).
>>>
>>> Best regards,
>>> Tobias
>>>
>>>
>>> _______________________________________________
>>> sword-devel mailing list: sword-devel at crosswire.org
>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>> Instructions to unsubscribe/change your settings at above page
>>
>>
>>
>>
>> _______________________________________________
>> sword-devel mailing list: sword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page
>
>
>
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page