<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=DE link=blue vlink="#954F72"><div class=WordSection1><p class=MsoNormal><span lang=EN-US>Hi Troy,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Thanks for the feedback! </span><span lang=EN-US style='font-family:"Segoe UI Emoji",sans-serif'>😊</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>Yes, lazy loading in SWMgr would probably help. Looking forward to getting this improvement in place.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Regarding my original question:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>With my newly developed method </span>getRepoModuleIds I can now also rather quickly (< 1s) determine the repository a module belongs to (dynamically). The performance could still be optimized further, but it’s already noticably better than by going through SWMgr construction for all remote repositories.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I’m still planning to add the repo information for installed modules in my own persistence layer to further improve performance for the usecases mentioned below.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I have one more question.</p><p class=MsoNormal>Is there an example / easy way on how a SWModule object can be created independently of SWMgr? As of now SWMgr acts as a factory for SWModules, right? When working with specific modules, it may be enough to just create the objects directly, rather going through SWMgr?</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Best regards,<br>Tobias</p><p class=MsoNormal><o:p> </o:p></p><div style='mso-element:para-border-div;border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal style='border:none;padding:0cm'><b>From: </b><a href="mailto:scribe@crosswire.org">Troy A. Griffitts</a><br><b>Sent: </b>Montag, 30. September 2019 16:29<br><b>To: </b><a href="mailto:sword-devel@crosswire.org">SWORD Developers' Collaboration Forum</a><br><b>Subject: </b>Re: [sword-devel] Determining the repository a SWModule belongs to</p></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal style='margin-bottom:12.0pt'>Hi Tobias,<br>Yes, thank you for all this. We certainly could optimize SWMgr c-tor for large repos. My guess is that the creation of every SWModule for large repos is taking the time. Many impls of SWModule check their data files on c-tor. SWMgr also adds all the filters to each SWModule. All of this could be delayed until use. That's my suspicion for the speed difference. I'll keep this in mind for the next release and have a look at your code for a new method to InstallMgr. Thank you!<br><br>Troy<o:p></o:p></p><div><p class=MsoNormal>On September 30, 2019 5:13:23 AM MST, contact@tklein.info wrote:<o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm'><pre style='margin-bottom:12.0pt'>Hi,<br><br>in my Sword backend (node-sword-interface) I've now implemented a new <br>method isModuleAvailableInRepo:<br><br>bool isModuleAvailableInRepo(std::string moduleName, std::string <br>repoName="all");<br><br>This method avoids the creation of a SWMgr instance (which is slow for <br>large repos), but just quickly goes through the .conf files of <br>sword::InstallSource.localShadow. It then extracts the module ids <br>(from the first line of each .conf file) and checks whether the given <br>module id is found in the overall list.<br><br>On my Windows tablet this operation completes in 0.5s, whereas <br>creation of SWMgr + going through the ModMap takes 5.8s.<br>When executed for a bunch of modules at start-up this change <br>significantly reduces start-up time of Ezra Project (especially on <br>slower computers).<br><br>The question remains whether a similar function would make sense <br>directly in the SWORD engine.<br><br>Best regards,<br>Tobias<br><br>Some links for you to see how this was implemented:<br><br>bool isModuleAvailableInRepo(std::string moduleName, std::string <br>repoName="all");<br><a href="https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L483">https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L483</a><br><br>std::vector<std::string> getRepoModuleIds(std::string repoName);<br><a href="https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L301">https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L301</a><br><br>std::string getModuleIdFromFile(std::string moduleFileName);<br><a href="https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L276">https://github.com/tobias-klein/node-sword-interface/blob/0.68.0/sword_facade.cpp#L276</a><br><br>Quoting contact@tklein.info:</pre><blockquote style='border:none;border-left:solid #729FCF 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm;margin-bottom:6.0pt'><pre style='margin-bottom:12.0pt'> To correct what I wrote earlier ...<br> It seems to be the construction of SWMgr that takes some time, in my<br> case InstallSource::getMgr().<br> The actual iteration over the ModMap is fast.<br><br> Best regards,<br> Tobias<br><br> Quoting contact@tklein.info:</pre><blockquote style='border:none;border-left:solid #AD7FA8 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm;margin-bottom:6.0pt'><pre style='margin-bottom:12.0pt'> Hi,<br><br> For now I'm thinking to store this information (Repository of a<br> module) in my own application-level persistence layer.<br><br> However, I'm still wondering about the performance of<br> InstallMgr::getModuleStatus.<br> It's not a big issue on current desktop PCs with fast SSDs. On my<br> Windows tablet, though - it takes ~3s to call this function using<br> the "eBible.org" repository. When calling it for all repositories to<br> gather a complete list of modules, it takes ~6s on my Windows tablet.<br><br> I guess the function walks through all the module files and a long<br> list of files just takes its toll ... Is there a possibility for a<br> performance improvement, though?<br><br> Best regards,<br> Tobias<br><br> Quoting contact@tklein.info:</pre><blockquote style='border:none;border-left:solid #8AE234 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm;margin-bottom:6.0pt'><pre style='margin-bottom:12.0pt'> Hi Troy!</pre><blockquote style='border:none;border-left:solid #FCAF3E 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm;margin-bottom:6.0pt'><pre>I guess InstallMgr could add a line into the .conf when<br>installing: InstalledFrom, but that could be a side load, a local<br>repo, another SWORD application install on a local lan, or one of<br>our remote sources which might not still host the module.</pre></blockquote><pre><br> That would be helpful!<br><br> I have 2 use cases:<br><br> 1) Show the repository that a module was installed from as part of<br> other module meta information ("Module About Dialog")<br> 2) Check, which repository a certain module is available from in<br> the context of a "sync feature". In Ezra Project, I'm storing a<br> list of installed modules and this list can be synced between<br> different computers. My expectation as a user is that I want to<br> work with the same set of modules whichever of my computers I'm<br> using. The "sync feature" in Ezra Project basically checks the<br> "list of installed modules" (from Ezra Project synced<br> configuration, synced via Dropbox or similar service) and then<br> automatically installs missing modules if there are any modules in<br> the list that have not been installed locally yet. For this to work<br> efficiently it would be good to know the source repository of a<br> module, because otherwise I have to search all repositories for it<br> (That's what I'm currently doing and the performance is the issue).<br><br> Best regards,<br> Tobias</pre><pre><img border=0 width=447 height=1 style='width:4.6527in;height:.0069in' id="Horizontal_x0020_Line_x0020_1" src="cid:image001.png@01D577F4.389DD110"></pre><pre> sword-devel mailing list: sword-devel@crosswire.org<br> <a href="http://www.crosswire.org/mailman/listinfo/sword-devel">http://www.crosswire.org/mailman/listinfo/sword-devel</a><br> Instructions to unsubscribe/change your settings at above page</pre></blockquote><pre><img border=0 width=453 height=1 style='width:4.7222in;height:.0069in' id="Horizontal_x0020_Line_x0020_2" src="cid:image002.png@01D577F4.389DD110"></pre><pre> sword-devel mailing list: sword-devel@crosswire.org<br> <a href="http://www.crosswire.org/mailman/listinfo/sword-devel">http://www.crosswire.org/mailman/listinfo/sword-devel</a><br> Instructions to unsubscribe/change your settings at above page</pre></blockquote><pre><img border=0 width=459 height=1 style='width:4.7847in;height:.0069in' id="Horizontal_x0020_Line_x0020_3" src="cid:image003.png@01D577F4.389DD110"></pre><pre> sword-devel mailing list: sword-devel@crosswire.org<br> <a href="http://www.crosswire.org/mailman/listinfo/sword-devel">http://www.crosswire.org/mailman/listinfo/sword-devel</a><br> Instructions to unsubscribe/change your settings at above page</pre></blockquote><pre><img border=0 width=466 height=1 style='width:4.8541in;height:.0069in' id="Horizontal_x0020_Line_x0020_4" src="cid:image004.png@01D577F4.389DD110"></pre><pre>sword-devel mailing list: sword-devel@crosswire.org<br><a href="http://www.crosswire.org/mailman/listinfo/sword-devel">http://www.crosswire.org/mailman/listinfo/sword-devel</a><br>Instructions to unsubscribe/change your settings at above page</pre></blockquote></div><p class=MsoNormal><br>-- <br>Sent from my Android device with K-9 Mail. Please excuse my brevity.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p></div></body></html>