[sword-devel] demo TEI modules

[Troy A. Griffitts]

> Could it be possible to add the list of the words into binary module
> data and then add a function to the API which would give that list? This
> would also solve the problem of the frontends freezing when loading e.g.
> Webster.

The issue with frontends freezing isn't because the engine is taking so 
long iterating the entire key list for them.  It is because (I believe I 
am correct) some frontends try to populate a GUI control with the entire 
list.


Also, Chris is correct about the current ld driver.  It first does a 
toupper_utf8 on the word and then stores it in order doing a binary 
search using standard strcmp.

http://crosswire.org/svn/sword/trunk/src/modules/common/rawstr.cpp
see RawStr::findOffset


We probably need to do a few things here besides toupper (to assure 
entry matches), as we've learned and done in our search code.  We 
probably should at least normalize the utf8.  This is not a big hit 
because it is only done on module creation for every key, and then once 
for the input word before the binary search starts.

We could change the actual order to use a utf8 strcmp method, but this 
would likely come with a relatively significant performance hit (though 
maybe not-- the binary search algol will significantly limit the number 
of actual utf8 strcmp operations we would need to perform).  This change 
would require remaking any modules which use multibyte utf8 keys.