[sword-devel] indexed search discrepancy

[Matthew Talbert]

> I had submitted a patch that did this and it was rejected because it did not
> preserve backward compatibility without providing a versioning system for
> each generated index.

If by backward compatibility, you mean that old indexes will still
work as they always have, then backwards compatibility is being
preserved (this is how I would interpret it). But new indexes will
obviously be different than the old ones. If this is what you mean,
then we really can't change anything in the indexing until some
versioning scheme is implemented, correct? The recent Hebrew changes
broke both of these principles: old indexes are unusable (will return
0 results for modules that have Hebrew vowels), and new indexes are
different than the old ones. The changes to the size of the fields
allowed will do the same thing, although old indexes will still be
usable (if you call returning 30% of the actual hits usable). I agree
with the need for versioning (I mentioned it first in this thread :)
), but to not fix bugs because of that seems silly.

> As to using a simple incrementing number to represent the version of the
> index, this may not be adequate. It is sufficient if the user has no control
> over the index and indexes that do not match the version number of the
> engine are ignored/discarded/automatically upgraded... by the front-end or
> engine.

I believe we should follow the principle of "do the simplest thing
that will possibly work". All we need at the moment is a simple
version number. Everything without version numbers will be presumed to
be older. In my opinion, if the version number is older than the
(index) version of the library, then the library should just return
false when asked if the module has fast search framework (I forget the
function name). Then the front-end can do whatever it needs in that
situation. This also has the advantage of not needing a new API.

> Give the user any control over the index or provide the front-end any
> indication of what is in the index and it is not sufficient. Further, once
> we get to analyzers per language each feature needs a version number as
> well.
>
> Very messy.

Yes, but we're not there today. Considering that currently none of the
non-English analyzers are ported to C++, to not do something now, or
to design a complicated system based on functionality that may never
arrive, seems backwards.

> The solution we have for BibleDesktop/JSword is to just let the user know
> that if search does not perform as expected to delete the index and rebuild
> it. Not at all a good solution, but we've not had any complaints.

The best solution is not always the most technically correct solution.
As above, many times it's the simplest solution that is best.

Matthew