[jsword-devel] A thought on search indexing.

[DM Smith]

I agree that it is too slow. I have put a line item into bugs.txt for 
it. My thought was that we would do it in the background (low priority) 
as part of the install rather than waiting until they search for the 
first time.

I like your idea, but we may want to wait until after 1.0 because I 
think we should spend some time looking at it to see if we can optimize 
it. I have cut the time significantly already, but I think it can be faster.

Another problem I have and I think Random J. User will have is that the 
first search seems to promise that there will be a result after the 
index is built. But no, they have to submit the search again.

Also, I once got frustrated with the indexing (when it was really slow) 
and killed it. It left junk on my machine, and the program thought it 
had a valid index. The only way I could fix the problem was to delete 
the index. I had to read the code to determine where it was and what it 
was called.

Mark Goodwin wrote:

>I can see Random J. User getting fed up with search indexes.  Firstly
>there's the time taken to generate them.
>
>It takes about 7 minutes on my PowerBook to generate a search index
>for the KJV sword module (yeah, I know, it's my fault for owning a
>Mac!).  Generating exactly the same index on my Athlon
>twenty-something-hundred-plus Linux box takes 3.5 minutes.
>
>Now, when I nuke my .jsword dir on the mac, I wonder about copying the
>indexes from the Linux box.
>
>I definitely think that indexed searches are the way forward, and
>whilst I'm sure there's room for optimisation in the indexing code,
>I'm wondering whether a significant proportion of users would be
>better served with a different solution.
>
>how about downloadable search indexes?  An uncompressed index is only
>1.3 MB for the same book.  If I compress it down, it squeezes to a
>mere 1 MB.
>
>We could leave the option of generating your own for the bandwidth impaired.
>
>The other thing I think may frustrate Random J. User is the point at
>which an index is created.  I think it might be a good idea to ask the
>user if they would like an index created / obtained at module install
>time; that way the hit of generating / downloading the index is
>grouped along with the other stuff that keeps the user waiting...
>
>Good idea? Bad Idea?
>_______________________________________________
>jsword-devel mailing list
>jsword-devel at crosswire.org
>http://www.crosswire.org/mailman/listinfo/jsword-devel
>
>  
>