[bt-devel] utf lexicons + bug

[Martin Gruner]

1. I changed the lexicon key handling to UTF8. Does anyone have objections to 
this? It requires a cache rebuild, which should happen automatically now. I 
tested it with Websters (see the last entries) and it seems to work.

2. I found what looks like a bug in sword: when playing with the cool new 
CEDICT module (http://www.web42.com/software/jianzidian/) I found out that 
opening some lexicons just hangs BT up -- endless loop. It hangs in this 
loop:

		(*module()) = TOP;  		
  		do {
   			m_entryList->append(QString::fromUtf8(module()->KeyText()));
  			(*module())++;
  		} while ( !module()->Error() );

Entries appear repeatedly. Can you confirm this? I shall attach a debug output 
of the keys obtained from the CEDICT module so that you can see the 
reoccurance of keys.

Martin

--

(BibleTime 1.3pre) WARNING: 'HOME TOWN CHICKEN' (A TYPE OF DISH)
(BibleTime 1.3pre) WARNING: 'OLD CHAP' (FORM OF ADDRESS BETWEEN MALE F
     [SKIPPED MANY ENTRIES]
(BibleTime 1.3pre) WARNING: (YEAR) A.D.
(BibleTime 1.3pre) WARNING: (YOU) HAD BETTER (DO WHAT WE SUGGEST)
(BibleTime 1.3pre) WARNING: (ZEN) BUDDHISM
(BibleTime 1.3pre) WARNING: (ZODIACAL SIGN)
(BibleTime 1.3pre) WARNING: )
(BibleTime 1.3pre) WARNING: ) AND DISCHARGE SURPLUS PERSONNEL
(BibleTime 1.3pre) WARNING: ) BREAK UP IN DISCORD
(BibleTime 1.3pre) WARNING: ) COME OFF
(BibleTime 1.3pre) WARNING: ) DISAPPEAR
(BibleTime 1.3pre) WARNING: ) FALL THROUGH AND STAND EXPOSED
(BibleTime 1.3pre) WARNING: ) FOR REFERENCE
(BibleTime 1.3pre) WARNING: ) GIVE SHADE
(BibleTime 1.3pre) WARNING: ) STAND LIKE A WALL
(BibleTime 1.3pre) WARNING: -D + INE (CHEM.)
(BibleTime 1.3pre) WARNING: -ER
(BibleTime 1.3pre) WARNING: -EST
(BibleTime 1.3pre) WARNING: -IAN
(BibleTime 1.3pre) WARNING: -ING (INDICATES AN ACTION IN PR
(BibleTime 1.3pre) WARNING: -ISM
(BibleTime 1.3pre) WARNING: -IST
(BibleTime 1.3pre) WARNING: -IST, -ER (PERSON)
(BibleTime 1.3pre) WARNING: -ITIS
(BibleTime 1.3pre) WARNING: -IZATION
(BibleTime 1.3pre) WARNING: -LESS
(BibleTime 1.3pre) WARNING: -LY
(BibleTime 1.3pre) WARNING: -OIN (CHEM.) AS IN ANISOIN
(BibleTime 1.3pre) WARNING: -OLOGY
(BibleTime 1.3pre) WARNING: -ORIENTED
(BibleTime 1.3pre) WARNING: -PLATED
(BibleTime 1.3pre) WARNING: -Z + OLE (CHEM.)
(BibleTime 1.3pre) WARNING: ... -IZE
(BibleTime 1.3pre) WARNING: ... AS WELL AS ...
(BibleTime 1.3pre) WARNING: ...OK?
(BibleTime 1.3pre) WARNING: ...RIGHT?
(BibleTime 1.3pre) WARNING: ...YEARS AGO
(BibleTime 1.3pre) WARNING: 0.6
(BibleTime 1.3pre) WARNING: 1
(BibleTime 1.3pre) WARNING: 1 00 MILLION
(BibleTime 1.3pre) WARNING: 1 5TH PARTY CONGRESS (OF CHINESE COMMUNIST PAR
(BibleTime 1.3pre) WARNING: 1 CHRONICLES
(BibleTime 1.3pre) WARNING: 1 CORINTHIANS
(BibleTime 1.3pre) WARNING: 1 JOHN
(BibleTime 1.3pre) WARNING: 1 KINGS
(BibleTime 1.3pre) WARNING: 1 PETER
(BibleTime 1.3pre) WARNING: 1 SAMUEL
(BibleTime 1.3pre) WARNING: 1 THESSALONIANS
(BibleTime 1.3pre) WARNING: 1 TIMOTHY
(BibleTime 1.3pre) WARNING: 1-3 A.M.
(BibleTime 1.3pre) WARNING: 1-3 P.M.
(BibleTime 1.3pre) WARNING: 10 OF A PECK
(BibleTime 1.3pre) WARNING: 10 PECKS
(BibleTime 1.3pre) WARNING: 10 PERCENT
(BibleTime 1.3pre) WARNING: 10,000
(BibleTime 1.3pre) WARNING: 100 MU
(BibleTime 1.3pre) WARNING: 100-METRE DASH
(BibleTime 1.3pre) WARNING: 1000 TONS
(BibleTime 1.3pre) WARNING: 105 BILLION
(BibleTime 1.3pre) WARNING: 10TH EARTHLY BRANCH
(BibleTime 1.3pre) WARNING: 10TH HEAVENLY STEM
(BibleTime 1.3pre) WARNING: 11 A.M.-1 P.M.
(BibleTime 1.3pre) WARNING: 11 P.M.-1 A.M.
(BibleTime 1.3pre) WARNING: 11TH EARTHLY BRANCH
(BibleTime 1.3pre) WARNING: 12
(BibleTime 1.3pre) WARNING: 1 00 MILLION
(BibleTime 1.3pre) WARNING: 1 5TH PARTY CONGRESS (OF CHINESE COMMUNIST PAR
(BibleTime 1.3pre) WARNING: 1 CHRONICLES
(BibleTime 1.3pre) WARNING: 1 CORINTHIANS
(BibleTime 1.3pre) WARNING: 1 JOHN
(BibleTime 1.3pre) WARNING: 1 KINGS
(BibleTime 1.3pre) WARNING: 1 PETER
(BibleTime 1.3pre) WARNING: 1 SAMUEL
(BibleTime 1.3pre) WARNING: 1 THESSALONIANS
(BibleTime 1.3pre) WARNING: 1 TIMOTHY
(BibleTime 1.3pre) WARNING: 1-3 A.M.
(BibleTime 1.3pre) WARNING: 1-3 P.M.
(BibleTime 1.3pre) WARNING: 10 OF A PECK
(BibleTime 1.3pre) WARNING: 10 PECKS
(BibleTime 1.3pre) WARNING: 10 PERCENT
(BibleTime 1.3pre) WARNING: 10,000
(BibleTime 1.3pre) WARNING: 100 MU
(BibleTime 1.3pre) WARNING: 100-METRE DASH
(BibleTime 1.3pre) WARNING: 1000 TONS
(BibleTime 1.3pre) WARNING: 105 BILLION
(BibleTime 1.3pre) WARNING: 10TH EARTHLY BRANCH
(BibleTime 1.3pre) WARNING: 10TH HEAVENLY STEM
(BibleTime 1.3pre) WARNING: 11 A.M.-1 P.M.
(BibleTime 1.3pre) WARNING: 11 P.M.-1 A.M.
(BibleTime 1.3pre) WARNING: 11TH EARTHLY BRANCH
(BibleTime 1.3pre) WARNING: 12
(BibleTime 1.3pre) WARNING: 1 00 MILLION
(BibleTime 1.3pre) WARNING: 1 5TH PARTY CONGRESS (OF CHINESE COMMUNIST PAR
(BibleTime 1.3pre) WARNING: 1 CHRONICLES