[sword-devel] optimizations

David White sword-devel@crosswire.org
14 Feb 2002 21:19:49 +1100


When I have a little more time, I will look over this in more detail,
but an obvious small change to me is the function
SWModule::filterBuffer. It contains the following code:

	FilterList::iterator it;
	for (it = filters->begin(); it != filters->end(); it++) {
		(*it)->ProcessText(buf, size, key);
	}

I suggest changing this to:

FilterList::iterator i1 = filter->begin;
const FilterList::iterator i2 = filter->end();
for(; i1 != i2; ++i1) {
	(*i1)->ProcessText(buf, size, key);
}

although, I think on a decent optimising compiler it shouldn't make a
difference. (Except the call to pre-inc. instead of post-inc. on the
iterator, which would likely still make a difference but didn't show up
on the profiling results as being terribly substantial).

It will only make a difference of a couple of percent at best, but it is
something obvious I saw...

Blessings,

David.

On Thu, 2002-02-14 at 13:40, Daniel Glassey wrote:
> On Wednesday 13 Feb 2002 11:48 pm, you wrote:
> > Then I will start an
> > optimization pass.  Seems we really need this as our .4 second, complete
> > bible search is up to about 2.9 seconds on my 1.13GHz laptop :)  We'll
> > get it back down, Lord willing.
> 
> Hi,
> I've attached a file showing the main culprits in a profile of mgrtest (with 
> functioncheck 1.4 www.sf.net/projects/fnccheck) with an added loop of 10 
> seaches for "in the beginning" for each of 4 modules. The profiling skews 
> things a bit by making calls to profiled functions relatively expensive, but 
> it still looks nasty.
> 
> A large amount of time is spent applying filters in StripText - 
> SWModule::filterBuffer. I don't know where to start with this but it really 
> needs to be got down. Are all the optionfilters and stripfilters really 
> needed or could there be just one filter that could be applied to strip the 
> text?
> 
> Another thing that takes a long time is the Index function which is called on 
> swmodule++	
> 
> hth,
> Daniel
> ----
> 

> Flat profile:
> 
> |      local      |      total      |          |           |
> |      sec. |  %  |      sec. |  %  | calls    |t. sec/call| name
> |-----------|-----|-----------|-----|----------|-----------|--------
> |   0.005565|  0.0| 184.190304| 97.8|         1| 184.190304| main
> |   6.514801|  3.5| 184.041110| 97.7|        40|   4.601028| SWModule::Search(char const *, int, int, SWKey *, bool *, void (*)(char, void *), void *)
> |   3.747166|  2.0| 111.277097| 59.1|    966940|   0.000115| SWModule::StripText(char *, int)
> |  20.721672| 11.0|  64.814857| 34.4|   3245702|   0.000020| SWModule::filterBuffer(list<SWFilter *, allocator<SWFilter *> > *, char *, long, SWKey *)
> |   2.960891|  1.6|  64.210781| 34.1|    966944|   0.000066| SWModule::operator char *(void)
> |   1.386309|  0.7|  63.396991| 33.7|    966940|   0.000066| SWModule::operator++(int)
> |   0.000039|  0.0|  62.034964| 32.9|        10|   6.203496| RawText::Search(char const *, int, int, SWKey *, bool *, void (*)(char, void *), void *)
> |   1.730713|  0.9|  49.825425| 26.5|   1244080|   0.000040| SWKey::operator+=(int)
> |   3.089347|  1.6|  48.094712| 25.5|   1244080|   0.000039| VerseKey::increment(int)
> |   8.106054|  4.3|  44.019000| 23.4|   1294280|   0.000034| VerseKey::Index(long)
> |   2.021123|  1.1|  40.069383| 21.3|    933060|   0.000043| SWModule::operator+=(int)
> |   2.331246|  1.2|  40.040147| 21.3|   1622833|   0.000025| SWModule::optionFilter(char *, long, SWKey *)
> |  17.975001|  9.5|  31.020983| 16.5|   2589096|   0.000012| VerseKey::_compare(VerseKey const &)
> |   2.466222|  1.3|  21.941299| 11.7|     33880|   0.000648| RawCom::operator+=(int)
> |   1.449787|  0.8|  21.262926| 11.3|    966940|   0.000022| SWModule::stripFilter(char *, long, SWKey *)
> |   1.067892|  0.6|  19.668616| 10.4|    655893|   0.000030| SWModule::RenderText(char *, int)
> |   3.120428|  1.7|  18.604349|  9.9|    311021|   0.000060| zText::getRawEntry(void)
> |   3.460448|  1.8|  15.703311|  8.3|    311021|   0.000050| RawText::getRawEntry(void)
> |   0.411379|  0.2|  12.802970|  6.8|    311020|   0.000041| SWKey::operator++(int)
> |   7.113064|  3.8|  10.378800|  5.5|   5111829|   0.000002| list<SWFilter *, allocator<SWFilter *> >::end(void)
> |   5.553859|  2.9|   9.080109|  4.8|   1866107|   0.000005| _List_iterator<SWFilter *, SWFilter *&, SWFilter **>::operator++(int)
> |   0.873697|  0.5|   8.166466|  4.3|    655923|   0.000012| SWModule::rawFilter(char *, long, SWKey *)
> |   7.867714|  4.2|   7.867714|  4.2|   1244052|   0.000006| GBFFootnotes::ProcessText(char *, int, SWKey const *, SWModule const *)
> |   5.746685|  3.1|   7.781893|  4.1|   1000813|   0.000008| RawVerse::findoffset(char, long, long *, unsigned short *)
> |   4.773375|  2.5|   6.893648|  3.7|   3245703|   0.000002| list<SWFilter *, allocator<SWFilter *> >::begin(void)
> |   3.716635|  2.0|   6.782717|  3.6|    311021|   0.000022| zVerse::findoffset(char, long, long *, unsigned short *)
> |   5.386023|  2.9|   5.386023|  2.9|   8357551|   0.000001| _List_iterator<SWFilter *, SWFilter *&, SWFilter **>::_List_iterator(_List_node<SWFilter *> *)
> |   4.534373|  2.4|   4.534373|  2.4|   7146029|   0.000001| VerseKey::Testament(void) const
> |   1.706959|  0.9|   3.789286|  2.0|    311021|   0.000012| RawFiles::getRawEntry(void)
> |   3.705156|  2.0|   3.705821|  2.0|   5663979|   0.000001| FileDesc::getFd(void)
> |   3.472051|  1.8|   3.472051|  1.8|    622040|   0.000006| GBFPlain::ProcessText(char *, int, SWKey const *, SWModule const *)
> |   3.425490|  1.8|   3.425490|  1.8|   4867294|   0.000001| VerseKey::Index(void) const
> |   3.298542|  1.8|   3.298542|  1.8|   5178272|   0.000001| VerseKey::Verse(void) const
> |   3.269110|  1.7|   3.269110|  1.7|   5178272|   0.000001| VerseKey::Book(void) const
> |   3.215319|  1.7|   3.215319|  1.7|   5178356|   0.000001| VerseKey::Chapter(void) const
> |   1.978193|  1.1|   3.127984|  1.7|   1311884|   0.000002| SWClass::isAssignableFrom(char const *) const
> |   3.124996|  1.7|   3.124996|  1.7|   5111810|   0.000001| _List_iterator<SWFilter *, SWFilter *&, SWFilter **>::operator!=(_List_iterator<SWFilter *, SWFilter *&, SWFilter **> const &) const
> |   2.322500|  1.2|   2.976985|  1.6|    344902|   0.000009| RawVerse::gettext(char, long, unsigned short, char *)
> |   0.469765|  0.2|   2.849019|  1.5|     33881|   0.000084| RawCom::getRawEntry(void)
> |   2.316283|  1.2|   2.316283|  1.2|    344902|   0.000007| RawVerse::preptext(char *)
> |   2.313358|  1.2|   2.313358|  1.2|   3732214|   0.000001| _List_iterator<SWFilter *, SWFilter *&, SWFilter **>::_List_iterator(_List_iterator<SWFilter *, SWFilter *&, SWFilter **> const &)
> |   0.000094|  0.0|   2.309578|  1.2|        22|   0.104981| SWConfig::SWConfig(char const *)
> |   0.033848|  0.0|   2.307988|  1.2|        22|   0.104909| SWConfig::Load(void)
> |   2.273740|  1.2|   2.273740|  1.2|   2588560|   0.000001| VerseKey::findindex(long *, int, long)
> |   0.000003|  0.0|   2.260472|  1.2|         1|   2.260472| global constructors keyed to LocaleMgr::LocaleMgr(char const *)
> |   0.000009|  0.0|   2.260469|  1.2|         1|   2.260469| __static_initialization_and_destruction_0
> |   0.000032|  0.0|   2.260460|  1.2|         1|   2.260460| LocaleMgr::LocaleMgr(char const *)
> |   0.000321|  0.0|   2.258041|  1.2|         1|   2.258041| LocaleMgr::loadConfigDir(char const *)
> |   0.000237|  0.0|   2.254065|  1.2|         8|   0.281758| SWLocale::SWLocale(char const *)
> |   2.114444|  1.1|   2.114444|  1.1|   3245702|   0.000001| _List_iterator<SWFilter *, SWFilter *&, SWFilter **>::_List_iterator(void)
> |   1.421993|  0.8|   2.082239|  1.1|    966940|   0.000002| VerseKey::NewIndex(void) const
> |   0.002928|  0.0|   2.003265|  1.1|      1322|   0.001515| SWCompress::Buf(char const *, unsigned long *)
> |   1.692117|  0.9|   2.000337|  1.1|       661|   0.003026| ZipCompress::Decode(void)
> |   1.941214|  1.0|   1.941214|  1.0|    311021|   0.000006| zVerse::preptext(char *)
> |   0.681688|  0.4|   1.261219|  0.7|    312202|   0.000004| SWKey::SWKey(SWKey const &)
> |   1.212892|  0.6|   1.212892|  0.6|   1866107|   0.000001| _List_iterator<SWFilter *, SWFilter *&, SWFilter **>::operator++(void)
> |   1.161573|  0.6|   1.161573|  0.6|   1866107|   0.000001| _List_iterator<SWFilter *, SWFilter *&, SWFilter **>::operator*(void) const
> |   1.150113|  0.6|   1.150113|  0.6|   1312354|   0.000001| stricmp(char const *, char const *)