Hi Greg,<br><br><div class="gmail_quote">On Sat, Nov 6, 2010 at 12:24 PM, Greg Hellings <span dir="ltr"><<a href="mailto:greg.hellings@gmail.com">greg.hellings@gmail.com</a>></span> wrote:<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
But the nature of multimap is different - and I'm confused as to how<br>
it is used at all. What you are showing above I would call a<br>
multi-dimensional associative array/dictionary. I make them all the<br>
time in Python and PHP. But a multimap is defined as "a<br>
generalization of a map or associative array abstract data type in<br>
which more than one value may be associated with and returned for a<br>
given key." I don't think we are actually using multimap<br>
functionality but rather just a multidimensional array.<br></blockquote><div>If I remember rightly, a multimap is just like a dictionary in python that can have multiple values for a given key (conceptually I guess it is like a dictionary of lists of something).</div>
<div>The getEntryAttributes stuff is not a multimap - just nested maps.</div><div><br></div><div>Multimaps are seen in SWConfig, where it is possible to have multiple entries for a given key (e.g. in modules, multiple GlobalOptionFilter's).</div>
<div><br></div><div>This allows you to iterate through them like this:</div><div>(annoyingly long fragment adapted from C++, untouched for a year probably...)</div><div><br></div><div># look first in the map</div><div><div>
<span class="Apple-style-span" style="white-space: pre;">sourcesSection = installConf.getSections().find(Sword.SWBuf("Sources"))
if sourcesSection != installConf.getSections().end():
ftp_source = Sword.SWBuf("FTPSource")
ss = sourcesSection.value()[1]
</span></div><div><span class="Apple-style-span" style="white-space: pre;"><meta charset="utf-8"> # this is where we start iterating over the multimap
sourceBegin = ss.lower_bound(ftp_source)
sourceEnd = ss.upper_bound(ftp_source)
while sourceBegin != sourceEnd:
install_source = InstallSource("FTP",
sourceBegin.value()[1].c_str())
print privatePath + "/" + install_source.source</span></div><div><span class="Apple-style-span" style="white-space: pre;">
sourceBegin += 1
</span></div></div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
SWIG gives us multimap support in Python for free and I found a<br>
version we include with the library to get Perl working. However,<br>
SWIG does not have an implementation of stl::multimap for us, so we<br>
either need to write one, or refactor our multimap usage to be nested<br>
associative arrays.<br></blockquote><div> Most people wouldn't need to touch multimap - I think it's only used in SWConfig, where mostly you can just use get and set on the SWConfig if needed (only if you need to iterate over multiple key values). So I don't think this should worry us too much. To some extent, if an area of the API isn't supported for a given language, there's no point in supporting it until someone needs it.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
><br>
> But this interface doesn't translate well to other languages. I have<br>
> had to implement a different interface for my custom language binding<br>
> for Java-jni (used by SWORD C++ on Android) and CORBA (used by SWORDWeb).<br>
><br>
> The binding interface I usually implement is described in simple IDL<br>
> here and is a fairly concise set of methods that give access to most all<br>
> of functionality of the engine:<br>
><br>
> <a href="http://crosswire.org/svn/sword/trunk/bindings/corba/swordorb.idl" target="_blank">http://crosswire.org/svn/sword/trunk/bindings/corba/swordorb.idl</a><br>
><br>
> You'll notice the method for obtaining entry attributes:<br>
><br>
> StringList SWModule.getEntryAttribute(in string level1, in string<br>
> level2, in string level3, in boolean filtered);<br>
><br>
> This less exotic interface maps much easier to other languages, but<br>
> implies you know what attribute you'd like to obtain (discovery by<br>
> iteration is not supported, but really, what frontend does that anyway?)<br></blockquote><div>That might be a better way of doing it, and would be very easy to add. </div><div>In BPBible we don't exactly iterate over it, but we do get values at different levels and check whether subvalues exist (look at: <a href="http://code.google.com/p/bpbible/source/browse/trunk/backend/book.py#415">http://code.google.com/p/bpbible/source/browse/trunk/backend/book.py#415</a></div>
<div>). This we could replace with a simpler interface in getEntryAttribute (which could return None if key not found).</div><div><br></div><div>I can imagine a situation where someone wanted to do iteration, but we don't seem to do that in BPBible at the moment...</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
><br>
> So, in summary, maybe we should add this to swmodule.i<br>
<br>
That is one possibility. However, I think there are other places<br>
within the bindings that should be addressed first. As you saw<br>
yourself, SWBuf plays great in C++ but SWIG doesn't seem to pick up<br>
any ability to auto-cast it back to a Python string/unicode object.<br>
At the very least, that functionality should be as easy as an explicit<br>
cast of unicode(myswbuf) and better yet it would be transparent.<br></blockquote><div>I tried doing that with a simple SWIG typemap, but it didn't play well with the STL stuff. Almost everything worked as expected, but when a SWBuf was a key in a std::map for example, it didn't transparently convert that, and there was no way to create SWBuf's then. </div>
<div>Occasionally it is useful not to transparently convert them but to look at the length or something like that for efficiency (not very common I think...)</div><div> </div><div>Depending on how smart you want to try to make it, SWBuf's have to end up as byte strings as well quite a bit of the time. There's no universal way from the API to tell whether something is utf-8 or cp1252. You could do it for a number of key interfaces, though - RenderText, getKeyText, get config entry from module.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Ideally, someone would write extra Python code so that the SWIG<br>
bindings were wrapped in a more naturally Pythonic binding. It might<br>
be perfectly natural in a language like C++ or Java to access a module<br>
with<br>
<br>
SWMgr* mgr = new SWMgr(FMT_HTMLHREF);<br>
SWModule* mod = mgr->getModule("KJV");<br>
VerseKey* key = (VerseKey*) mod->getKey();<br>
key->persist(true);<br>
key = 'john 3:1';<br>
cout << mod->RenderText();<br>
<br>
A natural pythonic way of doing this would probably look more like<br>
<br>
mod = Sword.module('KJV')<br>
print mod('john 3:1', format='HTMLHREF')<br></blockquote><div>You could easily put a simple python wrapper around Sword which did this, but this wouldn't easily cover everything. From my perspective:</div>
<div>1) SWMgr is very necessary to expose for BPBible at least. Probably not in a simple API though.</div><div>2) I don't like that function style calling, and I wouldn't say it is pythonic (not that I necessarily am a good judge of that). A method would work better I think - say text or get_text, but taking a string.</div>
<div>3) Passing in format like you did won't (easily) work as the format is a property of the SWMgr. Unless you keep a SWMgr per format, and look them up and create them when needed. Probably defaulting to HTMLHREF is correct, and then having a module level set format to call which creates a new SWMgr.</div>
<div><br></div><div>Thos problems aside, a simple python wrapper could create a much lower barrier to entry. Personally, I don't care too much about other languages than Python. </div><div><br></div><div>To some extent, I've simplified working with some of the SWORD stuff and done extra checking and whatnot in BPBible, mostly in the file below:</div>
<div><a href="http://code.google.com/p/bpbible/source/browse/trunk/swlib/pysw.py">http://code.google.com/p/bpbible/source/browse/trunk/swlib/pysw.py</a></div><div><br></div><div>This file does quite a bit, mostly with (Verse, Tree, SW, List)Keys. Keys will be the hardest thing to make work nicely, I think. In BPBible we do lots of special stuff with keys, for example formatting ranges nicely, keeping user input segregated from english keys, etc.</div>
<div>The stuff in this file is reasonably tied into BPBible, and so isn't usable as-is elsewhere.</div><div><br></div><div>An example of how this could look: </div><div>mod = Sword.module("KJV")</div><div>print mod.get_text("John 3:16")</div>
<div>print mod.get_text("John 3")</div><div>for verse, text in mod.chapter("John 3"):</div><div> print verse, text</div><div><br></div><div>for reference, text in mod.range("John 3:1-6"):</div>
<div> print reference, text</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
But in order to get to that point would take someone writing<br>
native-feeling Pythonic bindings and making design decisions like<br>
that. I'm tied up with other tasks at the moment and can't devote the<br>
time to learning SWIG to clean up the warnings and errors the bindings<br>
throw on compile PLUS write native feeling bindings for languages.<br>
But it'd be a great thing for someone to tackle if they really like<br>
their scripting languages and probably wouldn't require learning much<br>
(if any) C++.<br></blockquote><div>Trying to work with SWIG at all is likely to end up with using C++. If doing any stuff with stl, it is really needed, and even still very hard to get working (I did all the stl map/vector stuff, and it was painful and fragile). A simple wrapper API in python would require understanding SWORD, but possibly not too much C++ unless more SWIG binding needed writing.</div>
<div><br></div><div>As to the warnings from SWIG, some I tried to set SWIG to ignore, but it wouldn't. So I gave up. Others are possibly valid concerns, but I wasn't sure how to handle them. Writing SWIG bindings is painful, in case anyone didn't realise... :-)</div>
<meta charset="utf-8"><div><br></div><div>The bottom line is that SWORD is a complex API, but for a full-featured frontend, a lot of that complexity is necessary.</div><meta charset="utf-8"><br clear="all">God Bless,<br>
Ben<br>
-------------------------------------------------------------------------------------------<br>Multitudes, multitudes,<br> in the valley of decision!<br>For the day of the LORD is near<br> in the valley of decision.<br>
<br><div>Giôên 3:14 (ESV) </div></div>