[sword-devel] Ruby GEM and Rails plugin for the Sword API
Bill Burton
bburton at mail.com
Fri Nov 10 14:07:39 MST 2006
Hello Troy,
On 11/9/06, Troy A. Griffitts <scribe at crosswire.org> wrote:
>
> Hey guys,
> The flatapi.cpp file hasn't been updated for a bit (as you
> mentioned
> you had to patch it slightly to build).
Yes, that was to build a C program but patching wasn't required to use the
Ruby/DL interface ... except that somehow between Sword 1.5.8 and 1.5.9
libsword.so stopped being linked with stdc++. The result was that as Ruby
isn't linked with stdc++ either, there were unsatisfied dependencies. After
looking, I couldn't find out what caused that change so I manually changed
the Sword Makefile to link with stdc++ again and now Ruby/DL works again.
The CORBA interface is what we use with the current web frontend at
> (for example):
>
>
> http://crosswire.org/study/parallelstudy.jsp?del=all&add=KJV&add=TR&add=NASB&add=WHAC
>
> (click on a word in the text-- it's fun)
Yes, I was playing with it the other day.
The IDL can be found here:
>
> http://crosswire.org/svn/sword/trunk/bindings/corba/swordorb.idl
>
> Here seems to be a Ruby IDL compiler:
>
> http://sourceforge.net/projects/rinn/
Yes. That was my original attempt--building the ORB so I could run the
Bible Tool. However, I was completely unsuccessful as the ORB on RedHat ES
3 was too old and the dependencies for the newer ORB required by Sword
required building all kinds of other dependencies so I gave up. I'd like to
upgrade to a more modern Linux distribution but that's not an option now.
Maybe Nathan could get the ORB built and working on his OS but it's just not
an option for me at this time which is why I'd rather work on Ruby/DL or
Ruby SWIG interfaces.
This should allow you to talk with the C++ engine. Some simple Java
> examples using the generated Java from the java idl compiler are here:
>
> http://crosswire.org/study/examples/
>
> (I realize these are Java and not Ruby, but I'm guessing the generated
> Ruby classes from the same IDL would have similar usage)
>
> The bindings released in 1.5.9 were a complete rewrite contributed by a
> .NET project. You can read about the project from this post from the
> list archives. Jason Turner might be able to help you with his ideas
> for the current bindings.
>
> http://www.crosswire.org/pipermail/sword-devel/2006-August/024296.html
Thanks for pointing that out.
I'm also looking at the SWIG bindings and am having success with Ruby.
Since it's possible to conditionally define settings depending on the
language, e.g. #ifdef SWIGRUBY, I've been customizing the Ruby bindings
without affecting the other languages. For instance instead of the method
"isWriteable" in Ruby it's now "writable?" following the Ruby convention of
ending a method name with "?" if it returns a boolean, or, for setting the
key text, instead of mod.setKey("Psalm 150:1") you can do:
mod.key_text = "Psalm 150:1"
SWIG supports generating API docs from the .i files but because each .h file
is included, it's not possible to do this. Even if the API docs were
specified in the .i files, I don't know how well that would work for
different languages which all have different interfaces. I suspect what
I'll end up doing is creating Ruby classes that are stubs of the generated
SWIG interface and then adding documentation to them in Ruby's rdoc format.
Concerning threading. Since SWORDWeb uses the CORBA interface, it runs
> one orb (binary server) for each user session, so there is no
> multithreaded issues (mostly. I bet one might be able to find something
> if they played with 2 browser windows sharing the same session. Not
> encouraging anyone to do so, however :) ).
>
> Since SWORD is a crossplatform engine, we have to be careful what we
> have in the engine itself. There is no direct threading facilities in
> SWORD; however, threading was considered and the engine has been
> adjusted to help users of the engine deal with multithreaded issues.
>
> Basically, you need to understand which objects have state (you have
> already mentioned SWMgr and SWModule). These either need to have their
> own instances per thread, or else have synchronize blocks around usage.
> For example, it might be advantageous to have one SWMgr (and SWModule
> set therein) for display, and another for searching. Each user session
> for a web application might have their own SWMgr. The engine is
> intelligent enough not to ACTUALLY create duplicates of all resources,
> like open file handles-- as an example. When a module tries to open,
> say 7 files for accessing its data, and you have 100+ modules installed,
> and then you create duplicate SWMgr classes, each allocating 100+
> modules, you can see how we needed to have an intelligent class to
> really NOT open 1000s of files. You can tune FileMgr to give it maximum
> real open files and it should obey. But mostly you don't have to worry
> about it unless you find yourself needing to tune things.
>
> I thought we added .clone() methods to both SWMgr and SWModule to help
> with multithreading, but I think I remember not seeing them last time I
> went to use them. Not sure. Let me know if you'd like anything added
> to help in your efforts.
Thanks for the detailed explanation. I also spent some time looking an
SWORDWeb and the supporting code for the CORBA server.
As for the API, the generated docs are available but not much help
> usually. The API Primer is the best start of a primer that we have, but
> hasn't been updated for a while.
Yes. I noticed it was using SetKey (which is deprecated) instead of setKey
for specifying a verse reference to a module. Notheless, it's still
helpful.
We have a few examples in the source
> under sword/examples/, and a really good place to try things out are the
> numerous sword/tests/ and sword/utilities/, though they are very
> eclectic-- but serve as good examples, as well. Of course there are the
> numerous frontends themselves, and finally sword/include/
I haven't seen the SVN or CVS for the front ends. What are they called?
When I last checked, I couldn't browse at the /svn level or maybe I could
see them.
I would be more than grateful to have additions to sword/examples/
Wrote a simple C program to experiment with the different iterators in the
flatapi so I could then write a Ruby/DL version. It would be good to have at
least one C program example as part of the build or a test to ensure that
interface isn't broken again.
Finally, if you get anything working and want to start tracking changes
> in source control, we would love have a ruby section under
> sword/bindings/ with your work and READMEs and examples and anything
> else you think helpful. Please let us know and we will get you access
> to our source repository.
If this Ruby/DL version pans out, creating a rubydl folder would probably
the most appropriate place for it. As the Ruby SWIG version is looking
promising, I'd like to contribute any changes to support it--Lord willing.
Thanks again for considering using the engine. I realize you probably
> haven't made a firm decision yet, but any info you find, or work you
> develop, I'm sure will be helpful to the next person wanting to do Ruby
> development with SWORD.
Yes, very definitely.
Warmest Christian greetings,
>
> -Troy.
In His Service,
-Bill
lumin8 wrote:
> >
> > Also, right now, there's a limitation with the Sword C flatapi API
> > in that there's limited metadata available for Bible modules. What
> > I mean is you:
> > * Can't get a list of all the books in a Bible.
> > * Given a book name, can't get the number of chapters in the book.
> > * Given a book name and chapter, can't get the number of verses in
> > that chapter.
> >
> > If this information is available in the C++ classes, then it would
> > be worthwhile exposing in the C flatapi.
> >
> >
> > Does anyone else on this list know the answer to this question? Also,
> > where is the documentation for the Sword API - what are the features? I
> > must be overlooking something.
> >
> > I was thinking of eventually creating a GEM so it could be easily
> > installed and used in any Ruby application.
> >
> >
> > Nice, I would be glad to help with a GEM and especially a Rails plugin.
> > Would it be beneficial if I set up a SVN repository for this code? Or
> > maybe you have one available already for it.
> >
> > I'll try to clean up the module and send it to you so you can start
> > experimenting with it.
> >
> >
> > Looking forward to it. By the way, I was able to get Sword compiled,
> > and did some looking around and experimenting with things. I also
> > found these two:
> > Ruby/DL - http://ttsky.net/ruby/ruby-dl.html
> > Ruby/DLX = http://ruby-dlx.rubyforge.org/index.xhtml
> >
> > Are you using one of these for the flatapi?
> >
> > -=nathan
> >
> > On 11/8/06, *Bill Burton* <bburton at mail.com <mailto:bburton at mail.com>>
> > wrote:
> >
> > Hello Nathan,
> >
> > On 11/8/06, * lumin8* <lumi.n8 at gmail.com <mailto:lumi.n8 at gmail.com>>
> > wrote:
> >
> > Bill,
> >
> > Yes exactly. Sounds like you are doing some great work with
> this.
> > I'm trying to figure out how I can help - at the least I could
> > package
> > it as a rails plugin or gem making it easy for any ruby on rails
> > developer to get started with it.
> >
> >
> > I was thinking of eventually creating a GEM so it could be easily
> > installed and used in any Ruby application.
> >
> > However, I think a Rails plug-in would be a good idea to provide any
> > extra support that makes sense specific to a Rails application. At
> > the very least, providing a default controller implementation and/or
> > helper methods that could be used in other controllers would be
> > worthwhile. The ActionService web service client support might be a
> > good pattern to start with.
> >
> > Also, right now, there's a limitation with the Sword C flatapi API
> > in that there's limited metadata available for Bible modules. What
> > I mean is you:
> > * Can't get a list of all the books in a Bible.
> > * Given a book name, can't get the number of chapters in the book.
> > * Given a book name and chapter, can't get the number of verses in
> > that chapter.
> >
> > If this information is available in the C++ classes, then it would
> > be worthwhile exposing in the C flatapi.
> >
> > So in the meantime, an application using the C flatapi has to
> > provide it's own support to maintain this metadata. As
> > versifications are different in some translations, you can't
> > necessarily use the same information for all translations.
> >
> > In regards to creating a unique key to represent a Bible verse, it
> > may be helpful to create a utility class like VerseKey that can take
> > any verse reference with a variety of abbreviations and return it in
> > OSIS notation.
> >
> > Let me know if I can help with anything else.
> >
> >
> > I'll try to clean up the module and send it to you so you can start
> > experimenting with it.
> >
> > In His Service,
> > -Bill
> >
> > -=nathan
> >
> >
> > On 11/7/06, Bill Burton < bburton at mail.com
> > <mailto:bburton at mail.com>> wrote:
> > > Hello,
> > >
> > > I see. You want the module information in a database because
> > it's then
> > > trivial to program a web application using Ruby on Rails
> > ActiveRecord object
> > > relational mapping support. RoR also supports other logic in
> > controllers
> > > such as web services clients so it's entirely possible to
> write an
> > > application that calls Sword without using a database but the
> > scaffolding
> > > code generator won't be able to help much if at all.
> > >
> > > It just so happens I've been working on a Ruby interface to
> > Sword with a
> > > longer term goal of writing a Ruby on Rails application to
> > access it.
> > > Initially, I couldn't figure out how to build anything with
> > SWIG--there's no
> > > README in the bindings/swig directory. Then it happened I
> > discovered Ruby
> > > can load shared libraries and DLL's and dynamically bind to
> > them on the fly
> > > using the dynamic loader (DL) module. So I started to write
> a
> > Sword
> > > interface to the C flatapi in bindings/flatapi.cpp. In the
> > process, I found
> > > a few minor bugs with the flatapi but have created patches to
> > fix them
> > > (which I'll submit soon).
> > >
> > > Using this Ruby to Sword flatapi interface you can output
> > verses as follows
> > > (I'm doing this from memory so it's not exact):
> > >
> > > include Sword
> > > # SWMgr_new(FMT_HTML)
> > > Manager.new (Sword::FMT_HTML) do |mgr|
> > > module = mgr.get_module("KJV")
> > > # iteration using Sword API's under the covers to verseKey
> and
> > renderText
> > > module.each_verse_render("Psalm 133:1-3") do |verse, text|
> > > puts "#{verse} #{text}" # verse reference and text
> > > end
> > > end # performs an implicit SWMgr_delete
> > >
> > > So far, I've implemented about two-thirds of the Sword C
> > flatapi. However,
> > > the one issue I'm just starting to address is how to call
> > these API's within
> > > a multi-threaded multi-user environment such as a web
> > application. Because
> > > the SWMgr_new/newEx and SWModule_* functions have state, it
> > looks like a new
> > > Manager object will have to be created per user which means
> > establishing a
> > > session and then saving the Manager instance in the
> > session. So I have to
> > > refactor the code to allow multiple instances of a Manager
> > class without
> > > loading the shared library every time.
> > >
> > > But then last night I was about to send an email to this list
> > asking how to
> > > build the SWIG interface but I looked at it one more time and
> > discovered how
> > > to do it. With some more investigation I was then able to
> > generate SWIG
> > > bindings for Ruby and build the interface to Sword. So far
> > I've been able
> > > to access some of the methods from the SWMgr and SWModule
> > classes but can't
> > > output a verse because the binding of set_key to
> > SWModule.SetKey is
> > > incorrect. This may be a bug in the way SWIG generated the
> > binding but I
> > > don't know yet.
> > >
> > > The nice thing about the Ruby dynamic loader interface to
> > Sword is there's
> > > nothing extra to build which makes it much easier to install
> > as compared
> > > with the SWIG version. However, the C flatapi on which it's
> > based limits
> > > one to getting and setting global options, iterating over
> > modules and
> > > rendering verses. The SWIG interface to the C++ API's is
> much
> > more complete
> > > but has to be built. I don't know yet if the Sword C++ API's
> > also have state
> > > which is important to know for a web application.
> > >
> > > Right now, I'm developing on RedHat ES 3 but plan to test the
> > DL version on
> > > Windows against an installed BibleCS. My time is limited but
> > I'll try to
> > > get this binding in reasonable shape soon so it can be used
> > with Ruby on
> > > Rails. Building the SWIG version on Windows is not an option
> > for me at this
> > > time due to lack of tools and a dead machine.
> > >
> > > God is merciful,
> > > -Bill
> > >
> > > On 11/7/06, lumin8 < lumi.n8 at gmail.com
> > <mailto:lumi.n8 at gmail.com>> wrote:
> > > >
> > > > > I'll leave the question alone as to the value of a
> > relational database
> > > > > for this data over using the SWORD API.
> > > >
> > > >
> > > > Actually, I am interested in this question if you have time
> > / desire to
> > > enlighten me. I think I can manage the chunk of code you
> gave
> > me (thanks),
> > > but I have never compiled a program in my life. I have been
> > building web
> > > applications for about 7 years with PHP, and now Ruby for the
> > last year and
> > > a half.
> > > >
> > > > I am willing to learn the Sword API if I need to, but I can
> > build web
> > > applications very quickly with Ruby on Rails. The rest of
> my
> > data (user
> > > info, notes, user generated translations, a wiki like
> > interface for
> > > commentary) will all be in a relational database. Would
> there
> > be speed
> > > benefits from using the Sword API over the indexing provided
> > by mysql or
> > > postgre? What about application development time? In the
> > Rails framework,
> > > I hardly even have to write SQL, if I used the Sword API I
> > would have to
> > > learn a bit about c++ and bindings to Ruby... I am willing,
> > but what is the
> > > advantage?
> > > >
> > > > By the way, I have tried and use many of the Sword front
> > ends and highly
> > > respect the work you all are doing.
> > > >
> > >
> >
> >
> > _______________________________________________
> > sword-devel mailing list: sword-devel at crosswire.org
> > <mailto:sword-devel at crosswire.org>
> > http://www.crosswire.org/mailman/listinfo/sword-devel
> > Instructions to unsubscribe/change your settings at above page
> >
> >
> >
> >
> > --
> > _______________________________
> >
> > portland, or: 01 (503) 608-7950
> > cuernavaca, mx: 52 (777) 318-9094
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > sword-devel mailing list: sword-devel at crosswire.org
> > http://www.crosswire.org/mailman/listinfo/sword-devel
> > Instructions to unsubscribe/change your settings at above page
>
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
>
--
Bill Burton <bburton at mail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.crosswire.org/pipermail/sword-devel/attachments/20061110/ef7a4c4b/attachment-0001.html
More information about the sword-devel
mailing list