[bt-devel] BibleTime's Goals (was Re: New passage selector proposal)

[Patrick Zimmermann]

Hello,

I don't know Sword code at all and only very roughly know the Bibletime code.
Also I have less programming experience than most others here.

All the following is my personal opinion. I don't want to stir up a Bibletime 
vs Sword fight or something similar to that. I do want to understand the 
situation there currently is to know how to best progress from where we are.
In the following I try to present how I understand the situation. Hopefullly 
someone can correct me where my understanding is wrong.


To continue with my work in changing the GUI of Bibletime I was encouraged to 
refactor Bibletime in large parts. I can't speak for the other Bibletime 
developers, but I think that putting logic that Sword already provides back 
upstream is a good thing.

Overhauling the Bibletime code in one big rush to remove the Sword wrapper is 
impossible. There is not enough manpower available to do this. So the only way 
forward is an iterative approach.

Several features or future wishes are relying on Bibletime code that could, 
should or are already provided by Sword. I try to go through them in the 
following.

----
ICU and ftp. I don't know how enwoven QUnicode is with the rest of Bibletime, 
but both, ICU and ftp are not part of the core of Sword but auxiliary 
libraries provided for convenience. So I don't see a problem with using the Qt 
replacements, because the Qt ones might be easier to use in a Qt project.
In addition ICU and ftp are optional when compiling Sword. So when relying on 
Sword alone the only option to guarantee that internationalization and ftp 
work is by packaging a custom Sword with Bibletime. I think it will be hard to 
force this onto package providers.
-----
I think it's not in the scope of Sword to support arbitrary input formats. 
Sword does support several input formats, but all heavily preprocessed, so it 
boils down to Sword supporting several different *Sword internal* formats only.
I understand this is a lock-in situation that is hard to change, because of 
the large stock of existing modules there already are.

Changing Sword to also support native file formats directly (like OSIS or 
Perseus) seems out of scope of Sword, given that the Sword approach has always 
been to preprocess input formats to conform to the Sword standard.

Since there is neither enough manpower to do anything about this strange 
situation nor a plan how to proceed, I would just leave this topic for now.
-----
The filters there are in Bibletime seem to be duplicated from the Sword ones. 
On the following wiki page support of different filter features is distinguished 
between the different frontends. So it seems not only Bibletime is 
duplicating/overriding the filtering logic.
http://www.crosswire.org/wiki/OSIS_Bibles/BSPExample

I assume Sword does not provide all features Bibletime requires. To fix this 
situation one would probably have to change the Sword codebase to support the 
different features.

If I judge the situation correctly changes to the Sword codebase are accepted 
very conservatively. I guess this is the number one reason for duplicating 
features in the frontends in the first place.

Is there a possibility that this changes and larger commits and refactorings 
are accepted in Sword in the process of moving code from Bibletime to Sword?
-----
A question at the end:

There currently are several different Sword implementations:
-original Sword library in C++
-JSword in Java
-XUL Sword in C++

Doesn't that mean, that the core of the Sword project isn't a specific 
implementation, but a specification how the Sword modules look and everything 
that can process those modules is in fact an implementation of this 
specification?
-----

I hope someone takes the time to clarify some of these statements and 
questions above. I would really like to understand the relation of all the 
different components, what should be implemented where and what the reasons are 
for the code duplication. I don't want to start a refactoring of Bibletime  
and/or Sword without understanding the reasons that lead to the situation we 
have now.

Blessings,
Patrick Zimmermann


On Friday, 20. April 2012 07:18:15 Troy A. Griffitts wrote:
> So, in summary, you don't consider yourselves part of the CrossWire
> developer community? None of the things you've mentioned has been realized
> in 20 years, SWORD is practically a direct dependency, and if you would
> like access to Perseus, wouldn't it be better to put in the effort to make
> Perseus (and SWORD data) available through a well thoughtout Bible
> research framework instead of your patchwork backend you've lived with for
> 20 years because of 'in principle' arguments? SWORD is designed as a
> backend for Bible research software, with a modular architecture (as seen
> by your many reimplementations; just because you _can_ do something
> doesn't necessarily mean you _should_) allowing plugins to access
> different data sources and in different markups. If you think you can
> design a better class heirarchy, you may be right, but why? Is SWORD
> really so different than what you would design yourselves as a backend? Is
> it always 'do it ourselves', and 'differentiate from the other CrossWire
> projects' or are you willing to share your improvements with the community
> as you have used our work for the entire life of your project?
> 
> > Just a quick comment. I say this once every 3 years or so, but...
> > 
> > Of course my opinion is that you are making a huge mistake trying to
> > abstract the SWORD code away and not use the classes directly in your
> > frontend.
> > 
> > The classes in SWORD were written to facilitate quick and easy frontend
> > development. As Jaak notes, your abstractions are not well organize. BT
> > seems to chronically reimplement SWORD functionality for no real benefit,
> > e.g.,
> > BTStringMgr to use Qt Unicode functions instead of ICU
> 
> ICU is optional in SWORD. Without it that functionality is not
> available. By implementing it with Qt - which will always be available
> in places that BibleTime goes - we can always have ICU-like
> functionality available.
> 
> > Replacement for CurlFTPTransport to use Qt instead of curl
> 
> See previous comments which apply here as well.
> 
> > Your own lucene impl instead of the builtin SWORD search engine.
> 
> The story as it has been told me claims that not all fields of SWORD's
> data are searchable with SWORD's Lucene implementation. BibleTime's
> implementation supposedly allows more fields to be specified. Maybe
> this was a misperception on BibleTime's earlier developers or maybe it
> has been changed in SWORD or maybe I was told a tall tale. But that's
> the reason.
> 
> > Reimplementations of all the HTML filters in SWORD.
> 
> The ones I have seen are extensions of SWORD's filters, not
> reimplementing it. BibleTime had a unique display engine at one time.
> The filters haven't been changed much (if at all) since then. But it
> also displays some things in a very different way than e.g. Xiphos
> does. BibleTime allows our users to create their own CSS stylesheets,
> and we display parallel versions in vertical tabular columns. For that
> it needs modified HTML from what other frontends might require.
> 
> > Are you partners in software development with CrossWire or just using
> > SWORD to get access to our module library?
> 
> Pejorative and leading questions are unnecessary.
> 
> > If you really feel you have a need which isn't handled by the engine,
> > wouldn't you rather like to contribute code back to the engine for other
> > frontend projects to use, and take advantage of the numerous improvements
> > other projects have contributed to these things?
> 
> BibleTime is GPL. Anyone is welcome to see and take our code and
> modifications either into their application or back into SWORD.
> BibleTime simply isn't willing to wait for something to get pulled
> into SWORD and then wait for a new release to be made when we can
> already do what we want. No one on our team is willing to go into the
> engine like Karl is or Chris or DM, so when we need or want a change
> or improvement we do it ourselves in BibleTime.
> 
> > I always hear the argument that you want to add other data sources in the
> > future, or completely replace SWORD in the future. Why? Are we a body
> > working together, building common tools, sharing together, or do you only
> > take what we're offering?
> 
> This is a false dichotomy. Adding in the ability to draw from a source
> like the Perseus Tools or read a general e-book from Gutenberg does
> not mean incompatibility with SWORD if implemented properly. I haven't
> heard anyone saying that SWORD ought to be completely replaced in
> BibleTime, only that we want to not require SWORD as a direct
> dependency while simultaneously expanding what our users have access
> to. SWORD is the premier source of free Bible texts but it isn't the
> only good source of texts. And some things from e.g. Perseus are
> valuable not only for the texts that they contain but their
> functionality to parse classical languages in their original or
> transliterated forms.
> 
> > ;) Just wanted to rile up conversation. :)
> > 
> > Troy
> > --
> > Sent from my Android phone with K-9 Mail. Please excuse my brevity.
> 
> The fact that the current code is not well organized to make such
> abstractions and expand the feature set reflects that this wasn't even
> a goal when the current application was written. It doesn't reflect
> that we should just use what SWORD has when our desire is to be able
> to expand beyond just SWORD as a source of materials and features.
> 
> A second reason, not addressed above, that BibleTime wants to abstract
> apart multiple layers is to permit sharing of more code across GUIs.
> Qt is diverse enough (versions available that work on Windows, OS X,
> Linux, Symbian, Android, Windows Mobile, MeeGo, iPhone, WebOS, Kindle,
> Blackberry and more) that if BibleTime can be split into at least two
> portions - the abstraction layer that wraps SWORD and the GUI - then
> anyone could use Qt to develop a custom GUI on top of our work for
> that platform. This is our second hope with our abstraction work.
> There are already people who are using BibleTime as a base to create
> mobile-centric apps. If we can properly do this work then we can
> facilitate their efforts and possibly encourage others.
> 
> --Greg
> 
> _____________________________________________
> 
> bt-devel mailing list
> bt-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/bt-devel