[bt-devel] BibleTime's Goals (was Re: New passage selector proposal)

[Troy A. Griffitts]

Greg Hellings <greg.hellings at gmail.com> wrote:

Ok Greg, if you speak loudest for the Bibletime team, then I will answer your objections.

So if I understand you correctly, the Bibletime team doesn't contribute because they cannot introduce dependencies into the core engine?

>1) An insistence that no dependencies be introduced. I detailed why in
>my previous message. BT's developers are primarily people who aren't
>interested in meeting that objective.

Keeping the engine as dependency-free as possible is a good design decision for a library. Every dependency we introduce cascades out to all of our projects. I owe it to our users to keep the engine as dependency-free as possible. You've complained to me in the past about not allowing introduction of an XML and XSLT library dependency into our engine. You're wrong about this point Greg. SWORD would benefit nothing from adding an XML parser as a dependency. We don't work with DOM structures in practice. Pushing our filter logic from C++ to XSLT would only shift the logic and expertise. You may be good at XSLT, but I absolutely loath the same. We have an optimized XMLTag class which makes it super fast and easy to process XML tags. We have a SAX base filter which passes tags to your derived class allowing processing exactly the same way as an XML sax processor.

We do have dependencies in the engine, but I will not introduce dependencies for developer preference. If I can do a little extra work to provide similar or better functionality we need to prevent introducing dependencies, then I will make this extra effort for our users, to save them from being forced to add a dependency to their project.

Your comment about 'buggy' parser because we didn't handle XML comments is moot. Once ever did we have a problem with this, it is easily corrected if we start getting texts from publishers with lots of comments, and a patch has already been submitted and DM is applying the patch.

What is the argument? You actually want dependencies? This effort, and the effort to use if available, but not depend on: ICU, cURL, CLucene, zlib, libregex is for your benefit and simply good design and development for a library.


>2) A heavily centralized development model that goes through
>essentially one gatekeeper, maybe two.

It is important to not refactor not make casual changes to a library. There are many many projects which will be affected by the smallest changes.  We took great pains to introduce alternate versification into the engine, making it completely backward compatible with existing frontends for their current module base and as easy as possible to support the new av11n modules-- in many places also without code changes. This means tough decisions to push submissions off until they are proven. Bibletime and other frontends are the perfect proving ground for new engine features, but BT won't even use SVN to tell us how our new code is working for you. I don't apologize for being a strict gatekeeper. Again, this is for your benefit.

"Release early. Release often." Is not the mantra for a stable engine, the dependency of many projects.

>It's a conscious choice you've made with your
>strategies that falls at odds with out BT's people want to work. But
>not with what we want done, so we do it by building on SWORD rather
>than building into SWORD. 

So, if these are the 2 major barriers, then I hope the primary BT code contributors can sympathise, or at least respect the decisions we've made.

>Again, all our work is free for the taking -
>we're just as GPL as you are.

I'm not pleading for your code. I'm pleading for your cooperation with me and for all our frontend projects to actually share in work together by contributing to a common code base.

Any other major BT contributors have an opinion? I'm happy to have g+ sessions to get people intimately familiar with the engine internals and defend our design decision and hopefully earn your respect.

Troy


>
>--Greg
>
>On Fri, Apr 20, 2012 at 11:07 AM, Troy A. Griffitts
><scribe at crosswire.org> wrote:
>> I hear what you're saying Greg, though I find your detailed issues
>not
>> insurmountable.
>>
>> My comments in this thread are mostly rhetorical; I wasn't looking
>for
>> technical responses.
>>
>> We need to decide:
>>
>> Are we a community?  Why are we a community?  What EXACTLY do we have
>in
>> common?  How can we work together on that commonality?  I see that as
>SWORD.
>>
>> I don't plan on living forever and I would love for the many projects
>that
>> use the SWORD engine to start owning it more than they do.
>>
>>
>>
>> The vision for SWORD has always been to produce a lean and powerful,
>> cross-platform Bible research framework which many projects can share
>in
>> using and developing.  I believe we've attained that over its 20 year
>> life-cycle, but I don't believe we've seen the contributions to the
>core
>> engine that we could see if people would buy in and own it as their
>backend.
>>  I've personally written at least 5 frontends with the engine and all
>my
>> frontend work has driven improvements to the engine.  I haven't
>worked on a
>> frontend in a long time and am no longer on the front lines--
>situated to
>> know what improvements now need to be made.  I would love it if each
>> frontend project had one developer who was intimately familiar with
>SWORD
>> and could liaise between the backend and frontend teams and
>contribute to
>> both.
>>
>> Greg, I appreciate your contributions to the engine.  Please don't
>think my
>> goal is to accuse and blame anyone for never contributing to the
>engine.
>>  I'm just attempting to make our projects think about our involvement
>with
>> each other and how we can work more closely together.
>>
>>
>> Troy
>>
>>
>>
>>
>>
>> On 04/20/2012 05:23 PM, Greg Hellings wrote:
>>>
>>> On Fri, Apr 20, 2012 at 12:18 AM, Troy A. Griffitts
>>> <scribe at crosswire.org>  wrote:
>>>>
>>>> So, in summary, you don't consider yourselves part of the CrossWire
>>>> developer community? None of the things you've mentioned has been
>>>> realized
>>>> in 20 years, SWORD is practically a direct dependency, and if you
>would
>>>> like
>>>> access to Perseus, wouldn't it be better to put in the effort to
>make
>>>> Perseus (and SWORD data) available through a well thoughtout Bible
>>>> research
>>>> framework instead of your patchwork backend you've lived with for
>20
>>>> years
>>>> because of 'in principle' arguments?
>>>> SWORD is designed as a backend for Bible research software, with a
>>>> modular
>>>> architecture (as seen by your many reimplementations; just because
>you
>>>> _can_
>>>> do something doesn't necessarily mean you _should_) allowing
>plugins to
>>>> access different data sources and in different markups. If you
>think you
>>>> can
>>>> design a better class heirarchy, you may be right, but why? Is
>SWORD
>>>> really
>>>> so different than what you would design yourselves as a backend? Is
>it
>>>> always 'do it ourselves', and 'differentiate from the other
>CrossWire
>>>> projects' or are you willing to share your improvements with the
>>>> community
>>>> as you have used our work for the entire life of your project?
>>>>
>>>
>>> Troy,
>>>
>>> If you want to see BibleTime as not part of the CrossWire community,
>>> that is entirely your prerogative. But you are most
>>> definitely putting words in my mouth when you ask me to agree with
>>> you. I prefer to think of BibleTime and SWORD as
>>> being two separate and complementary objectives. SWORD providing
>>> low-level functionality like storing texts, fetching
>>> them over the network, providing rapid access to them, parsing user
>>> input, and all the other things that SWORD is good
>>> at. BibleTime concentrates on pulling SWORD's text into
>Qt-compatible
>>> formats (QString) and displaying them with a GUI.
>>> These are different and complementary objectives. Yes, SWORD is
>still
>>> a direct dependency because until 5 years ago
>>> no one even conceived of pulling data from elsewhere and none of us
>>> have stepped up to do the work required to decouple
>>> BibleTime's SWORD-access code from its GUI code (something which
>>> supports our likewise new goal of permitting multiple
>>> GUIs to spring out of one Qt wrapper for SWORD. If I'm the one to
>get
>>> to the work first I intend a three-layered
>>> separation between a SWORD Qt binding in the engine, a
>>> display-template layer all Qt-based GUIs could use, and then
>>> the current BibleTime desktop GUI).
>>>
>>> I haven't heard anyone say they think that SWORD's design and
>>> architecture are poor or in need of replacement. Where the
>>> BibleTime developers differ from SWORD is in their opinion of
>>> implementation. SWORD has a nearly single-minded goal of
>>> never relying on external libraries other than the standard C
>library.
>>> To that end SWORD implements its own search
>>> functionality if Lucene is missing, includes an FTP library for when
>>> cURL is absent, and simply throws up its hands when ICU
>>> is not available. Additionally you implement your own XML parsing in
>>> the importers, the older filters, and the newer
>>> faux-SAX utility based filters. In some ways this is good - SWORD
>can
>>> go where ICU, cURL, Lucene, and<your favorite
>>> XML library here>  cannot go. In practice I don't know what benefit
>>> that gains SWORD as I have yet to work with a
>>> platform those did not support, but that is SWORD's philosophy.
>>>
>>> BibleTime currently only goes places where Qt and SWORD are
>available.
>>> We have also agreed that we are not going to
>>> eschew library dependencies the way SWORD does. So we require Qt for
>>> its added functionality, we require Lucene and our
>>> implementation of the search because we can't guarantee that SWORD
>>> will have it. You are welcome to pull anything that
>>> BibleTime has done back into SWORD but those of us working on
>>> BibleTime are not interested or willing (at least not up
>>> to this point) to reimplement our filters using C-style strings and
>>> SWBuf and ICU because Qt provides us what we need
>>> without the necessity of reimplementing. This is not an
>unwillingness
>>> to participate in SWORD and CrossWire but rather
>>> an unwillingness to reimplement what other libraries already provide
>>> us for free.
>>>
>>> To the specific examples you cite - I've already iterated why
>>> BibleTime has its extensions technically in my previous
>>> response and philosophically here. Our code is open, if our filters
>>> are better than SWORD's or fix its problems then
>>> you are welcome to adapt our QString to your SWBuf/char* functions
>and
>>> enjoy that. I, for one, won't do such because
>>> the mantra of "no dependencies" means the work would be much more
>>> difficult than it needs to be in some cases.
>>>
>>> And for a few examples of my own:
>>> *) SWORD's CMake scripts are only put together because I pushed back
>>> that functionality from BibleTime to the engine
>>> after hearing from Linux packagers and Matthew Talbert that they
>would use
>>> that.
>>> *) PERL bindings were only fixed up when I completed the CMake work
>>> because I was able to easily identify the one
>>> missing piece of functionality
>>> *) SWORD's CLucene 2 support that I helped push was only made
>possible
>>> by work I was already doing to track down the
>>> issues in BibleTime's CLucene support.
>>> *) I asked for better CSS support on the engine side so that users
>and
>>> module creators could define their own style-
>>> sheets. I was told no way. But BibleTime was willing to go at least
>>> half that way, so I implemented user-editable
>>> sheets there. It could use some better exposure, but that's
>something
>>> SWORD wasn't willing to do.
>>> *) BibleTime developers have lamented several times that SWORD does
>>> not use an XML library for its XML
>>> processing, but every time such an issue has been raised with SWORD
>>> the answer comes back: "No external dependencies.
>>> We'd rather have our buggy implementation than possibly introduce a
>>> dependency." This goes contrary to how most of us
>>> here at BibleTime are willing to behave - so we left things like
>lack
>>> of XML comment support alone because we
>>> saw no reason to need to reimplement XML parsing just to satisfy
>this
>>> mantra.
>>>
>>> Now when we talk about other sources of text we at BibleTime feel
>the
>>> same way as with CLucene and XML. We would be
>>> happy to have access to Perseus or e-Book reading or the like. But
>>> we're not willing to implement that functionality
>>> from scratch and if such was in SWORD it would likely be optional,
>so
>>> BibleTime would probably reimplement it
>>> anyway so that we can guarantee access to it for our users as we do
>>> with Lucene search and ICU. We don't want to have
>>> a user on Fedora access it and on Ubuntu be unable to do so because
>>> they come to us and say, "Your application is
>>> broken", and then we have to do as Karl does often when Xiphos
>search
>>> breaks - say "Well we just expose what SWORD has
>>> and if it doesn't have Lucene then oh well". We'd rather give our
>>> users a consistent experience across all platforms.
>>>
>>> How is what we do different from Xiphos who maintains a patch
>>> introducing GLib dependency to SWORD in order to fix a
>>> Windows Unicode bug? How is what we do different from PocketSword
>who
>>> forces users to download Lucene indexes for
>>> search on iOS - a functionality none of the rest of us have? How is
>>> what we do different from JSword who maintains a
>>> complete reimplementation of the SWORD library lacking in
>>> functionality? By your metric none of those are members of
>>> the CrossWire community either because they haven't pushed those
>>> implementations back to the engine.
>>>
>>> So you see, the issue is not a lack of community involvement or a
>lack
>>> of community sense. The difference is one of
>>> philosophy and objective. BibleTime wants to present what
>>> functionality and features we can access to our users. SWORD
>>> wants to be a low-level library that can go anywhere and have no
>>> dependencies. Our objectives are complementary but
>>> our methods are at odds. So if you don't get many direct patch
>>> contributions from the BibleTime team, this does not
>>> indicate we see ourselves at odds with CrossWire or in competition
>>> with it. We are simply composed of people with a
>>> different itch to scratch and a different set of fingernails to get
>>> the scratching done.
>>>
>>> --Greg
>>>
>>> _______________________________________________
>>>
>>> bt-devel mailing list
>>> bt-devel at crosswire.org
>>> http://www.crosswire.org/mailman/listinfo/bt-devel
>>
>>
>>
>> _______________________________________________
>>
>> bt-devel mailing list
>> bt-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/bt-devel
>
>_______________________________________________
>bt-devel mailing list
>bt-devel at crosswire.org
>http://www.crosswire.org/mailman/listinfo/bt-devel

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.