[sword-devel] Tagging verses and verse lists

[Jonathan Morgan]

On Dec 19, 2007 11:24 PM, Eeli Kaikkonen <eekaikko at mail.student.oulu.fi> wrote:
> Thanks for comments, I have to say that you have probably thought this
> more thoroughly than I have. Your point of view is valid throughout but
> I still try to give a second opinion...
>
> On Tue, 18 Dec 2007, Jonathan Morgan wrote:
> > There are a few good reasons (at least in theory) for being more specific:
> > 1. Restricting work to being done once (in the engine).  If I have a
> > "general framework" then it is far more likely that every application
> > will do the same things in slightly different ways.  If I have an API
> > at the level of intent for a feature then it will work in exactly the
> > same way across applications, and applications will only need to use
> > that API.
>
> That is true. This is, however a tradeoff.

True.  IMHO, the best such solution should try and choose an 80/20
style tradeoff - do 20% of the work to get a simple and standardised
API that can do almost anything that a reasonably average user would
want to do.  But I still feel that without focusing on particular
problems to solve, there is going to be a lot of "I think we might
need this, so I had better include it now".  If we can formulate our
problem and our target audience, then it will be more readily apparent
which tradeoffs are reasonable and which aren't (or at least, I hope
it will).

> > The problem with such a standard is:
> > 1. Each application has to "follow the standard" - things like the
> > Personal Commentary shows that this is far from reality.  This again
> > leads to more work and more incompatibility, as per my previous
> > comment.
>
> Yes, it leads to more work when coding the frontends. But why the
> Personal Commentary has failed? Was there ever any such standard? Did
> the developers discuss about interoperability?

No idea.

> Here we come to an interesting point. Why there is no api for getting
> OSIS div, passage, footnote, word etc? Because the api has to solve a
> general problem and move the responsibility to the frontends. Actually
> this is exactly the same situation than with the Personal Commentary: we
> have (or should have) a standard which has to be obeyed in modules and
> frontends.

I don't know this area in detail, but is it possible that this just
didn't occur to the author?  Some of these things might be able to be
standardised in the API now, since we have around half a dozen
frontends and the JSword API to look at to get an idea of typical
usage.  If it turns out that there is only one sensible way to do a
certain thing, then I think it should be moved into the engine if at
all possible.

> My point here is that there are different situations that call for
> different solutions. The question is now if this situation (tags, verse
> lists, bookmarks...) calls for a general or a specific solution, and how
> general or how specific.

Where possible I will move to the more specific solution.  I will say
again that the solution should be as simple as possible (and no
simpler), but it will need a bit more discussion before we can say how
simple that is.

> > 2. There is no way to tell that a particular module (let us say "x")
> > actually contains our bookmarks.  We could make another "standard" for
> > dealing with this, but again it means that every application has to
> > follow that standard.  It is much easier to provide an API at the
> > level of intent and solve the problem once.
>
> But again, how general is this problem? If we talk about bookmarks there
> would be api for bookmarks. But if you are right and bookmarks are not
> the right solution for verse lists there should be api for verse lists.
> And if your application uses the verse list in a specific way there are
> two possibilities:
>
> 1) All applications must use the verse list with the same semantics, not
> only with same syntax. If you use it for tagging there will be UI and
> end user problems if others use the same list as bookmarks. 2) The api
> has to have a way to differentiate between verse lists which are meant
> to be used differently. I think this is the only good solution,
> otherwise the lists can not be used e.g. for both saveable search result
> lists and user defined tags.
>
> And now we are again in the situation which you wanted to avoid: we
> don't know what the list (module) contains. We have to have a standard
> which says that this is a tag list, that is a search result or some app
> specific list.

I will repeat that I am drawing this distinction between different
types at the user level, not at the implementation level.  I would be
very surprised if there did not exist a common format which can be
used for all three.  I was certainly planning to use verse lists for
tagging, with the semantics:
1. A passage is tagged with a topic if it is contained in a verse list
with that topic's name.
2. Tagging a passage with a topic adds the verse to the end of the
verse list with that topic's name.
3. If hierarchical verse lists are supported, then the verse is tagged
with a tag like Topic > Subtopic > Sub-sub-topic (in typical URL
breadcumb) style.  Tagging adds to the end of the sub-verse list in
the same way as in (2).

In this way users of my tags from an application using verse lists
would see verses in the order that I tagged, and I would see verses as
tagged if they added them to the verse list (no matter what order they
added them).

I think bookmarks are identical in implementation.  The only
difference I see is the potential semantic difference to the user I
highlighted earlier.  Different applications are welcome to call them
different things, though I would request that within an application
consistency of terminology is used (just for the poor average user's
sake).

> > > The standard could be flexible enough to enable specialized uses and
> > > application specific tags. HTML already works like that - if a tag is
> > > not recognized it's ignored. That way the applications could add their
> > > own data to the data which the other apps could still use.
> >
> > HTML is a good example of why this is not a good approach.  Most HTML
> > parsers are flaky and incompatible - why, when we have a comparatively
> > simple feature to add, do we need to add application incompatibility
> > (sorry, "flexibility") and duplicated effort?
>
> Why then OSIS allows for custom tags (or at least attributes)? Why the
> Sword engine handles (allows) them? Whey there is no api which prevents
> duplicated effort?  Because we have a too general problem to be solved
> with a specific api.

Still not entirely convinced of that.  I suspect that at least some of
the things could be solved with a specific API, but it was easier not
to.  Again, I do not know that particular domain well enough to make
concrete statements about it, so it is just my suspicions.

> Besides, when we talk about bookmarks or even Personal Commentary
> standard we are talking about a very small subset of any markup. The
> problem with html is that the standard has not been followed at all by
> the html designers or app developers. We could do better.

We could do better, and I would be very glad if we would.  I just give
my suspicion, based on long experience, that we would not.

> > Again, IMHO "this system is more
> > general" != "this system is a better solution to the problem".
> > Frequently it is the other way round.
>
> You are right about that, though I again say that it depends on the
> situation. The solution should be as simple as possible as you also
> said, but first we have to see if there really *IS* a more general
> problem to solve. I have a bad habit of overengineering, I hope our
> views complement each other and we can come to some solution here.

I used to have the habit of over-engineering, and still do (to some
extent).  The main things that changed my mind about over-engineering:
1. I found it was wasting too much of my time, and making it harder to
improve software (not easier, as useful generality should do).
2. I was adding features to meet my idea of perfection, rather than to
meet the goals of users.  Finding out what users wanted to do and then
coming up with a better solution (because it drew on the specifics of
their problem rather than my ideas about what it should look like)
which didn't waste my time on things that they wouldn't use, struck me
as a much better way of doing things.

I also have some amount of HCI background, which makes me care more
about the user and usability (it helps in this case that I am my own
target user, so I know the exact problem I am trying to solve).

At work, I hear far too much of "it would be really nice to have...",
and I know that most of these features are unnecessarily complicated,
will not be understood (let alone used) by the average user, and will
make me miss deadlines in other areas.  My antidote is try and reduce
the complexity.  I still over-engineer just to make myself look good /
clever at the expense of my users, but I try to rein it in.

> Therefore, I repeat this:
> > > Anyways, I would like to see more co-operation between applications
> > > whether it be in the engine or in the form of common standards.

So would I, but again we need to be sure of the problem we are solving
before we solve it.  The more general the problem you want to solve,
the better you need an understanding of what is required and what
isn't.

Jon