[sword-devel] Making Import Easier

[Jonathan Marsden]

Greg Hellings wrote:

> I was just chatting with Matthew Talbert about the process of making
> module import a much easier task than it is now.  ...

Interesting.  To me (I am probably a very atypical module creator, but I 
have played with the mod2* and *2mod tools as part of packaging them!), 
the hard part was that the various "source" forms of modules are not 
well documented, or if they are, finding that documentation takes work.

For example, tei2mod.  TEI - OK, Google it, discover what it is, find 
out that there is a schema for that... but then it turns out (after 
further googling) that the schema that tei2mod uses is a special 
SWORD-unique variant of it :)  And the output of tei2mod --help does not 
point the user towards that schema, or any related documentation, and no 
man page for tei2mod was provided with the library (but see below!). 
Also... why is there no mod2tei for the inverse conversion operation? 
This process of discovery appeared to me as somewhat "unfriendly" 
(unnecessarily difficult).  Normal humans might well decide to give up 
before I did.

I think the idea of a GUI importer is good, for those who are 
commandline-impaired, but I suspect there are other obstacles to module 
creation by novices, which just having a GUI will not solve.

I was hoping that one thing our packaging team can contribute back to 
SWORD could be the man pages for each utility.  Right now our man pages 
are very minimalist, but with a little refinement they could be turned 
into something that helps newcomers figure out exactly what form their 
source needs to be in before doing a whatever2mod on it.

On the issues Greg listed that impede novice module creators:

> can't find the utilities because they're not packaged in their
> distro's libsword package,

That's a packaging issue, and one we should fix.  They will all be in 
both Debian and Ubuntu packages once we get this current round of 
packaging actually into the distributions; if this is an issue for 
RPM-based distributions, maybe the answer is to work with the SWORD RPM 
packagers (who are they, BTW -- it might be good for us .deb folks to 
talk to them!) to get the utilities included?  They may be able to use 
"our" man pages, too, if that helps at all.

 > they can't use a command line comfortable,

This is the one where a GUI may help.  A simple GUI wrapper that just 
exec's the appropriate utility binary should be fairly trivial to write? 
  This would avoid needing every single GUI front end from different 
development teams from adding this functionality.  PythonCard might be a 
good tool for creating such a GUI wrapper.

> they have trouble finding the Windows build of the binaries,

This sounds more like an advertising/publicity/documentation issue than 
a technical one?  Or it could be a packaging issue -- how are such users 
getting their SWORD library onto their Windows PCs, and doesn't it come 
with the utilities??  If not, why not?

> Why shouldn't a front-end have the ability to be pointed to a file on
> the user's hard drive (or even remotely to a web-based resource or any
> other number of resources) and have them press a button and voila!
> their module appears

Perhaps because that isn't the function of bible-reading software? :) 
The percentage of folks who create modules compare to the total users of 
a GUI SWORD front must be miniscule.  Why add bloat to a tool that is 
not designed as a SWORD module development tool, but *is* designed to be 
a good Bible reading tool?

Also, your approach of adding import functions to the library itself 
would (I think?) mean that every new source format requires a library 
version bump, instead of just a new import utility.  This is surely a 
bad thing longer term -- if someone wants to create a word document to 
SWORD module converter, an ODF to module converter, or even (dare I say 
it!) an esword2mod tool, I would think that they should be able to do it 
*without* modifying the underlying SWORD library.

 > ... If those utilities were refactored so
> that the option handling and file reading was in one file and the
> actual text processing and module writing were in a second file, that
> would allow those second files to be included directly into the
> library and be linked against by any front end.

This sounds like API bloat?  There is a way any program,including 
current front ends, can do such conversions now -- exec the existing 
utility programs with appropriate parameters.

> Then any front end who wants to play nice to the user like this would
> not have to reduplicate the efforts of the current utilities, nor
> would they have to hack the code out of the SWORD source and shoe
> horn it directly into their own application.

They don't need to do any of that now, do they?  They can just exec the 
existing utility programs!  It may well be less efficient, but we're not 
expecting anyone to be bulk-converting tens of thousands of modules via 
a GUI, are we?

> I think it would be a very user-friendly thing to have available, ...

Agreed.  I think a GUI tool for module creation is a good idea.

I'm a lot less sure that creating one by adding several conversion 
functions to the library API is a sound approach, when the existing 
utilities can be used for that same purpose today.  I'm also less sure 
that it makes sense to add GUIs to every single front end bible reading 
program for this (specialized) task; instead, I'd suggest a smallish 
separate GUI tool that "knows" how to grab source material and "knows" 
the command syntax of the existing import utilities.  To me this seems 
simpler, and has lower (zero!) impact on the work of both SWORD library 
and front end developers, because in essence this would be an 
independent small project.

Jonathan
--
Jonathan Marsden