[sword-devel] Improving documentation and its visibility (was: Re: Making Import Easier)

[Jonathan Marsden]

Chris Little wrote:

> Basically, none of the source formats are of our invention, they're 
> standards maintained by other groups and documented by other groups. 

True, but that shouldn't prevent docs and --help output that point to 
the relevant external documentation being supplied as part of a SWORD 
library release.

> (The IMP format is the exception, but it's very simple and well 
> documented to the extent that it is documentable at 
> http://www.crosswire.org/wiki/DevTools:Modules.) We do offer basic 
> documentation for all formats and about their use within Sword in the 
> Wiki. We do expect users to actually read this documentation or, failing 
> that, ask questions via any of the many support channels.

Fair enough.  Would CrossWire be willing to consider including a 
snapshot of the relevant wiki pages in each library source tarball? 
That way, anyone who has the sources also has the wiki docs, in the 
state they were at time of that library release.  This still requires 
people to read them if it is to do any good (!), but it does make them 
considerably more visible and readily accessible to someone who 
downloads the sources and builds the library and utilities.

I just did a quick check and all the wiki pages except those starting 
with Special:, Talk: or User: come to 78 pages and 1.7MB, and they 
compress into a tar.gz file of only 217KB.  That's a 12% size increase 
over the sword-1.5.11.tar.gz file, which IMO is well worth it.

I confess that I had seen the wiki as being more of an internal resource 
for CrossWire developers, rather than an external publication for 
CrossWire library users (front end developers, module creators, etc.). 
What you've said (that people, me included, should see the wiki as a 
publication for users, and so read it!) makes sense -- I just didn't see 
it that way until now.  To my mind, the main CrossWire SWORD website 
does not seem to treat the wiki that way, either... unless I missed 
something, it mostly ignores it.

Maybe adding a link to the wiki in the navigation area at the top of 
both http://crosswire.org/sword/index.jsp and especially to 
http://crosswire.org/sword/develop/ would help people like me realize 
who the wiki is for, and how valuable a resource it is?  A sentence or 
two in the text of this second page, suggesting that people read the 
wiki (and linking to it of course) would also be good, IMO.  I only 
found the CrossWire wiki because it showed up in one of my Google 
searches, just a few days ago -- that suggests to me that  it could 
probably benefit from improved visibility from the main web site :)

> Our use of TEI in particular does not require the use of *our* TEI 
> schema. Obviously it would make the most sense for a new user of TEI to 
> check our Wiki, where everything will be spelled out.

I read http://www.crosswire.org/wiki/TEI_Dictionaries and it says in part:

   For the purpose using TEI P5 in SWORD, we have developed a special XML
   Schema that includes the basic set of P5 modules necessary for
   dictionaries and adds osisID and osisRef attributes (with their normal
   OSIS syntax) to many elements. This will permit cross-referencing with
   OSIS modules and the use of standard Bible references in TEI
   documents. Our customized TEI schema is available at
   http://www.crosswire.org/osis/teiP5osis.1.4.xsd.

Nothing there that I can see "spells out" to me that this is in any way 
optional.  The Validation section of the same wiki page links to this 
custom schema also, which strengthened my (apparently incorrect) belief 
that this was the one and only schema which the tei2mod tool requires 
its input to conform to.

> Any standard TEI P5 doc should work fine for our importer (to the extent 
> that we support TEI, which is to say, for dictionaries--as our Wiki 
> shows). The additional attributes in our version of the schema are just 
> that: additional. They aren't a necessity, but are useful for marking 
> Bible references in a standard format (namely, in the OSIS way).

Thanks.  Your paragraph just quoted is a clarification which (minus its 
parenthetical aside) should be added to the relevant wiki page, IMO.

> There aren't any plans to write mod2tei. mod2osis exists in order to 
> encourage adoption of OSIS. If you want to see module internal
> markup, use mod2imp.

While I can live with that, I'm a bit uncomfortable with it.  In 
general, format conversion tools are much more useful if they are 
bidirectional, as this allows one to readily check what information is 
lost or changed by doing the conversion, and allows quickly fixing 
broken modules by filtering:

   mod2X A |sed -e 's/mistakenthing/correctedthing/g' |X2mod B

and so forth.  More automated checking/validation of modules might well 
result from this kind of capability (so that things like the Judges 
references would in future be caught before a module is released).

> Also, I've posted some example docs and announced them in the past, but 
> it does seem prudent to link them from the Wiki, so I've just done that.

Thanks... now we can point to those from our man pages :)

What I would like to see in the Debian/Ubuntu world is that if someone 
installs libsword7 (and especially libsword-dev), there is sufficient 
documentation (and pointers to external documentation) included so that 
they can readily bootstrap themselves from there, as a user of this 
library and the tools that come with it.  That may be asking too much 
for this release of the .deb packages, but it's the direction I'd like 
to see us head in longer term.

At some point I'd really love to see doxygen formatted comments added to 
the public SWORD API sources, for every non-private method, so that API 
documentation generated from those sources becomes more readable and 
more complete than it is at present.  But that's a whole separate 
subject -- useful man pages are a smaller and perhaps more attainable 
initial goal :)

> Windows builds are static and found at 
> http://crosswire.org/ftpmirror/pub/sword/utils/win32/.

OK.  How hard would it be to include dynamically linked versions with 
the Windows binary library installation?  And how big would they be?  if 
they shrink back down to ~300K, I'd *really* suggest including them!

Jonathan