[bt-devel] PDFs and other object files in source tarballs (was: Re: Committed English handbook pdf, r1829)

[Jonathan Marsden]

Martin Gruner wrote:

> Am Dienstag, 17. November 2009 23:26:14 schrieb Jonathan Marsden:

>> I'd prefer not, or I'll probably have to do extra work to create a
>> special PDF-less tarball each time!  PDF is not the preferred source
>> form of the information, 

> Who defines that?

Whoever creates and edits the documentation.  If they actually edit it
as PDF files using a PDF editing application, then sure, in that case,
those PDFs are the preferred source form of that documentation!
However, if they edit it as XML, docbook or whatever other schema they
choose, then *that* is the preferred source form of that documentation.

In the case of BibleTime, it appears to me that DocBook XML is the
preferred source form of that documentation, by this definition.  Am I
correct about that?

The GPLv2 says:

    The source code for a work means the preferred form of the work for
    making modifications to it.

Therefore, as I understand this, a "source tarball" is a tarball in
which the files are in the preferred form for making modifications to them.

Similarly, GPLv3 says:

    The "source code" for a work means the preferred form of the work
    for making modifications to it.  "Object code" means any non-source
    form of a work.

This suggests to me that HTML and PDF files of the BibleTime
documentation are considered to be "object code" forms of that
documentation.

The Ubuntu Packaging Guide says:

  Packages must not be accepted if any of these points is not fulfilled:

  ...

    * Files shipped under the GPL must be in the 'preferred form of
      modification' format. This applies to some other free licenses
      like the MPL, too (but e. g. not to BSD). Negative examples are
      Flash animations (*.swf), most PostScript/PDF files, and
      automatically generated source code.

Now I'm looking over things in detail in response to your message, I
think the *.qm files which are apparently being included in the
BibleTime "source" tarballs may also be a problem, for the same reason.
 They do not seem to be in a preferred editable form, to me.  I strongly
suspect they are object format files derived from their *.ts
counterparts.  I did not notice these *.qm files until now.

>> so the PDF file should not be in a source tarball. Just like object
>> files are not in the source tarball.

> Well, following that philosophy, we could also not have the generated
> HTML files in our sources, because the information is maintained in
> the docbook files.

Right, omitting them would be much better.  They are derived files.  See
above.  Please leave out the HTML files from the source tarball (and
from svn).

I confess, I knew about those, but had been "conveniently ignoring" the
issue.  I'd hoped to continue that approach, even when the issue if PDFs
was raised.  But apparently it is not to be.  This 'can of worms' is now
open -- you raised the issue.  So, let's solve the problem.  From my
perspective, the easy way to do so is to create a true source tarball,
and then make sure the cmake/makefile rules do their thing to create all
needed derived files during the build process.

My sense is that, as a practical matter, derived HTML files in a source
package will slip through the usual checks on Debian packages, usually
(because some people do write documentation directly in HTML).  GPLed
PDFs in a source package will almost certainly be challenged (because
almost no-one writes docs directly in PDF).  If a source tarball
includes derived files, non-source material, then it is (by definition)
no longer a source tarball, but a mixture of source and "object"
(derived, compiled, etc.) format files.

> We do include them, however, to ease the burden of compiling and 
> installing BibleTime for end-users not as skillful as an experienced
> packager or end-users on other platforms. That is the same, or even
> more so, in the case of PDFs.

That is a worthy motive.  However, unskilled users do not generally
choose to use obscure platforms for which no binary packages are
available, do they?

Further, what would an unskilled end user be doing with a *source*
tarball in the first place?  Such end users should probably not be using
such a thing at all, they should be downloading a compiled and packaged
version of the software, or using an automated system like the *BSD
Ports Collection to obtain and compile and install it for them.

Under what circumstances do unskilled end users really need to manually
download and work with a source tarball for BibleTime?  If such
circumstances do still exist, how can we minimize them?

If there really are such end users, why not make a "precompiled
documentation" tarball, containing the documentation in whatever
precompiled forms are believed to be useful, just for such folks?  It
could be automatically generated from the source tarball, by definition.
If such users strongly object to having to download two tarballs, for
some reason, then we could generate some kind of a
"source-plus-precompiled-documentation" tarball, just for them.

> I think it is reasonable to distribute the documentation along with
> the application.

Yes.  I agree 100%.  Source documentation should usually be distributed
along with source of the application.  Compiled documentation should
usually be distributed along with compiled form of the application.

The source tarball contains the source form of the application, so it
should contain the source form of the related documentation.  Similarly,
if the BT project distributes binary installers (for Windows, for
instance) or images (.dmg files for Mac, for instance), then clearly
*those* should contain appropriate compiled forms of the BT
documentation, not just the XML source form of that documentation.
Those binary installers are what unskilled BT end users will download
and use anyway in practice on those platforms, I would think, just as on
Debian or Ubuntu (or Fedora or SuSE) such users will (generally) install
the BT binary packages for their distribution, not BT source packages.

If the BT team really believes that there is significant value to end
users in having BT release "source-plus-extra-precompiled-documentation"
tarballs, please also release actual source tarballs if at all possible.

One more related request: if you really *have* to include these object
format documents and files, please could we at least make sure they will
be correctly regenerated from their corresponding source files if they
are deleted, and the project is then built from source?

I'm not entirely sure if this is currently the case, not being any kind
of Cmake expert at all.  I can see that rules in
cmake/BTDocumentation.cmake should do this generation, using xsltproc
for the HTML generation and lrelease for the qm file generation.
However, I can't see how these rules are being invoked from a top level
make, at all.  It looks like the main make just assumes the *.qm files
are already built and present in the source tree, and fails if they do
not exist?  Am I just not understanding Cmake here?

Is the current process for generating both the HTML and the qm files
documented somewhere?  If not, can someone please document it for me :)
 I found http://devel.bibletime.info/wiki/Translation but it does not
seem to specify how the Cmake stuff related to documentation/translation
works, and it doesn't mention lrelease at all.

Thanks,

Jonathan