[sword-devel] What markup format is SWORD's internal markup, and/or where is it documented?

Sat Dec 16 04:19:56 EST 2023

I have written a set of markdown filters which I never committed as it was incomplete and I never finished it. 

I can look and see if I can find them. 

One aspect is that our “plain” output actually contains a bit of markdown and is not _really_ plain. It would be good to fix that in the same go. 

Also I think I found while leaning on the xhtml filters for inspiration was good, there were better, simpler starting points. The latex filters I think I built mainly on top of the rtf output. 

I am not a programmer, i just copy and play with until things do as I want. 

Peter

Sent from my phone. Please forgive misspellings and weird “corrections”

> On 16 Dec 2023, at 06:43, Timothy Allen <thristian at gmail.com> wrote:
> 
> The SWORD project's command-line tool, diatheke, mentions the following in its command-line help:
> 
> > Valid output_format values are: CGI, GBF, HTML, HTMLHREF, LaTeX, OSIS, RTF,
> >   ThML, WEBIF, XHTML, plain, and internal (def)
> > The option LaTeX will produce a compilable document, but may well require
> >   tweaking to be usable.
> 
> "HTML" produces a reasonable HTML document, and if you ask for footnotes to be included you'll get the footnote *markers* without the footnote *content*.
> 
> "CGI" produces something similar, but only the closing BODY and HTML tags, so you can have a CGI script that prints out a pretty header and lets the SWORD output close it off.
> 
> "HTMLHREF" produces the same document, but if you ask for footnotes, they are turned into links whose URLs match CrossWire's online Bible study tool, https://crosswire.org/study/
> 
> "XHTML" is like HTMLHREF but includes some static CSS and XML-style self-closing tags.
> 
> For conversion to Markdown, I'd recommend getting the HTML version, as it's the simplest and the closest to the document module Markdown uses.
> 
> 
> Timothy.
> 
>> On 16/12/23 15:12, Aaron Rainbolt wrote:
>> I think I'm beginning to understand - there is no one internal markup
>> format, but rather there's the binary format, and then the SWORD
>> engine converts that (sorta) into something very similar to the
>> original markup language. Is that about right? I think the markup
>> format I'm seeing then is a subset of OSIS, and probably other modules
>> will have a different syntax. Which would explain why the best way to
>> do this is via filters since I'll need to handle three or so different
>> markup formats depending on what each module uses.
>> 
>> In that instance, I guess my question becomes, what markup formats
>> does SWORD support emitting a subset of? OSIS is clearly one, and it
>> sounds like ThML is another. Is there a list of these so I can write
>> the needed Markdown filter for each one?
>> 
>> I may attempt to get the Markdown filter collection (if I manage to
>> write it) contributed back to SWORD if that's something the devs are
>> interested in. I'm not used to normal C++, but I can work my way
>> through Qt C++ with little difficulty, so I should hopefully be able
>> to adapt to Qt-less C++ without too many issues.
>> 
>> Thanks for your help!
>> 
>>> On Fri, Dec 15, 2023 at 9:45 PM Troy A. Griffitts <scribe at crosswire.org> wrote:
>>> Hi Aaron,
>>> 
>>> The SWORD engine tries to preserve the best it can the markup from an imported text for few supported markup formats. The markup used by a SWORD module is specified in its .conf file. Most new SWORD modules from our Modules Team are in OSIS markup. If you want to add markdown as a new output format for SWORD, that would be great. To do this, you would want to follow the pattern you see for one of out existing output formats. I would suggest XHTML. E.g., copy the 3 or so *XHTML filters you see here and modify the output from XHTML to markdown.
>>> 
>>> https://crosswire.org/svn/sword/trunk/src/modules/filters/
>>> 
>>> Hope this gets you started. Let me know if you have questions,
>>> 
>>> Troy
>>> 
>>> 
>>> On December 15, 2023 20:26:22 MST, Aaron Rainbolt <arraybolt3 at gmail.com> wrote:
>>>> I didn't mean the actual compiled files - I meant the kind of markup you end up with if you use swModule->getRawEntry(). It's the same kind of markup you see if you use mod2imp on a module. So far the modules I've used this on seem to have some sort of pattern to them (there's <w> tags with "lemma" and "morph" attributes for Strong's numbers and morphology, <q> tags for... something, looks like it's part of how red-letter Bibles work because of the "who" attribute, and things like that). I assume it's this markup that is parsed by the existing filters, and that I would need to parse were I to write my own filter.
>>>> 
>>>> My text renderer does indeed support HTML, but it's ability to output Markdown is sorely lacking (I *can* tell it to give me whatever's in my text editor widget in Markdown format, but it loses information that Markdown is perfectly capable of containing). I need to be able to convert between Markdown and rich text both ways. On top of all of that I'm trying to support a particular flavor of Markdown that isn't normal (the variant Reddit uses in particular), so I have to do the parsing myself to implement things like superscripts and strikethroughs. Implementing a filter sounds like a good idea, but I think I'll have to parse this "internal markup format" to do so.
>>>> 
>>>> On 12/15/23 21:12, Greg Hellings wrote:
>>>> 
>>>> The actual files are a custom binary format which is not documented and is not intended to be any sort of standard accessed by anything other than the library itself.
>>>> 
>>>> Most newer works are imported from an OSIS file. Some older ones were imported from GBF (I think?) or ThML (which is basically some basic HTML display components mixed with a few tags for identifying things like words of Christ or divine names). However, once they are imported as modules some of that structure is lost to the proprietary binary format of the SWORD module files.
>>>> 
>>>> If you want the text in Markdown the best way is to create a filter like the existing filters in the engine which can be used to generate HTML, LaTeX, etc and write some which produce Markdown output.
>>>> 
>>>> Although, since Markdown is basically simplified HTML that is specifically intended to make HTML easier to write, why wouldn't you just render out HTML from the existing filters and drop that into your Markdown editor? Every md editor and renderer I've used will pass HTML through unchanged, allowing the author to use its full syntax when they wanted to.
>>>> 
>>>> On Fri, Dec 15, 2023, 21:04 Aaron Rainbolt <arraybolt3 at gmail.com> wrote:
>>>>> I had an idea of making a primarily Markdown-centric SWORD frontend
>>>>> that would help with writing Bible studies and whatnot for
>>>>> Markdown-based platforms like Reddit or Obsidian notes. For this
>>>>> purpose, I want to parse the internal markup used by SWORD in its
>>>>> modules, and then use my own custom code to generate Markdown from
>>>>> that.
>>>>> 
>>>>> Obviously I can learn a lot about this markup by simply looking at
>>>>> modules that use it, but I do wonder, is this markup at all
>>>>> standardized? Is it documented anywhere? Does it have a name of some
>>>>> sort that I can use to find handlers and tools for it in the SWORD API
>>>>> docs?
>>>>> 
>>>>> Thanks,
>>>>> Aaron
>>>>> _______________________________________________
>>>>> sword-devel mailing list: sword-devel at crosswire.org
>>>>> http://crosswire.org/mailman/listinfo/sword-devel
>>>>> Instructions to unsubscribe/change your settings at above page
>>>> 
>>>> _______________________________________________
>>>> sword-devel mailing list: sword-devel at crosswire.org
>>>> http://crosswire.org/mailman/listinfo/sword-devel
>>>> Instructions to unsubscribe/change your settings at above page
>>> --
>>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>> _______________________________________________
>> sword-devel mailing list: sword-devel at crosswire.org
>> http://crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page