[sword-devel] Monitoring the sources for our modules?

[Jonathan Marsden]

David Haslam wrote:

> Has anyone given any thought to the ongoing monitoring of where we get the
> text (& graphics) for our modules from?

> ..., but first
> someone would have to generate a single page containing a table listing
> every module in one column, and the "supplier" URL (from the conf files) in
> another.  Making one would be a simple task for a scripting techie.

For a very quick first attempt, giving the module names and the URLs in
the TextSource field as a single list rather than two columns, here is a
one-liner:

wget -O- -q ftp://ftp.crosswire.org/pub/sword/raw/mods.d.tar.gz |tar
zxOf - |egrep "^\[|TextSource=.*http" |sed -e 's/^.*http/http/' -e
's/,.*$//'

I'll attach a file containing the output this generates.

Many of the resulting URLs seem to me to point to the top of a web site,
or a directory, rather than a specific document, which could make them
less useful than you seem to expect for your intended purpose, I would
think?

Jonathan

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: module-urls.txt
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20090906/2bd5f63f/attachment.txt>