[sword-devel] Monitoring the sources for our modules?

[Jonathan Marsden]

David Haslam wrote:

> I am impressed, even though the line is somewhat obscure to the uninitiated.
> I suppose it would be "icing on the cake" to have the output sorted?  ;>}

No problem.  Just add |sort to the end of the line :)

> We should do the same for those in the beta repo.

> The checking for broken links using Xenu's LinkSleuth would be a very useful
> start.
> 
> For that, we'd need to pipe the script output to a standalone web-page
> somewhere on crosswire.org,
> but one with no links back to anything else on our server. 

OK, so we need to convert the text into HTML, so the URLs show up as
links, and we need a place to store the results... which is the kind of
think that ~/public_html subdirectories are for... so we tweak the
(growing!) one-liner, and we get:

http://crosswire.org/~jmarsden/module-urls.html and
http://crosswire.org/~jmarsden/beta-module-urls.html

These were generated by

wget -O- -q ftp://ftp.crosswire.org/pub/sword/raw/mods.d.tar.gz |tar
zxOf - |egrep "^\[|TextSource=.*http" |sed -e 's/^.*http/http/' -e
's/,.*$//' |tr '\n' ' ' |sed -e 's/\(http[^ ]\+ \)/\1\n/g' |sed -e
's/^.*\(\[.*\]\)/\1/' |head --lines=-1 |sort |sed -e 's/\(http[^
]\+\)/<a href="\1">\1<\/a><br>/' >~jmarsden/public_html/module-urls.html

and

wget -O- -q ftp://ftp.crosswire.org/pub/sword/betaraw/mods.d.tar.gz |tar
zxOf - |egrep "^\[|TextSource=.*http" |sed -e 's/^.*http/http/' -e
's/,.*$//' |tr '\n' ' ' |sed -e 's/\(http[^ ]\+ \)/\1\n/g' |sed -e
's/^.*\(\[.*\]\)/\1/' |head --lines=-1 |sort |sed -e 's/\(http[^
]\+\)/<a href="\1">\1<\/a><br>/'
>~jmarsden/public_html/beta-module-urls.html

respectively.  Now you (or anyone else) can run whatever web-oriented
tools you want over them!

Jonathan