[sword-devel] installmgr (and xiphos) crashes (svn 2831)

[Greg Hellings]

On Wed, Jun 26, 2013 at 8:35 AM, Jaak Ristioja <jaak at ristioja.ee> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 26.06.2013 16:25, Greg Hellings wrote:
> > bserver directory listing output (which is what I think it does).
> >
> > V E R Y   V E R Y   B A D   P R A C T I C E ! ! !
> >
> >
> > Parsing server output is bad practice? How in the world did you
> > reach that over-zealous conclusion?
> >
> > --Greg
>
> I didn't. It is not safe to assume that web servers provide similar
> output for directory listings. They might even do AJAX or something
> which might only be reasonably interpreted by human (if at all:).
>
> Example: The current code doesn't work when links start with
>   <a id="x" href="
> because it only searches for "<a href=". Even parsing the HTML
> properly would not be enough in some cases.
>

This is different than "V E R Y   V E R Y   B A D   P R A C T I C E ! ! !"

Since the code was originally written and presented with the caveat that it
only works on the default Apache listings and has only been tested against
CrossWire and we make no guarantees about it working elsewhere (although I
have successfully used it on my own default Apache install), it is far from
the horrible practice you make it out to be.


>
> Therefore, it is not reasonable to try to parse the directory listing
> output of the webserver. Instead, better mechanisms should be used,
> e.g. custom-format index files, e.g. with a format like "one
> filename/URL per line, separated by '\n' characters.". Sword would
> just need to download a single file (instead of the webserver
> directory output listing), and parse that.
>

It is by design that we do not require this behavior. Some people wanted
it, as it would do exactly as you say, but one of the goals of the
FTPTransport is to make a minimum barrier to entry. As such, it requires
nothing more than pointing an FTP server at the root of a directory it has
read access to. Likewise the same requirement was placed on an HTTPTranport
(which was initially only put together by Nic because FTP was impossible
from iOS devices). We talked about having an index file like you suggest,
about requiring WebDAV, or something similar. But the choice was made for
simplicity (plus, it was "good enough" to fulfill the requirements of
PocketSword being unable to leverage FTP).

Now our list has expanded to include FTP, HTTP, HTTPS and SFTP. Everyone
agrees that the HTTP parsing is very much not ideal, but it does what it
needs to, and if you would like to improve it to do so better, you're free
to put in patches. I've made the patches for HTTPS and SFTP in the interest
of Wycliffe's requirements for security and access control. However, the
SFTP patch exposed an issue with our assumption of username-password
authentication techniques as it will also attempt private key
authentication if no username-password are given, but we have no way of
specifying the private key with the current InstallMgr configuration system
so it will only try for ~/.ssh/id_rsa.pub.

If you would like to improve it, without breaking PocketSword's reliance on
the functionality, feel free to submit patches.

--Greg


>
> This is much easier to parse and handle error conditions, no need to
> try to interpret HTML, filter links etc...
>
> Blessings,
> Jaak
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.20 (GNU/Linux)
>
> iQgcBAEBAgAGBQJRyu4qAAoJEEqsYmEt1rCO8X0//1TklYXolDw4Izp67jNUz+UG
> bA3D9+1cCQQ6+3GXm5qHDLSDIwhrY34s7S9bEf2NXZgna294PK+zTmZ0soruiV0O
> iwxp2AKX7P4vz0Or+3qfkX0g+1PEInd2I34uV21f5lsnY8p7ZmGlfxMKpAvO5my9
> MT0QnNupsbt8jWitmkVH2Ivd1iVUHcU+XU9MC66VkufTorIZqYz+yPT5Q9mAeNoq
> cph6m4tWGNMknM4EmOQderD0BjzxQ7COWFdCNcxwV6z+VKwzCXdfOyi3LH2RQjHb
> puRe7UMOUbOGr63h/w8dRuOaoQDF0EXR6E9q/W22yauQoiLNTwhu5Y19SwXh2MjN
> r8RJpfLxqEvw7iFVd+O0O6y4gdjxhGV61vHMLe5iftni9FZeRfq6s8JxOt1QCrFz
> j0s3ZuWSkDb1B3Uefmx2T6uQXOWKIFKvK7eHPSKtNFb+wGNBPzB0kEZCfaf3jQ4b
> yyY1Ff2YoIAqWiyalWwqCxT8HXGEQCtZOJt6vG7Quow/7wpoawM9uH+A6ijgJ5ZH
> YLtPZBqVOmbTi4fpwVMipjjSw9PfSfOsfVlV6JabVQv9w2zH76stXvMDM+C0/aiT
> staXI7UD4a9MuyyMxc/fYsbjGzx5zyzyGPhVd4o73ZZ+PBGzW/sePHTnftWYdtsx
> BWcmuPoBohF07gCo20g932wIsIpShM+TztYaozYOfJzuQKX5LfqKeKLss8rL8JPZ
> H3FJHp12a0+DcdGzGfZ4/Vkgb5NzCv8GXoNLvG0R5Rgq3tbQqPc2vT4lt+fgXr6F
> ecRhdso5Mw7MEfU08RCyPczoeU9Zx5EBadIYjrBHsAl7argkjEgUEWW1BC/5WBO3
> G999NhkZuyeLMozzpstEcGUkOfffx/k/E+lnTdcbgnZmHi9j4F3blgxPTboyuY/t
> chm4FYkxEXjObHcTXbIa1i3hhnLWxgjHTzWE6pYWhg4S9uJW2DLGPrssMP2ehw4Q
> VGYLVIcRLap6a53EVnPbkkKE/TZYmHXzwM2NUfE8QgceDBzBgeFPIFh7nFw7HZuk
> RPtuX11RQmoAD9Np6hr9NlB98Mx0SC11NUcn62wjiuHgK1unPj4KfdWHmTH3v4iD
> SouXy9h2Rnj6N/mth6nEvjmETPvRFhtnUIdkNwXyu8uXsYWIwIURwy/qGxoAYBhy
> 7S4PRqxwh4RSemuYvCeTix462ARekq9ANcGKSEZNi46JpBLHNLXDaoJCr1Sj5lCe
> rsClTMcqtPxj5Z9uvyHwp+6q4G0gkwSoGuD3Z77o6Hg0/pSottarjc+y9mW2srWX
> eaVvbccpgwDsW9hn5JJxNQ9o5xRSrkMrmzM0GuR/Zl44zsXWIe9hC1nxIh4v8L+L
> tWtKgrv/GcyGb/yRwzGf/17OpyHJB7rtvarQLi+pyzWQpEPt0Pxp5bEGpjxJT+a7
> cilGKPBXYH0MEmrq75bqlGVwtVqSnBPN9N8H7iCY0eF4EkSPYGhACC7YYD58ziQB
> yY+AaAJz0A7aqvUAhv6K6s+5uXmugaSvXYYOo5kS6EiM5SdjCitLpt1THDJ1EHPT
> nA+USbnsP3BMML6g3FRKQmln7vrmlaY5ofRvi9VfVe7P0V+2l+UJ5bhhMsZrlB+p
> loH/qItGMNzIZeu1QQRbgWuVsm20/QdH0KIH+dOw7Geid/i0IH2kW48PUOUj2WeJ
> noGD3Un+oH7UvQx1zRZEN4Pa4s4bn8ryI578qn6laQn39hfe2TqjXgrfMIcu8omg
> lyk7leYWl/jNy6DhlNuiUaAARWJ9cMkYI0L7JoqoBZ+vwBB1H6U9I41R5Y/ZNIbk
> frJbnG9L1aQqSkwOmlkaunhDOyYShIDFxMQlI4HjyFLMnMKRHJFKCMDyw2Uobn5K
> nA09C6sdiPkvdBv/EzyUOhkU5iBVYUhDw+DrFCOoWIyeqwHNtGqLz2EKE7nNgYue
> g9igpFeFmdMPlw9jIicEm+rmRIlYNwlUBxfufkEpMWzhNyPzt+uSKGqb7Fzv4BQw
> VlHnwhElYMnWLu14R2SPuPw9UABadA+ejMYiiColH2bz+yhfCyNkOfESl6OZc0z8
> vwcNR3eVeHRwypOA9gL0QCEaNnixqVw770LASolx+0qw8ftOCJMks1KaBhYPTQAH
> EawB1BgY/38cZCwcSYR0nLq+Wnv9V7TTmpBDQ3evwR9DTmppmTWbTrJadtdjp0H5
> KZoTuxssEVjmrAhIwIq8cUq9TlS1KTk+9kn2lHm7W18MrVab0d2Ka6A35mZFirkG
> g24d8DzuClyGn3NPVAvYKF9OM22tBG1XOac+7/kLSeuKekrdYbGLTQxiAbQgTe36
> ixiJXx5LvUSS/SBkQWJ8GaQAuH0OBHQxKHm46uZkh82llS0KZVEK/GhHvGi9+zcc
> 7kX76EGPj7eJI5H+LZKb9tV9of/36CVCTOsw7HNCW4dl5dBKeU3Ej1/C9FiCqDa3
> 7+tkSbEuGU0BsgEj1RZVeWD+hhrEL9TJ8Uz/ZzEjl0VTh8Ahn9u+eEWN5SntUe9b
> nhmjphXtq4/Yo5yKZbhRu8eOKaVcX8pVMfgjDbYz8GX6OTZiJcNWy/Z02pyJhUmc
> NeNjh4iqVt+I1M73+B63zx39XPQtrWGa0UqGKBssC51ed56/dHYRiIKAElWUplO9
> vZDyvZ2iVIjtc8otiI96GFYStyYBF0630Ya8bwZoxbLd13/YxskxEqbHqmAPWh1Q
> s7UXb9jv4LpK43n0Zx9C
> =oZdy
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20130626/40193b4c/attachment-0001.html>