[sword-devel] Project "Free Scriptures" started

Wed Feb 26 03:50:23 MST 2014

Sorry for the top-post-reply, but here it is, so I guess I'm not all that sorry ;)

The main bit of code you are referring to (parsing the HTML) is my code. There is other code that parses the return from an FTP server, which is ancient code. My code is (relatively) new, only about 3 years old? (I'm sure you can look it up?)
I agree it is completely a hack. I have had no time to fix it, but TBH, when I do "fix" it I will be ripping curl out of PocketSword and using native iOS stuff and will do all downloads that way. (Currently I download various bits using the build-in SWORD methods & various bits using native iOS Obj-C methods.)

I agree that switching to DVCS is a sane move & that sticking with SVN is like shooting yourself in the foot. However, it seems like it's never going to change, so I'm not going to fight that battle (insert comment about loosing battles in order to win the war, and the "war" is producing excellent software for iOS, which I'm actually currently loosing, but that has nothing to do with CrossWire and everything to do with myself and lack of time right now).

Backstory (I tell my students at school stories all the time and, sadly, they always pay more attention to me in the classroom when I pre-empt what I'm about to say with saying it's a story from my past): when I took over PocketSword, there was an SVN repo for it. If you go look for it you'll see that I never did a single commit to it. I pulled and forked & threw it in Mercurial (doesn't matter what I threw it in except to say it was a DVCS). The bonus tools I get by having it (freely) hosted on BitBucket are awesome and I get notifications when people fork it & submit a pull-request.

FWIW, I have given up on submitting patches because they appear to get lost. They may just be rejected, ignored or simply lost, but the appearance is that they generally aren't worth doing. And when I'm asked to resubmit them I generally don't bother. Using something where there are pull-requests, such as Jaak has suggested, would mean that these aren't lost. Also, the appearance will also be the reality because you can look up the request, see if it's still open, rejected or commented up and then know where the request is at in the merge process. But in this scenario I can be sure it's not lost as long as I can see it there. :)

Anyway, the powers that be have made their decision. I have made my decision with PocketSword. Others are free to make their decisions with their front-ends. Actually, you can look at my current patched version of SWORD (so, basically, I maintain a fork of SWORD for PS) if you like? I don't bother submitting patches anymore but instead patch my fork whenever SWORD gets updated. It's easier. :P Plus, on a closed platform such as iOS, someone isn't going to come along and switch the version of libsword on you & break everything...  ;)

Ok, I've said enough. :)

Thanks, ybic
	nic...  :)

ps: complete sidetrack & don't reply to this thread to mention this, cause I don't wanna hijack this thread: so much amusement in that other thread! It made me smile, especially some of the fun replies. :) Thanks for the joy you guys gave me. :)

On 26 Feb 2014, at 8:04 pm, Jaak Ristioja <jaak at ristioja.ee> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi, Jonathan!
> 
> On 25.02.2014 15:49, Jonathan Morgan wrote:
>> I've been around CrossWire for quite a few years without actually 
>> being part of the official group, so I think my opinion might be 
>> helpful here as I don't have any turf to defend (CrossWire 
>> regulars: please chip in if I'm misrepresenting anything).
> 
> Thank you for your response which I really appreciate.
> 
>> I think it's perfectly reasonable to criticise choices such as VCS 
>> and the patch approval process. What is unreasonable is to expect 
>> project maintainers to listen just because you have made that 
>> criticism.  SWORD is a mature, feature-ful and complex library,
>> and as result I'm fairly happy with a default attitude of "if it
>> ain't broke, don't fix it".
> 
> I agree that it were unreasonable to expect maintainers to act without
> thinking. I agree that SWORD is mature, featureful and complex. And I
> agree with the attitude "if it ain't broke, don't fix it". However,
> that exactly is the problem. SWORD is broken in different ways. Please
> bear with me and let me demonstrate:
> 
> While SWORD does what it is supposed to do in 99% of its use cases,
> this does not mean it is correct. One example of this kind of
> brokenness was how it retrieved module information from web servers.
> It basically parsed the directory index information HTML page returned
> by the Apache server and failed if the directory index was not present
> or contained a bit different HTML. The code parsing it also contained
> flaws which led to invalid memory accesses. But it for 99,99% of the
> time it was never triggered (at least nobody complained). This leads
> us to a different type of brokenness.
> 
> The HTML parsing code was really a hack, simply searching for "<a
> href=" strings from the HTML and trying to extract links to the files
> of that directory. It still would not work if those links contained
> non-alphanumeric characters, e.g. a forward slash (/). So it's really
> fragile (fortunately Apache HTTPD has not changed it's default
> directory listing format much). The parsing code doesn't really
> correctly parse the HTML anyway, and is really a hack. It works
> 99,9999% of the time for us, maybe has never failed, but its a hack.
> This isn't proper parsing. This leads us to yet another type of
> brokenness.
> 
> Architectural brokenness. In this context and in long-term, it is a
> fundamentally wrong choice to try to parse HTML generated by the web
> server. I mean for a quick short-term
> (we-just-want-it-to-work-fix-it-later) hack - sure, of course. But
> this code has probably been there for ages. An architecturally better
> solution would have been to put SWORD-specific index files on the web
> server, which have strict semantics defined by SWORD and would be much
> easier and safer to parse, and more reliable. In addition,
> architectural brokenness often results in more complex code and APIs
> which are not easy to understand (especially for new developers, which
> I assume SWORD needs to attract) and not easy to fix. In my opinion,
> such brokenness leads us to yet another type of brokenness.
> 
> The SWORD project doesn't encourage such changes to be made.
> Architectural changes most often require huge changes in code. When
> refactoring for this, one likely hits several new architectural flaws
> which need to be fixed before the original can be fixed properly... A
> kind of cascade or snowball effect. And if I were to submit a large
> patch for this, it would probably get rejected. For me it would
> probably take a week to fix this (and the cascaded things) in code,
> but as a volunteer I simply do not have the several weeks or months
> trying to convince the gatekeepers of the SVN. I agree that of most
> likely the first patch ain't perfect, but messing with mailing these
> patches and all of that just too much (compared to branching,
> pull-requests and merging in DVCS). I'm sorry but I simply can't
> afford the time and effort for all this process, I too have other
> responsibilities in my life.
> 
> To sum up this comment: SWORD most likely has more (hidden) flaws
> which result from poor (or outdated) architectural decisions, which
> need big changes in code. IMHO the SWORD project is rather stagnant
> and doesn't do much encourage such changes to be made nor make the
> best decisions in favor of attracting new developers.
> 
>> I could be wrong, but my gut feel is that there are not many in the
>> different front end teams that have actually made changes to SWORD,
>> and I don't see that changes to VCS / patch approval process are
>> likely to change that.
> 
> But I think it were a step in the right direction.
> 
>> While sharing code is good in theory, trying to make changes / 
>> improvements which are acceptable to all frontends is a lot harder 
>> than just making something which will work for your personal 
>> frontend (I've tried).  As a result, unless we want to have a 
>> different fork of SWORD for every frontend, with the resulting 
>> chaos, changes to SWORD affect more people and so will be harder
>> to get in (and should have a higher quality control / review 
>> barrier).
> 
> How about we fork Sword into Sword 2 (NG) and Sword 1 (Legacy)? Lets
> keep Sword 1 as stable as possible, and do a total makeover on Sword 2
> as a software and as an open source development project? If such
> efforts don't happen inside the Sword project, then these are more
> likely to happen outside the Sword project (e.g. project "Free
> Scriptures").
> 
>> It's also unreasonable to connect these choices to the "freeness" 
>> or otherwise of the software.  As with all GPL software, you have 
>> the absolute freedom to modify it to your heart's content and to 
>> host it on a different version control system (though you will
>> have to identify how it is changed and possibly rename it).  If you
>> want those changes to go back to the main repository, then people
>> need to check that they work and that they don't break anything
>> else.  I would say most large open source projects have a fairly
>> careful review process, some more stringent than CrossWire's.
>> There's nothing special about the way CrossWire has done it.
>> Contributing code back to open source software is not a right -
>> it's a privilege, and the privilege has to be earned.
> 
> Agreed. However, please consider in this context that not getting
> Sword forked by outsiders is also becoming a privilege to be earned.
> 
> Blessings! :)
> Jaak
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (GNU/Linux)
> 
> iQgcBAEBAgAGBQJTDa4EAAoJELozJlbjIn79+hs//17U8REunrX7pi3p5r+7M4P+
> MMzGmHlYg3W/Rgn2rO75IaBdTZ7jpsvy6GQfGTvhIw2r7iWq8ItSdui+SDsMsBxK
> hMp/RTp+qqQ8pkLHv5rW+W3p2gwZKD9Ib/joBTlFtwUQEGeD6cEd7t5YKZwp0TcO
> +tlVgiCUfqmsxH1zBmLbT6gfI7koDD/ScZwKtlzbQOsZIj055WLCNGhHdSSs1jJl
> xoH+c2AJrFD4+XkG/fZcI4C3iPNETuKsaqr8bT6I2TfH/aTWxS9Nt+Tk5YAx89sn
> 7SRmRXxD6Td2b61rcdtvUEIKaprKbkFm13Qjlci7/kbX16OvTFgZL36GAmskLJ/u
> ikNBYvw62qE7nprCxv83FviKScqiuXkdAPYSljbg5i4Ouz4hp6BUAuIZgQPi/0+Q
> bGmClIQjBj04tF0Wi+PX2+P5fBclpldEsNdhKy0IlcCQsyqaHTk0Ki2ZNn+8xr+A
> +ZbVHrOU1Zl/rzbvBTl6W3iYxHEYXp7ZYTevn+puD0Bw4/6kl9ZeuuPHgb8btYfL
> gqe6/nWfwcuaphRJcQlajpHnwnhsbxyESYOMWsDI307sVY7M4WPV/+Lw6hGFy1L5
> 3KIhsP49uldcfT/J6butbpG0QtOUeyRu65ClvplO9vpXYs+kpmlUnyoa1/++EV8l
> 4mxXoeXnfetNYKb6vJmzosK4nCejfCSKPsdy7HDniHedXn4wB4rnz5r+YePi01rf
> 0rjgYZ9aaQDe5w4HGmpc1vAifQZBVnVZWtccz0G138A7Xn0v87bNfXc3Z0W199Hs
> iPLjC+sRS6Z/nUsyYY5dGf4EG7/UfCSqA0bRdA1dX/kn+9XT3XNWS5BLrQYGpMs1
> We9GJd+hbiXNAY8RGiGm3qmDA5cLoyzAklmUu81RMcA9wmu5vXjnEVdpaOTeroke
> uA9H/SU4UR8JaxamWPlBxCP4NVrNnEYCeJuIlLniC4ePmg1GHDM9zCiaGb0ZrOa+
> KAoWyaXYRvDicLjO82NVYSi8yHdOTr8oLELiFolBLoSIN+cBc+EBqzPjhGZl85HH
> ApBYfmNo6nnaXBKPoRk69EAiQsyzFetj5Sq2YSBlItfYZQYz8mgBrJN6KtigFzKf
> HKHMtntSUBPABu1pjhhNg77yB8Dt4EBqEpKErLTv9pOJXSb5fZrfm0fDXecowwlX
> 5JJcNGEI7P9gDgt3rJ5vMHHkPXJG5yK4uTlmGWSU4aymq69/x6fbeYxtGibU3YmK
> EYJ84H3zYxWO8lGh+EPRGPaB9G8unle8DCUGZb45M7rPxABXjd7OJERfOl0tScoR
> fU17V392pdDOwrMT4uamqmvDtw8SAt98UiexOG91x+rrs0vBba9JGdaKoh2Bf7uh
> Bw1+vZgXGptuYja+hSMX/q0ehj/sgkstR8gcXqA5O6h1yo4jWO7RmiDKnTBfekXU
> XdNjLU7uaSx0qPQ5aJ4Q+AEa2ID22ke01Ti5+xwe2uUsgAuhKtuZG+UNaWjdFFMH
> aMYyBpsOQ3OQcmfq1XlmCBwT4Wumag0T6eBhj3TeID6t59U/WnRFY5a8+1wFmPtM
> Fm/AsqAailLiz4q30+r7hzBNyr3TbJcoerrZ0fL95g6wwe6Au5squAG9HiwJ2OJT
> A/YrViudXkiYbAHHMancKsifL9JHko0X+s0hjsYoElYLB3IzBKWWdr67lcW6qtP0
> JwstH4nTH1DRFWFn3PiGg1cEWPKIZO8aNhg8aKpoFZXnlWuRaXyJfvv6O0EsZssq
> dszL9Xc63CC2+LPCYQN5qVrdoeq4TJox7Oe6HAY7i/655Fia0tbKShn8jZ+Ig1re
> 7AAI4L0DKOQxk+wrIiUR4aHwfymDV3YhPwrySNPaR2vrDlcAJGy6UkQazqdvbk0Y
> bxTNdoW0Ioy9IuAEnjiSnJ3bU4Z572cDoFMMJ/Iv1ocnBK+zF75vk7+FKdnvj73I
> NeJestNNsyQ9h+7/F/IQ/PeDl7gMkzRVYLdQhTTAsf90g1RStlSph1dNZlnb4aB8
> FFY2oeXU8a/2SnnErg/pnRS6wXUdPgjHN2vUddiluB7awWJwRQHumZlG2jrs3hyP
> ePRZPGMlo9x771xjTIx/KcptoiI8T9I4xieMREBPh5YvcmNHLZeGTGaxwOipzm8Z
> kcyAMrHKHnVsiCOPe+YzX/83tiNipQlFFF61nSUIkc4+2BUi0JtqUYEkVa3GrFW4
> XeepLodR+XshpWbe0KcMtJJS37LH7fpAhnv7tDQBgiu8rBfpuw2p72TA9vIoiqux
> 9pb2MGr2FNqDxxUB+Im7ozv11euWnA2ncVXZVZiDxL/2drl207uo8v69Z22JJh91
> lpMtLg5IN6LXLaqrHjSGl+KW/D7PB+EZ4xL9A9qifuePYrU3QRxVYeWPBYSeCJl/
> wd6t610k3HJNm9a88uEe/pHZB3YfPJizJ2E37H/Z0Wh53XbZPitTffpFQKL4sxVr
> xOMN1K6aHn0p61SLWAvkHT8NcjWXg0g3pCxOYOT1u11dMAkyIJXhlXGsIi8RSf5x
> TKcJMLpeeX8UG/xsjIulSyF7f09ggm2zEQGtAqbYw9PNzKOvHg82P1xONkHH+9NL
> bdYbQJwieW/06t/uW2YaH8lLiWVnUs59Aft8/uyyNSxr9RwELWCZPs6XUNbSc5cc
> S1sOrrdlrxn47WSgR7ZejSs9rMhir3YTnKVzWSBYD/nrDDYZZTNfURJuOe9+25KR
> B9J/KL+dbsTqMkh9ufq6
> =EAjj
> -----END PGP SIGNATURE-----
> 
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page