[sword-devel] Windows ntfs utf16 file or path name issue

[Troy A. Griffitts]

Regarding the NTFS UTF-8 patch, yes, just confirming that Greg has 
faithfully relayed my thoughts on this.  We have had to work around this 
issue in WinCE because MS only has UTF-16 calls available for this 
platform.  Basically we provide our own open, lseek, read, write, close 
, and dirent calls which relay the request to the real win32 api calls 
FileOpenW, or whatever and we keep around our own ordinal fd values 
which correspond to the HANDLE returned from the win32 call-- or 
something like that.  It's pretty basic and just bypasses the MS libc 
fileio calls which do the wrong thing on NTFS (and strangely work on 
FAT32).  Anyway, all this to say, I'm not planning on working this out 
for 1.6.0.

If Bibletime thinks supporting extended characters in the filesystem 
path is too important that they should wait before they release, they 
may want to link in our WinCE code.  It should come close to what we 
need to do for the desktop, but probably needs quite a bit of cleanup 
(which will benefit both windows desktop and wince projects), and can be 
had here:

http://crosswire.org/svn/swordreader/trunk/src/Dll1/winceSword/include/
http://crosswire.org/svn/swordreader/trunk/src/Dll1/winceSword/src/

If you do work on it and get things running well, please let me know and 
we can commit it to the main library source somewhere like:

http://crosswire.org/svn/sword/trunk/src/utilfuns/win32/

so windows projects can link it in.

Sorry for not getting something in for 1.6.0, but we really need a 
branch and release for projects already starting to depend on 1.6.x

	-Troy.



Greg Hellings wrote:
> On Fri, Apr 17, 2009 at 1:42 PM, Jonathan Marsden <jmarsden at fastmail.fm> wrote:
>> Eeli Kaikkonen wrote:
>>
>>> Xiphos has patched the Sword library to handle all utf16 pathnames
>>> correctly on Windows. I reproduced this with BibleTime on Windows, so it's
>>> not Xiphos alone. Greg Hellings told he doesn't want to release BT on
>>> Windows before this issue has been solved in library level without patching.
>>> What should we do?
>> I'm an outsider in some ways, but my suggested next steps would be:
>>
>> (1) Try to duplicate the issue using just the library and diatheke (so it is
>> clearly not dependent only any application outside of SWORD code provided by
>> CrossWire itself).
> 
> I'm not aware of a diatheke build for Windows, but it would not be
> difficult to replicate the issue: create a method that initializes an
> instance of SWMgr that points it to modules that are on an NTFS drive
> under a file with non-isoLatin characters in it.  Try to open that
> path and you should find, at best, it fails to open it and at worst it
> might produce more esoteric errors.  I haven't written this myself,
> but the original reporters have worked with Troy and the issue is
> known.
> 
>> (2) Submit a bug report to the SWORD Jira bugtracker.  Include a clear step
>> by step description of how to reproduce the issue, preferably on a US
>> English version of Windows XP (if it needs an international Windows OS
>> version to trigger the problem, it will be hard for many of us to work on
>> it).
> 
> It has already been made known and described.  I don't know if it's in
> Jira, although I doubt it.
> 
>> (3) Attach the patch that fixes the issue to the bug report.  Ideally, make
>> sure the patch applies cleanly to either SWORD 1.6.0RC1 or SWORD svn head.
> 
> The only current patch relies on GLib and therefore is not applicable
> to SWORD.  Troy knows how he wants to solve the issue without relying
> on GLib (using Unicode-based Windows system calls in Windows).
> 
>> (4) Test the patched library to make sure the patch does not introduce any
>> regressions, both on Windows and on other platforms.  Document what testing
>> has been done in the bug tracker.
> 
> Done for the Xiphos version.  However, it is this testing that is
> holding up Troy from making the change in HEAD/1.6.  He wants 1.6
> final released "last month" and this would, of course, take more
> testing.  So he wants to delay the change until 1.6 goes final and we
> branch 1.6 from HEAD.  Then it can be part of the rapid-fire release
> of the 1.6 branch after a few weeks while new features are going into
> HEAD.
> 
>> (5) Here on sword-devel, point to the bug tracker item and ask the SWORD
>> library developers to please get your patch into the official library
>> sources for SWORD 1.6.0 :)
> 
> It's already pretty well known around this list.  The real issue is
> just getting it into the library after 1.6.0, getting it tested by the
> Xiphos team, BibleTime, BibleCS and any other Windows applications
> that rely on the C++ library.  I think Eeli was trying to get Troy's
> confirmation on the above information, because I don't recall hearing
> Troy state on the list that the above is his plan.  I'm just relaying
> what he told me in IRC the other day.
> 
> --Greg
> 
>> Jonathan
>>
>> _______________________________________________
>> sword-devel mailing list: sword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page
>>
> 
> _______________________________________________
> sword-devel mailing list: sword-devel at crosswire.org
> http://www.crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page