[sword-devel] Win32 FileMgr Subclass

David Haslam dfhdfh at protonmail.com
Mon Jul 27 14:32:03 EDT 2020 

Thanks Greg,

I have just downloaded the two Xiphos builds and successfully installed the x32 edition of Xiphos.

I observe afterwards that the Sword utilities all have the filedate 2020-04-26 07:26
Was this to be expected?
And is this commensurate with the issue that this new build addressed?
i.e. The scope of the fix was entirely within the SWORD API and nothing in the utilities themselves also required fixing to cope with UTF-8 filenames and paths.

While it's still fresh in my mind, I have a further question. My Win7 PC is an x64 machine.
When I first installed Xiphos therein back in 2010 there was only the 32-bit edition available.
Are there any special considerations or precautions advised for a user wishing to install the x64 edition in place of an already installed x32 edition?

Best regards,

David

Sent with [ProtonMail](https://protonmail.com) Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Monday, 27 July 2020 15:24, Greg Hellings <greg.hellings at gmail.com> wrote:

> On Mon, Jul 27, 2020 at 4:42 AM Troy A. Griffitts <scribe at crosswire.org> wrote:
>
>> That crazy cmake line certain did work to configure a cross-compile. Someone needs to write that one down somewhere.
>
> That line was taken from my RPM build script. Most of it is the standard MinGW cross-compile macro used, then a few extra options were my specific addons for Sword options.
>
>> So, I've included <wchar.h> in filemgr.cpp and this seems to resolve this for mingw. I get a bunch of .EXEs in utilities, so I think it worked.
>
> Worked beautifully for me, as well.
>
>> I'll commit, and then go back and be sure I haven't broken the build on BCB. I trust Microsoft VC++ has wchar.h available so it shouldn't break with the new change.
>>
>> And now I think we have 3 Microsoft compilers building (I hesitate to say "working" until we get confirmation that app using pristine SWORD can indeed work in paths above UTF-8 single byte on Windows.
>
> Fedora 30 and 31 cross-compile builds are here: http://dl.thehellings.com/sword/ (Xiphos still uses Fedora 30 to build the Windows versions due to needing GtkHTML). If any other Xiphos developers want to do testing, or if BibleTime does any cross compiling from Fedora. You can also find Xiphos installers building the latest Xiphos head against this latest Sword head. I'm very far from any Windows machine I can use as a test, so if anyone else has a Windows machine to test this on - preferably one with a username that includes non-ASCII characters in it - then feel free to grab that. If the BibleTime Windows builder (Gary?) wants to generate builds against the latest SVN HEAD and test in the same manner, it would be a huge help.
>
> Yes, David, this is the very latest set of Sword Utility builds you can currently find for Windows, as well. It represents the exact State Of The Art SVN HEAD built for Windows. It will be the ones installed at the root of the Xiphos binary instead of my standalone pack. And no, it's still not officially supported. 😋
>
>> Thanks for your help Greg and Tobias. It was great to have support through this long dreaded task.
>
> Thank you for the dedication to get it working and remove that patch point.
>
> --Greg
>
>> Happy Monday!
>>
>> Troy
>>
>> On 7/27/20 7:57 AM, Greg Hellings wrote:
>>
>>> If you're missing dependencies you should be able to do, "dnf builddep mingw-sword".
>>>
>>> --Greg
>>>
>>> On Mon, Jul 27, 2020 at 1:33 AM Greg Hellings <greg.hellings at gmail.com> wrote:
>>>
>>>> On Sun, Jul 26, 2020 at 4:56 PM Tobias Klein <contact at tklein.info> wrote:
>>>>
>>>>> Hi Troy,
>>>>>
>>>>> The latest version builds successfully.
>>>>>
>>>>> I created a new intermediate release of sword-build-win32 for further testing on Windows:
>>>>
>>>> It's still failing for me trying to do a cross-compile with MinGW:
>>>>
>>>> [ 6%] Building CXX object CMakeFiles/sword.dir/src/mgr/filemgr.cpp.obj
>>>> /usr/bin/i686-w64-mingw32-g++ -DCLUCENE2 -DCURLAVAILABLE -DCURLSFTPAVAILABLE -DEXCLUDEBZIP2 -DEXCLUDEXZ -DGLOBCONFPATH=\"/usr/i686-w64-mingw32/sys-root/mingw/etc/sword.conf\" -DUSEICUREGEX -DUSELUCENE -DU_USING_ICU_NAMESPACE -D_FTPLIB_NO_COMPAT -D_ICU_ -Dsword_EXPORTS @CMakeFiles/sword.dir/includes_CXX.rsp -D_ICUSWORD_ -g3 -Wall -O0 -D_ICUSWORD_ -o CMakeFiles/sword.dir/src/mgr/filemgr.cpp.obj -c /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp
>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp: In static member function 'static int sword::FileMgr::createParent(const char*)':
>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp:439:5: error: '_wmkdir' was not declared in this scope; did you mean 'mkdir'?
>>>> 439 | _wmkdir((const wchar_t *)utf8ToWChar(buf).getRawData());
>>>> | ^~~~~~~
>>>> | mkdir
>>>> make[2]: *** [CMakeFiles/sword.dir/build.make:282: CMakeFiles/sword.dir/src/mgr/filemgr.cpp.obj] Error 1
>>>> make[2]: Leaving directory '/builddir/build/BUILD/sword-1.8.900/build_win32'
>>>> make[1]: Leaving directory '/builddir/build/BUILD/sword-1.8.900/build_win32'
>>>> make[1]: *** [CMakeFiles/Makefile2:277: CMakeFiles/sword.dir/all] Error 2
>>>> make: *** [Makefile:152: all] Error 2
>>>> make: Leaving directory '/builddir/build/BUILD/sword-1.8.900/build_win32'
>>>>
>>>> Commands to do the build, from Fedora 32, include this lovely invocation of CMake from within a subfolder of the source (build_win32 for the above):
>>>> /usr/bin/cmake -DCMAKE_TOOLCHAIN_FILE=/usr/share/mingw/toolchain-mingw32.cmake -DBUILD_SHARED_LIBS:BOOL=ON -DSYSCONF_INSTALL_DIR:PATH=/usr/i686-w64-mingw32/sys-root/mingw/etc -DSHARE_INSTALL_PREFIX:PATH=/usr/i686-w64-mingw32/sys-root/mingw/share -DCMAKE_INSTALL_PREFIX:PATH=/usr/i686-w64-mingw32/sys-root/mingw -DCMAKE_INSTALL_LIBDIR:PATH=/usr/i686-w64-mingw32/sys-root/mingw/lib -DICU_CONFIG_BIN_PATH=/usr/i686-w64-mingw32/sys-root/mingw/bin -DINCLUDE_INSTALL_DIR:PATH=/usr/i686-w64-mingw32/sys-root/mingw/include -DCMAKE_VERBOSE_MAKEFILE=ON -DLIBSWORD_LIBRARY_TYPE=Shared -DSWORD_BUILD_EXAMPLES=Yes -DCMAKE_BUILD_TYPE=Debug -DICU_CONFIG_OPTS=--noverify -DCROSS_COMPILE_MINGW32=TRUE ..
>>>>
>>>> Followed by
>>>>
>>>> make
>>>>
>>>> --Greg
>>>>
>>>>> https://github.com/tobias-klein/sword-build-win32/releases/tag/v1.8.900-2020-07-26
>>>>>
>>>>> Best regards,
>>>>> Tobias
>>>>>
>>>>> On 7/26/20 8:37 PM, Troy A. Griffitts wrote:
>>>>>
>>>>>> Thanks Tobias,
>>>>>>
>>>>>> I've made these updates and should have fixed the error in filemgr.cpp on line 410. I appreciate the feedback. Please update and try this out and let me know. Thanks for testing your compiler.
>>>>>>
>>>>>> Troy
>>>>>>
>>>>>> On 7/26/20 8:25 PM, Tobias Klein wrote:
>>>>>>
>>>>>>> To address the errors below I had to add the include for <windows.h> both in src/mgr/filemgr.cpp and in src/modules/commons/zipmgr.cpp.
>>>>>>>
>>>>>>> After that I'm getting the next error:
>>>>>>>
>>>>>>> 1>C:\Users\tobi\Dev\sword-build-win32\sword\src\mgr\filemgr.cpp(410): error C2664: 'BOOL FindNextFileA(HANDLE,LPWIN32_FIND_DATAA)': cannot convert argument 2 from 'WIN32_FIND_DATAW *' to 'LPWIN32_FIND_DATAA'
>>>>>>> 1>C:\Users\tobi\Dev\sword-build-win32\sword\src\mgr\filemgr.cpp(410): note: Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Tobias
>>>>>>>
>>>>>>> On 7/26/20 4:44 PM, Troy A. Griffitts wrote:
>>>>>>>
>>>>>>>> Can one of you guys try simply including <windows.h> at the top of filemgr.cpp and see if this fixes it for you?
>>>>>>>>
>>>>>>>> On 7/26/20 4:01 PM, Tobias Klein wrote:
>>>>>>>>
>>>>>>>>> I'm getting similar error messages with Visual Studio 2019. Note that I'm also generating the make files via CMake.
>>>>>>>>>
>>>>>>>>> First couple of error messages:
>>>>>>>>>
>>>>>>>>> 2>D:\a\sword-build-win32\sword-build-win32\sword\src\mgr\filemgr.cpp(395,2): error C2065: 'WIN32_FIND_DATAW': undeclared identifier
>>>>>>>>> 2>D:\a\sword-build-win32\sword-build-win32\sword\src\mgr\filemgr.cpp(395,19): error C2146: syntax error: missing ';' before identifier 'fileData'
>>>>>>>>>
>>>>>>>>> 2>D:\a\sword-build-win32\sword-build-win32\sword\src\mgr\filemgr.cpp(395,19): error C2065: 'fileData': undeclared identifier
>>>>>>>>>
>>>>>>>>> 2>D:\a\sword-build-win32\sword-build-win32\sword\src\mgr\filemgr.cpp(398,2): error C2065: 'HANDLE': undeclared identifier
>>>>>>>>>
>>>>>>>>> 2>D:\a\sword-build-win32\sword-build-win32\sword\src\mgr\filemgr.cpp(398,9): error C2146: syntax error: missing ';' before identifier 'findIterator'
>>>>>>>>> 2>D:\a\sword-build-win32\sword-build-win32\sword\src\mgr\filemgr.cpp(398,9): error C2065: 'findIterator': undeclared identifier
>>>>>>>>> 2>D:\a\sword-build-win32\sword-build-win32\sword\src\mgr\filemgr.cpp(398,51): error C2065: 'fileData': undeclared identifier
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Tobias
>>>>>>>>>
>>>>>>>>> On 7/26/20 3:00 PM, Greg Hellings wrote:
>>>>>>>>>
>>>>>>>>>> On Sun, Jul 26, 2020 at 6:42 AM Troy A. Griffitts <scribe at crosswire.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> I've just committed the last bit for fixing the WIN32 Unicode issues. If anyone can try compiling and running Xiphos without the Xiphos patch or any other projects for Windows, and let me know if things work for them in folders which include Unicode character, I would appreciate it. I've tested BibleCS and it now works with a SWORD_PATH defined to /books/χαρις. I've tested using and installing modules in this configuration and believe all the bugs are squashed, but I would love confirmation from other projects.
>>>>>>>>>>>
>>>>>>>>>>> Thanks for any feedback,
>>>>>>>>>>>
>>>>>>>>>>> Troy
>>>>>>>>>>
>>>>>>>>>> During cross-compile I'm getting the following errors:
>>>>>>>>>>
>>>>>>>>>> [ 6%] Building CXX object CMakeFiles/sword.dir/src/mgr/filemgr.cpp.obj
>>>>>>>>>> /usr/bin/i686-w64-mingw32-g++ -DCLUCENE2 -DCURLAVAILABLE -DCURLSFTPAVAILABLE -DEXCLUDEBZIP2 -DEXCLUDEXZ -DGLOBCONFPATH=\"/usr/i686-w64-mingw32/sys-root/mingw/etc/sword.conf\" -DUSEICUREGEX -DUSELUCENE -DU_USING_ICU_NAMESPACE -D_FTPLIB_NO_COMPAT -D_ICU_ -Dsword_EXPORTS @CMakeFiles/sword.dir/includes_CXX.rsp -D_ICUSWORD_ -g3 -Wall -O0 -D_ICUSWORD_ -o CMakeFiles/sword.dir/src/mgr/filemgr.cpp.obj -c /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp
>>>>>>>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp: In static member function 'static std::vector<sword::DirEntry> sword::FileMgr::getDirList(const char*, bool, bool)':
>>>>>>>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp:395:2: error: 'WIN32_FIND_DATAW' was not declared in this scope
>>>>>>>>>> 395 | WIN32_FIND_DATAW fileData;
>>>>>>>>>> | ^~~~~~~~~~~~~~~~
>>>>>>>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp:398:2: error: 'HANDLE' was not declared in this scope
>>>>>>>>>> 398 | HANDLE findIterator = FindFirstFileW(wcharPath, &fileData);
>>>>>>>>>> | ^~~~~~
>>>>>>>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp:399:6: error: 'findIterator' was not declared in this scope
>>>>>>>>>> 399 | if (findIterator != INVALID_HANDLE_VALUE) {
>>>>>>>>>> | ^~~~~~~~~~~~
>>>>>>>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp:399:22: error: 'INVALID_HANDLE_VALUE' was not declared in this scope
>>>>>>>>>> 399 | if (findIterator != INVALID_HANDLE_VALUE) {
>>>>>>>>>> | ^~~~~~~~~~~~~~~~~~~~
>>>>>>>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp:401:35: error: 'fileData' was not declared in this scope
>>>>>>>>>> 401 | SWBuf dirEntName = wcharToUTF8(fileData.cFileName);
>>>>>>>>>> | ^~~~~~~~
>>>>>>>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp:405:49: error: 'FILE_ATTRIBUTE_DIRECTORY' was not declared in this scope
>>>>>>>>>> 405 | i.isDirectory = fileData.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY;
>>>>>>>>>> | ^~~~~~~~~~~~~~~~~~~~~~~~
>>>>>>>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp:409:40: error: 'fileData' was not declared in this scope
>>>>>>>>>> 409 | } while (FindNextFile(findIterator, &fileData) != 0);
>>>>>>>>>> | ^~~~~~~~
>>>>>>>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp:409:12: error: 'FindNextFile' was not declared in this scope
>>>>>>>>>> 409 | } while (FindNextFile(findIterator, &fileData) != 0);
>>>>>>>>>> | ^~~~~~~~~~~~
>>>>>>>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp:410:3: error: 'FindClose' was not declared in this scope; did you mean '_findclose'?
>>>>>>>>>> 410 | FindClose(findIterator);
>>>>>>>>>> | ^~~~~~~~~
>>>>>>>>>> | _findclose
>>>>>>>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp:397:17: warning: unused variable 'wcharPath' [-Wunused-variable]
>>>>>>>>>> 397 | const wchar_t *wcharPath = (const wchar_t *)wcharBuf.getRawData();
>>>>>>>>>> | ^~~~~~~~~
>>>>>>>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp: In static member function 'static int sword::FileMgr::createParent(const char*)':
>>>>>>>>>> /builddir/build/BUILD/sword-1.8.900/src/mgr/filemgr.cpp:436:5: error: '_wmkdir' was not declared in this scope; did you mean 'mkdir'?
>>>>>>>>>> 436 | _wmkdir((const wchar_t *)utf8ToWChar(buf).getRawData());
>>>>>>>>>> | ^~~~~~~
>>>>>>>>>> | mkdir
>>>>>>>>>> make[2]: *** [CMakeFiles/sword.dir/build.make:283: CMakeFiles/sword.dir/src/mgr/filemgr.cpp.obj] Error 1
>>>>>>>>>>
>>>>>>>>>> I'm assuming there's a new package or macro I need to define? On my system the WIN32_FIND_DATAW struct is defined in both minwinbase.h and shtypes.h. I'm building with MinGW which might have a different header structure than your compilers, if you're using Borland?
>>>>>>>>>>
>>>>>>>>>> --Greg
>>>>>>>>>>
>>>>>>>>>>> On 7/20/20 7:18 PM, Greg Hellings wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Sorry for the previous blank email - user error when I tried to reply:
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Jul 19, 2020 at 2:40 PM Tobias Klein <contact at tklein.info> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for giving me the background on this, Karl! I appreciate it!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is Xiphos the only frontend that has been patching Sword for this purpose? Then I suppose all other frontends suffer from this issue, huh?
>>>>>>>>>>>>
>>>>>>>>>>>> When I first encountered this patch in Xiphos I tested with BibleTime and The Sword Project for Windows and both of them do crash under these circumstances.
>>>>>>>>>>>>
>>>>>>>>>>>> Yes, other toolkits such as Qt do have wrappers for this shortcoming already, but none of the other front ends I've worked with have bothered to put in the effort to produce a patched version of Sword to fix the crash.
>>>>>>>>>>>>
>>>>>>>>>>>> --Greg
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> sword-devel mailing list:
>>>>>>>>>>>> sword-devel at crosswire.org
>>>>>>>>>>>>
>>>>>>>>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>>>>>>>>>> Instructions to unsubscribe/change your settings at above page
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> sword-devel mailing list: sword-devel at crosswire.org
>>>>>>>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>>>>>>>>> Instructions to unsubscribe/change your settings at above page
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> sword-devel mailing list:
>>>>>>>>>> sword-devel at crosswire.org
>>>>>>>>>>
>>>>>>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>>>>>>>> Instructions to unsubscribe/change your settings at above page
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> sword-devel mailing list:
>>>>>>>>> sword-devel at crosswire.org
>>>>>>>>>
>>>>>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>>>>>>> Instructions to unsubscribe/change your settings at above page
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> sword-devel mailing list:
>>>>>>>> sword-devel at crosswire.org
>>>>>>>>
>>>>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>>>>>> Instructions to unsubscribe/change your settings at above page
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> sword-devel mailing list:
>>>>>>> sword-devel at crosswire.org
>>>>>>>
>>>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>>>>> Instructions to unsubscribe/change your settings at above page
>>>>>>
>>>>>> _______________________________________________
>>>>>> sword-devel mailing list:
>>>>>> sword-devel at crosswire.org
>>>>>>
>>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>>>> Instructions to unsubscribe/change your settings at above page
>>>>>
>>>>> _______________________________________________
>>>>> sword-devel mailing list: sword-devel at crosswire.org
>>>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>>>> Instructions to unsubscribe/change your settings at above page
>>>
>>> _______________________________________________
>>> sword-devel mailing list:
>>> sword-devel at crosswire.org
>>>
>>> http://www.crosswire.org/mailman/listinfo/sword-devel
>>> Instructions to unsubscribe/change your settings at above page
>>
>> _______________________________________________
>> sword-devel mailing list: sword-devel at crosswire.org
>> http://www.crosswire.org/mailman/listinfo/sword-devel
>> Instructions to unsubscribe/change your settings at above page
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.crosswire.org/pipermail/sword-devel/attachments/20200727/8545f4f9/attachment-0001.html>

More information about the sword-devel mailing list