[sword-devel] Download statistics from server

[DM Smith]

On Jun 14, 2008, at 5:26 PM, Peter von Kaehne wrote:

> As part of the agreement I have made with the copyright holders of the
> Iranian Bibles - 2 still only in my own download area and on Karl's
> server as prelim modules  - I need to report from time to time back to
> them how many have been downloaded.
>
> I am doing this right now by running a grep and a wc on the apache and
> the FTP logs. This is fine, but for one problem: the ftp logs get  
> turned
> over every few months or so and old ones get deleted. This is not
> relevant yet to my requirements as none of the modules I really need  
> to
> report on are yet in the public repositories, but it will become so. I
> can overcome this by doing my own statistics more frequently.
>
> I am sure though that the more copyrighted modules we acquire the more
> reporting requirements we will get and the more importrant it will
> become that we are on top of such things.
>
> I am also sure that there must be some webtools who deal with exactly
> this - individual file download statistcis - similar, but more  
> extensive
> to the current /usage pages. If installed we could simply point any
> copyright holders to those pages and be done with it.
>
> Does anyone have any suggestions?

On the crosswire server logs are kept forever. Or at least it seems so.

I would think that a policy of 2 years should be sufficient. We can  
probably do a sweep of the logs and store the results in a database.

With regard to the crosswire server, it is easy to miss measure the  
number of downloads, because the module is downloaded in parts and  
only the d/l of the whole is meaningful. If one part is not  
downloaded, the module is incomplete. This can happen if the user  
cancels a download.

Measuring the number of times the conf is hit is not a proper measure.  
This can happen when the user uses an older app to update the module  
list (instead of d/l mods.d.tar.gz), when the user looks at info via  
the module browser or the bible tool.

I have an updated version of the script that gets the statistics for  
crosswire.org/sword in a more complete manner. It is still heuristic.  
I haven't updated it since the server logs changed.


DM