www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Assistance with >~95% CPU usage when calling pread64 when using LUKS

reply abraunegg <alex.braunegg gmail.com> writes:
Hi all,

I was wondering if I could seek some guidance on tracking an 
issue down.

As the current maintainer of the Linux OneDrive client 
(https://github.com/abraunegg) I have two open issues which are 
both high CPU, on Ubuntu, where both file systems reside on a 
LUKS encrypted volume.

* https://github.com/abraunegg/onedrive/issues/394
* https://github.com/abraunegg/onedrive/issues/404

Essentially, from the strace, it appears that the application is 
spending nearly all of it's time calling pread64.

Currently I am also not able to replicate this issue locally.

Is there any sort of extra debugging that I could do to 
understand what else is going on, or can you suggest any code 
changes which may improve the reading of files?

Any suggestions are greatly appreciated.
Mar 10
parent reply Boris-Barboris <ismailsiege gmail.com> writes:
On Sunday, 10 March 2019 at 19:59:40 UTC, abraunegg wrote:
 Is there any sort of extra debugging that I could do to 
 understand what else is going on, or can you suggest any code 
 changes which may improve the reading of files?
I would think about finding a better solution than this one: https://github.com/abraunegg/onedrive/blob/master/src/util.d#L140
Mar 10
parent reply abraunegg <alex.braunegg gmail.com> writes:
On Sunday, 10 March 2019 at 20:55:16 UTC, Boris-Barboris wrote:
 On Sunday, 10 March 2019 at 19:59:40 UTC, abraunegg wrote:
 Is there any sort of extra debugging that I could do to 
 understand what else is going on, or can you suggest any code 
 changes which may improve the reading of files?
I would think about finding a better solution than this one: https://github.com/abraunegg/onedrive/blob/master/src/util.d#L140
Thanks - yes that function / check could be better I agree, however that is not called when the system is simply checking the status of any changed files & folders. That is only used when checking if a 'new file' should be uploaded. In the issues posted, it simply seems to be the checking of each and every file / folder when there is zero rate of change - the only thing here is the enumeration of DirEntry items.
Mar 10
parent reply Boris-Barboris <ismailsiege gmail.com> writes:
On Monday, 11 March 2019 at 03:43:19 UTC, abraunegg wrote:
 Thanks - yes that function / check could be better I agree, 
 however that is not called when the system is simply checking 
 the status of any changed files & folders. That is only used 
 when checking if a 'new file' should be uploaded.

 In the issues posted, it simply seems to be the checking of 
 each and every file / folder when there is zero rate of change 
 - the only thing here is the enumeration of DirEntry items.
Sorry, I skimmed through the issues quickly. Given the sparsity of info in the logs, I would be caughtious in assuming the program internal state. I can recommend trying to compile an easy to use instruction for "callgrind" trace generation (there is a good page on dlang wiki about it), wich, given a debug build of a program, should give you an insight about what's happening. I don't think there's much use of statically analyzing a source code, you should try to extract more runtime info from the issue reporters.
Mar 11
parent abraunegg <alex.braunegg gmail.com> writes:
On Monday, 11 March 2019 at 08:29:56 UTC, Boris-Barboris wrote:
 Sorry, I skimmed through the issues quickly. Given the sparsity 
 of info in the logs, I would be caughtious in assuming the 
 program internal state.
Agreed there - I have a feeling it is related to the use of encryption & the overheads with that.
 I can recommend trying to compile an easy to use instruction 
 for "callgrind" trace generation (there is a good page on dlang 
 wiki about it), wich, given a debug build of a program, should 
 give you an insight about what's happening.

 I don't think there's much use of statically analyzing a source 
 code, you should try to extract more runtime info from the 
 issue reporters.
Thanks - will look into that.
Mar 11