digitalmars.D.learn - Directory Size
- Vino (38/38) Dec 05 2017 Hi All,
- Andrea Fontana (15/54) Dec 06 2017 Something like:
Hi All,
Is there any better ways to get the size of folders , The below
code perfectly works , but i need return type as
Array!(Tuple!(string, string)) rather then using the
"Result.insertBack(d);
Result.insertBack(to!string(SdFiles[].sum))" as per the below
example.
E.g:
Array!(Tuple!(string, string)) Result;
Result = (d, to!string(SdFiles[].sum));
Program:
import std.algorithm: filter, map, sum, uniq;
import std.container.array;
import std.file: dirEntries, SpanMode, isDir, isFile;
import std.stdio: writeln;
import std.typecons: tuple, Tuple;
import std.conv: to;
/******************************************/
/* Sub Function : Size of Dir List */
/******************************************/
auto mSize () {
string FFs = "C:\\Temp\\BACKUP";
Array!string Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir)).map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong((dirEntries(d,
SpanMode.depth).filter!(a => a.isFile)).map!(a => a.size));
if (SdFiles[].sum / 1024 / 1024 > 30) { Result.insertBack(d);
Result.insertBack(to!string(SdFiles[].sum)); }
}
return Result;
}
void main() {
writeln(mSize[]);
}
From,
Vino.B
Dec 05 2017
On Tuesday, 5 December 2017 at 17:21:29 UTC, Vino wrote:
Hi All,
Is there any better ways to get the size of folders , The
below code perfectly works , but i need return type as
Array!(Tuple!(string, string)) rather then using the
"Result.insertBack(d);
Result.insertBack(to!string(SdFiles[].sum))" as per the below
example.
E.g:
Array!(Tuple!(string, string)) Result;
Result = (d, to!string(SdFiles[].sum));
Program:
import std.algorithm: filter, map, sum, uniq;
import std.container.array;
import std.file: dirEntries, SpanMode, isDir, isFile;
import std.stdio: writeln;
import std.typecons: tuple, Tuple;
import std.conv: to;
/******************************************/
/* Sub Function : Size of Dir List */
/******************************************/
auto mSize () {
string FFs = "C:\\Temp\\BACKUP";
Array!string Result;
auto dFiles = Array!string ((dirEntries(FFs,
SpanMode.shallow).filter!(a => a.isDir)).map!(a => a.name));
foreach (d; dFiles[]) {
auto SdFiles = Array!ulong((dirEntries(d,
SpanMode.depth).filter!(a => a.isFile)).map!(a => a.size));
if (SdFiles[].sum / 1024 / 1024 > 30) {
Result.insertBack(d);
Result.insertBack(to!string(SdFiles[].sum)); }
}
return Result;
}
void main() {
writeln(mSize[]);
}
From,
Vino.B
Something like:
auto mSize () {
string FFs = "C:\\Temp\\BACKUP";
return dirEntries(FFs, SpanMode.shallow)
.filter!(a => a.isDir)
.map!(a => tuple(a.name,
a.dirEntries(SpanMode.depth).filter!(a=>a.isFile).map!(a =>
a.size).sum))
.filter!(a => a[1] > 1024*1024*30)
.map!(a => tuple(a[0], a[1].to!string))
.array;
}
?
Dec 06 2017
On Wednesday, 6 December 2017 at 09:16:56 UTC, Andrea Fontana wrote:On Tuesday, 5 December 2017 at 17:21:29 UTC, Vino wrote:Hi Andrea, Thank you very much, as your code is pretty good for our scenario, just one request, the above is a part of our main code where we have many such sub code and all of our sub code use the container array(std.container.array) rather than standard array(std.array), so can you please guide me on how to implement the same code using the container array. From, Vino.B.Hi All, Is there any better ways to get the size of folders , The below code perfectly works , but i need return type as Array!(Tuple!(string, string)) rather then using the "Result.insertBack(d); Result.insertBack(to!string(SdFiles[].sum))" as per the below example. E.g: Array!(Tuple!(string, string)) Result; Result = (d, to!string(SdFiles[].sum)); Program: import std.algorithm: filter, map, sum, uniq; import std.container.array; import std.file: dirEntries, SpanMode, isDir, isFile; import std.stdio: writeln; import std.typecons: tuple, Tuple; import std.conv: to; /******************************************/ /* Sub Function : Size of Dir List */ /******************************************/ auto mSize () { string FFs = "C:\\Temp\\BACKUP"; Array!string Result; auto dFiles = Array!string ((dirEntries(FFs, SpanMode.shallow).filter!(a => a.isDir)).map!(a => a.name)); foreach (d; dFiles[]) { auto SdFiles = Array!ulong((dirEntries(d, SpanMode.depth).filter!(a => a.isFile)).map!(a => a.size)); if (SdFiles[].sum / 1024 / 1024 > 30) { Result.insertBack(d); Result.insertBack(to!string(SdFiles[].sum)); } } return Result; } void main() { writeln(mSize[]); } From, Vino.BSomething like: auto mSize () { string FFs = "C:\\Temp\\BACKUP"; return dirEntries(FFs, SpanMode.shallow) .filter!(a => a.isDir) .map!(a => tuple(a.name, a.dirEntries(SpanMode.depth).filter!(a=>a.isFile).map!(a => a.size).sum)) .filter!(a => a[1] > 1024*1024*30) .map!(a => tuple(a[0], a[1].to!string)) .array; } ?
Dec 06 2017
On Wednesday, 6 December 2017 at 14:49:48 UTC, Vino wrote:Hi Andrea, Thank you very much, as your code is pretty good for our scenario, just one request, the above is a part of our main code where we have many such sub code and all of our sub code use the container array(std.container.array) rather than standard array(std.array), so can you please guide me on how to implement the same code using the container array. From, Vino.B.Just use Array! constructor. auto mSize () { string FFs = "/home/andrea/Scaricati"; return Array!(Tuple!(string,string))( dirEntries(FFs, SpanMode.shallow) .filter!(a => a.isDir) .map!(a => tuple(a.name, a.dirEntries(SpanMode.depth).filter!(a=>a.isFile).map!(a => a.size).sum)) .filter!(a => a[1] > 1024*1024*30) .map!(a => tuple(a[0], a[1].to!string)) ); }
Dec 06 2017
On Wednesday, 6 December 2017 at 15:04:55 UTC, Andrea Fontana wrote:On Wednesday, 6 December 2017 at 14:49:48 UTC, Vino wrote:Hi Andrea, I test your code, initially it error ed out stating "patch does not exist", the reason for this error is that the length of the path is more than 256 , so added the UNC path to the code as below. auto mSize () { string FFs = "C:\Temp\BACKUP"; ulong SGb = 1024 * 1024; int SizeDir = 10; return Array!(Tuple!(string,real))( dirEntries(join(["\\\\?\\", FFs]), SpanMode.shallow).filter!(a => a.isDir) .map!(a => tuple(a.name, a.dirEntries(SpanMode.depth).filter!(a=>a.isFile) .map!(a => a.size).sum)).filter!(a => a[1] > (SGb * SizeDir)) .map!(a => tuple(a[0], ((a[1].to!real)/ SGb )))); } The output of the code is as below \\?\C:\Temp\BACKUP\dir1 34.90 \\?\C:\Temp\BACKUP\dir2 36.18 So how do we print the output without UNC path C:\Temp\BACKUP\dir1 34.90 C:\Temp\BACKUP\dir2 36.18 From, Vino.BHi Andrea, Thank you very much, as your code is pretty good for our scenario, just one request, the above is a part of our main code where we have many such sub code and all of our sub code use the container array(std.container.array) rather than standard array(std.array), so can you please guide me on how to implement the same code using the container array. From, Vino.B.Just use Array! constructor. auto mSize () { string FFs = "/home/andrea/Scaricati"; return Array!(Tuple!(string,string))( dirEntries(FFs, SpanMode.shallow) .filter!(a => a.isDir) .map!(a => tuple(a.name, a.dirEntries(SpanMode.depth).filter!(a=>a.isFile).map!(a => a.size).sum)) .filter!(a => a[1] > 1024*1024*30) .map!(a => tuple(a[0], a[1].to!string)) ); }
Dec 07 2017
On Thursday, 7 December 2017 at 09:04:19 UTC, Vino wrote:On Wednesday, 6 December 2017 at 15:04:55 UTC, Andrea Fontana wrote:Hi Andrea, Was able to find a solution to the above issue by adding the replace function as below, the the code is working as expected, is there any chance of using parallel function as the file system contains about 500+ folders and the size of each folder ranges from 5GB - 500 GB so the run time of the above code is about an 2 hours. .map!(a => tuple(a[0].replace(",\\?\", ""), ((a[1].to!real)/ SGb )))); From, Vino.BOn Wednesday, 6 December 2017 at 14:49:48 UTC, Vino wrote:Hi Andrea, I test your code, initially it error ed out stating "patch does not exist", the reason for this error is that the length of the path is more than 256 , so added the UNC path to the code as below. auto mSize () { string FFs = "C:\Temp\BACKUP"; ulong SGb = 1024 * 1024; int SizeDir = 10; return Array!(Tuple!(string,real))( dirEntries(join(["\\\\?\\", FFs]), SpanMode.shallow).filter!(a => a.isDir) .map!(a => tuple(a.name, a.dirEntries(SpanMode.depth).filter!(a=>a.isFile) .map!(a => a.size).sum)).filter!(a => a[1] > (SGb * SizeDir)) .map!(a => tuple(a[0], ((a[1].to!real)/ SGb )))); } The output of the code is as below \\?\C:\Temp\BACKUP\dir1 34.90 \\?\C:\Temp\BACKUP\dir2 36.18 So how do we print the output without UNC path C:\Temp\BACKUP\dir1 34.90 C:\Temp\BACKUP\dir2 36.18 From, Vino.B[...]Just use Array! constructor. auto mSize () { string FFs = "/home/andrea/Scaricati"; return Array!(Tuple!(string,string))( dirEntries(FFs, SpanMode.shallow) .filter!(a => a.isDir) .map!(a => tuple(a.name, a.dirEntries(SpanMode.depth).filter!(a=>a.isFile).map!(a => a.size).sum)) .filter!(a => a[1] > 1024*1024*30) .map!(a => tuple(a[0], a[1].to!string)) ); }
Dec 07 2017
On Thursday, 7 December 2017 at 12:19:00 UTC, Vino wrote:On Thursday, 7 December 2017 at 09:04:19 UTC, Vino wrote:Hi All, Any update on the above request. From, Vino.B[...]Hi Andrea, Was able to find a solution to the above issue by adding the replace function as below, the the code is working as expected, is there any chance of using parallel function as the file system contains about 500+ folders and the size of each folder ranges from 5GB - 500 GB so the run time of the above code is about an 2 hours. .map!(a => tuple(a[0].replace(",\\?\", ""), ((a[1].to!real)/ SGb )))); From, Vino.B
Dec 08 2017
Vino <vino.bheeman hotmail.com> On Friday, 8 December 2017 at 19:10:09 UTC, vino wrote:On Thursday, 7 December 2017 at 12:19:00 UTC, Vino wrote:Hi Andrea, Your code is good, when compared to the performance between your code and my earlier code , my code seem to be much faster, my code execution takes 40 mins where as your code takes 2+ hours(no change in the environment) as in my code i have used parallel and tired to add the parallelism to your code but was not able to succeed. Hence i need to switch back to my code, in case if you find any method to add parallelism you your code please do let me know. Thank you very much for you help. From, Vino.BOn Thursday, 7 December 2017 at 09:04:19 UTC, Vino wrote:Hi All, Any update on the above request. From, Vino.B[...]Hi Andrea, Was able to find a solution to the above issue by adding the replace function as below, the the code is working as expected, is there any chance of using parallel function as the file system contains about 500+ folders and the size of each folder ranges from 5GB - 500 GB so the run time of the above code is about an 2 hours. .map!(a => tuple(a[0].replace(",\\?\", ""), ((a[1].to!real)/ SGb )))); From, Vino.B
Dec 09 2017