www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Downloading a file and showing progress via curl.

reply BoQsc <vaidas.boqsc gmail.com> writes:
Hello everyone,
I found this snippet on 
https://dlang.org/phobos/std_net_curl.html#.HTTP


 import std.net.curl : HTTP;
 import std.stdio : writeln;
 
 void main()
 {
     auto http = HTTP();
     // Track progress
     http.method = HTTP.Method.get;
     http.url = 
 "https://upload.wikimedia.org/wikipedia/commons/5/53/Wikipedia-logo-en-big.png";
     http.onReceive = (ubyte[] data) { return data.length; };
     http.onProgress = (size_t dltotal, size_t dlnow, size_t 
 ultotal, size_t ulnow) {
         writeln("Progress ", dltotal, ", ", dlnow, ", ", 
 ultotal, ", ", ulnow);
         return 0;
     };
     http.perform();
 }


This snippet is showing Download Progress in bytes, but I'm 
unsure how to save the
downloaded file into filesystem after download is completed.
Aug 20 2019
next sibling parent reply Daniel Kozak <kozzi11 gmail.com> writes:
On Tue, Aug 20, 2019 at 1:46 PM Daniel Kozak <kozzi11 gmail.com> wrote:

 On Tue, Aug 20, 2019 at 1:40 PM BoQsc via Digitalmars-d-learn
 <digitalmars-d-learn puremagic.com> wrote:

 Hello everyone,
 I found this snippet on
 https://dlang.org/phobos/std_net_curl.html#.HTTP


 import std.net.curl : HTTP;
 import std.stdio : writeln;

 void main()
 {
     auto http = HTTP();
     // Track progress
     http.method = HTTP.Method.get;
     http.url =
 "https://upload.wikimedia.org/wikipedia/commons/5/53/Wikipedia-logo-en-big.png";
     http.onReceive = (ubyte[] data) { return data.length; };
     http.onProgress = (size_t dltotal, size_t dlnow, size_t
 ultotal, size_t ulnow) {
         writeln("Progress ", dltotal, ", ", dlnow, ", ",
 ultotal, ", ", ulnow);
         return 0;
     };
     http.perform();
 }


 This snippet is showing Download Progress in bytes, but I'm
 unsure how to save the
 downloaded file into filesystem after download is completed.


 You just need to save data in onReceive callback


For that you can use https://dlang.org/phobos/std_file#append
Aug 20 2019
next sibling parent BoQsc <vaidas.boqsc gmail.com> writes:
On Tuesday, 20 August 2019 at 11:51:03 UTC, Daniel Kozak wrote:

 For that you can use https://dlang.org/phobos/std_file#append


Thank you, seems to work.


 import std.net.curl : HTTP;
 import std.stdio    : writeln;
 import std.file     : append;
 
 void main()
 {
     auto http = HTTP();
     // Track progress
     http.method = HTTP.Method.get;
     http.url = 
 "https://upload.wikimedia.org/wikipedia/commons/5/53/Wikipedia-logo-en-big.png";
     http.onReceive = (ubyte[] data) {
         append("Wikipedia-logo-en-big.png", data);
         return data.length;
     };
     http.onProgress = (size_t dltotal, size_t dlnow, size_t 
 ultotal, size_t ulnow) {
         writeln("Progress ", dltotal, ", ", dlnow, ", ", 
 ultotal, ", ", ulnow);
         return 0;
     };
     http.perform();
 }
Aug 20 2019
prev sibling parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Tuesday, 20 August 2019 at 11:51:03 UTC, Daniel Kozak wrote:

 For that you can use https://dlang.org/phobos/std_file#append


Don't do that. It will reopen and close the file on every 
received chunk. Not only is it slow, but if the file is 
renamed/moved/deleted while the download is occurring, the file 
will be corrupted. The same will happen if you run the program 
twice, if you don't clean up.

The correct way is to open a File once, then use rawWrite for 
every received chunk.
Aug 20 2019
prev sibling parent Daniel Kozak <kozzi11 gmail.com> writes:
On Tue, Aug 20, 2019 at 1:40 PM BoQsc via Digitalmars-d-learn
<digitalmars-d-learn puremagic.com> wrote:

 Hello everyone,
 I found this snippet on
 https://dlang.org/phobos/std_net_curl.html#.HTTP


 import std.net.curl : HTTP;
 import std.stdio : writeln;

 void main()
 {
     auto http = HTTP();
     // Track progress
     http.method = HTTP.Method.get;
     http.url =
 "https://upload.wikimedia.org/wikipedia/commons/5/53/Wikipedia-logo-en-big.png";
     http.onReceive = (ubyte[] data) { return data.length; };
     http.onProgress = (size_t dltotal, size_t dlnow, size_t
 ultotal, size_t ulnow) {
         writeln("Progress ", dltotal, ", ", dlnow, ", ",
 ultotal, ", ", ulnow);
         return 0;
     };
     http.perform();
 }


 This snippet is showing Download Progress in bytes, but I'm
 unsure how to save the
 downloaded file into filesystem after download is completed.


You just need to save data in onReceive callback
Aug 20 2019