www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - how to use curl to download a file

reply =?iso-8859-1?b?RnLpZOlyaWM=?= Galusik <fredg salixosnospam.org> writes:
Hi,

As the curl documentation is a little bit ...wow.
http://www.digitalmars.com/d/2.0/phobos/etc_c_curl.html

Do someone have a simple example on how to download a simple file ?

Thank you.

++
Nov 01 2011
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On 11/2/2011 3:20 AM, Frédéric Galusik wrote:

 Hi,

 As the curl documentation is a little bit ...wow.
 http://www.digitalmars.com/d/2.0/phobos/etc_c_curl.html

 Do someone have a simple example on how to download a simple file ?

 Thank you.

 ++


I don't think we should expect detailed documentation for modules in 
etc.c. These are interfaces to existing C libraries, not Phobos-specific 
modules. There's no need to duplicate existing documentation. If you 
want to learn how to use libcurl, the place to look is the libcurl homepage.


http://curl.haxx.se/libcurl/
Nov 01 2011
parent reply Ary Manzana <ary esperanto.org.ar> writes:
On 11/1/11 11:49 PM, Mike Parker wrote:

 On 11/2/2011 3:20 AM, Frédéric Galusik wrote:

 Hi,

 As the curl documentation is a little bit ...wow.
 http://www.digitalmars.com/d/2.0/phobos/etc_c_curl.html

 Do someone have a simple example on how to download a simple file ?

 Thank you.

 ++


 I don't think we should expect detailed documentation for modules in
 etc.c. These are interfaces to existing C libraries, not Phobos-specific
 modules. There's no need to duplicate existing documentation. If you
 want to learn how to use libcurl, the place to look is the libcurl
 homepage.


 http://curl.haxx.se/libcurl/


Wrong.

If you want programmers to use D, give them the solution, now.

It's not that hard to put a little example at the top of the page 
showing a basic usage.
Nov 02 2011
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Ary Manzana:


 Wrong.
 
 If you want programmers to use D, give them the solution, now.
 
 It's not that hard to put a little example at the top of the page 
 showing a basic usage.


That wall of links at the top of the page is useless. I don't want to use that
C API. This is how you download a file in Python (urllib2 is fully inside the
Python standard library) and save it in a local file:

import urllib2
data = urllib2.urlopen("http://server.com/file.html").read()
open('file.html', 'w').write(data)

I think in Clojure or Rebol (http://en.wikipedia.org/wiki/REBOL ) languages
it's even simpler.

Bye,
bearophile
Nov 02 2011
parent reply Jesse Phillips <jessekphillips+d gmail.com> writes:
etc.c.curl is meant for those that know curl and wish to use it. etc.curl 
has been delayed but is intended for those that wish to access the 
internet.
Nov 02 2011
parent reply "Nick Sabalausky" <a a.a> writes:
"Jesse Phillips" <jessekphillips+d gmail.com> wrote in message 
news:j8rj3c$uc2$1 digitalmars.com...

 etc.c.curl is meant for those that know curl and wish to use it. etc.curl
 has been delayed but is intended for those that wish to access the
 internet.


In the meantime, the proposed etc.curl is at least available somewhere to 
grab and use, isn't it?
Nov 02 2011
parent reply Graham Fawcett <fawcett uwindsor.ca> writes:
On Wed, 02 Nov 2011 11:10:17 -0400, Nick Sabalausky wrote:


 "Jesse Phillips" <jessekphillips+d gmail.com> wrote in message
 news:j8rj3c$uc2$1 digitalmars.com...

 etc.c.curl is meant for those that know curl and wish to use it.
 etc.curl has been delayed but is intended for those that wish to access
 the internet.


 In the meantime, the proposed etc.curl is at least available somewhere
 to grab and use, isn't it?


Good point. Probably better to grab that than my binding (which I 
suggested in another post).

https://github.com/jcd/phobos/blob/curl-wrapper/etc/curl.d

docs: http://freeze.steamwinter.com/D/web/phobos/etc_curl.html

reference: http://www.mail-archive.com/digitalmars-d puremagic.com/
msg63605.html

Best,
Graham
Nov 02 2011
next sibling parent Graham Fawcett <fawcett uwindsor.ca> writes:
On Wed, 02 Nov 2011 15:23:36 +0000, Graham Fawcett wrote:


 On Wed, 02 Nov 2011 11:10:17 -0400, Nick Sabalausky wrote:
 

 "Jesse Phillips" <jessekphillips+d gmail.com> wrote in message
 news:j8rj3c$uc2$1 digitalmars.com...

 etc.c.curl is meant for those that know curl and wish to use it.
 etc.curl has been delayed but is intended for those that wish to
 access the internet.


 In the meantime, the proposed etc.curl is at least available somewhere
 to grab and use, isn't it?


 
 Good point. Probably better to grab that than my binding (which I
 suggested in another post).
 
 https://github.com/jcd/phobos/blob/curl-wrapper/etc/curl.d
 
 docs: http://freeze.steamwinter.com/D/web/phobos/etc_curl.html
 


Hm, that 'docs' link is broken, but the github link is still good.

Graham


 reference: http://www.mail-archive.com/digitalmars-d puremagic.com/
 msg63605.html
 
 Best,
 Graham
Nov 02 2011
prev sibling parent reply =?iso-8859-1?b?RnLpZOlyaWM=?= Galusik <fredg salixosnospam.org> writes:
Le Wed, 02 Nov 2011 15:23:36 +0000, Graham Fawcett a écrit :


 On Wed, 02 Nov 2011 11:10:17 -0400, Nick Sabalausky wrote:
 

 "Jesse Phillips" <jessekphillips+d gmail.com> wrote in message
 news:j8rj3c$uc2$1 digitalmars.com...

 etc.c.curl is meant for those that know curl and wish to use it.
 etc.curl has been delayed but is intended for those that wish to
 access the internet.


 In the meantime, the proposed etc.curl is at least available somewhere
 to grab and use, isn't it?


 
 Good point. Probably better to grab that than my binding (which I
 suggested in another post).
 
 https://github.com/jcd/phobos/blob/curl-wrapper/etc/curl.d
 
 docs: http://freeze.steamwinter.com/D/web/phobos/etc_curl.html
 
 reference: http://www.mail-archive.com/digitalmars-d puremagic.com/
 msg63605.html
 
 Best,
 Graham


Yes, etc.curl is well documented and is really more understandable
Thank you for this wrapper.
Is there a chance that it will become std.curl one day ?

Regards.

++
Nov 02 2011
parent Jonas Drewsen <jdrewsen nospam.com> writes:
Den 02-11-2011 17:17, Frédéric Galusik skrev:

 Le Wed, 02 Nov 2011 15:23:36 +0000, Graham Fawcett a écrit :


 On Wed, 02 Nov 2011 11:10:17 -0400, Nick Sabalausky wrote:


 "Jesse Phillips"<jessekphillips+d gmail.com>  wrote in message
 news:j8rj3c$uc2$1 digitalmars.com...

 etc.c.curl is meant for those that know curl and wish to use it.
 etc.curl has been delayed but is intended for those that wish to
 access the internet.


 In the meantime, the proposed etc.curl is at least available somewhere
 to grab and use, isn't it?


 Good point. Probably better to grab that than my binding (which I
 suggested in another post).

 https://github.com/jcd/phobos/blob/curl-wrapper/etc/curl.d

 docs: http://freeze.steamwinter.com/D/web/phobos/etc_curl.html

 reference: http://www.mail-archive.com/digitalmars-d puremagic.com/
 msg63605.html

 Best,
 Graham


 Yes, etc.curl is well documented and is really more understandable
 Thank you for this wrapper.
 Is there a chance that it will become std.curl one day ?


Andrei asked me to rename it to std.curl so I believe it will yes.
Still working on it but not at max speed since I'm quite busy at work 
currently.

/Jonas
Nov 02 2011
prev sibling parent Mike Parker <aldacron gmail.com> writes:
On 11/2/2011 9:43 PM, Ary Manzana wrote:

 On 11/1/11 11:49 PM, Mike Parker wrote:

 On 11/2/2011 3:20 AM, Frédéric Galusik wrote:

 Hi,

 As the curl documentation is a little bit ...wow.
 http://www.digitalmars.com/d/2.0/phobos/etc_c_curl.html

 Do someone have a simple example on how to download a simple file ?

 Thank you.

 ++


 I don't think we should expect detailed documentation for modules in
 etc.c. These are interfaces to existing C libraries, not Phobos-specific
 modules. There's no need to duplicate existing documentation. If you
 want to learn how to use libcurl, the place to look is the libcurl
 homepage.


 http://curl.haxx.se/libcurl/


 Wrong.

 If you want programmers to use D, give them the solution, now.


It's a D binding to a C library. The interface is the same.


 It's not that hard to put a little example at the top of the page
 showing a basic usage.


Wrappers over these libraries should be documented, sure. But for the C 
interface, IMO, a link to the project homepage is sufficient. Any 
differences on the D side should be mentioned, of course. But that's 
all. Otherwise, how many examples should be given? Which functionality 
should be shown? libcurl, specifically, has several examples already 
available from the project page. Adding more to the D docs is redundant.
Nov 03 2011
prev sibling parent reply Graham Fawcett <fawcett uwindsor.ca> writes:
On Tue, 01 Nov 2011 18:20:25 +0000, Frédéric Galusik wrote:


 Hi,
 
 As the curl documentation is a little bit ...wow.
 http://www.digitalmars.com/d/2.0/phobos/etc_c_curl.html
 
 Do someone have a simple example on how to download a simple file ?
 
 Thank you.


Until the high-level bindings for curl are available in Phobos, I have a 
little module you could use.

https://github.com/gmfawcett/d-play-libcurl/blob/master/fawcett/curl.d

The default version prints the result to stdout, and there's a commented 
main() that shows how to get a callback when data is received. If you 
wanted to download directly into a file, you could do this:

import std.stdio;
import fawcett.curl;

class FileCurl : Curl 
{
  this(string url, File f) {
    handle = curl_easy_init();
    set(CURLoption.URL, url);
    set(CURLoption.FILE, cast(void*) f.getFP);
  }

}

void main() {
  auto f = File("/tmp/stuff", "w");
  auto c = new FileCurl("http://example.com/", f);
  c.perform();
}

Note, it doesn't call super() in the constructor, which is bad form, but 
the current default constructor is overkill for just downloading a file. 
(It's not a production-quality curl binding.)

Rather than putting my module in 'fawcett/curl.d', you might just want to 
remove the 'module' line in 'curl.d' and put it in the same directory as 
your program.

Best,
Graham
Nov 02 2011
parent =?iso-8859-1?b?RnLpZOlyaWM=?= Galusik <fredg salixosnospam.org> writes:
Le Wed, 02 Nov 2011 15:09:17 +0000, Graham Fawcett a écrit :


 On Tue, 01 Nov 2011 18:20:25 +0000, Frédéric Galusik wrote:
 

 Hi,
 
 As the curl documentation is a little bit ...wow.
 http://www.digitalmars.com/d/2.0/phobos/etc_c_curl.html
 
 Do someone have a simple example on how to download a simple file ?
 
 Thank you.


 
 Until the high-level bindings for curl are available in Phobos, I have a
 little module you could use.
 
 https://github.com/gmfawcett/d-play-libcurl/blob/master/fawcett/curl.d
 
 The default version prints the result to stdout, and there's a commented
 main() that shows how to get a callback when data is received. If you
 wanted to download directly into a file, you could do this:
 
 import std.stdio;
 import fawcett.curl;
 
 class FileCurl : Curl
 {
   this(string url, File f) {
     handle = curl_easy_init();
     set(CURLoption.URL, url);
     set(CURLoption.FILE, cast(void*) f.getFP);
   }
 
 }
 
 void main() {
   auto f = File("/tmp/stuff", "w");
   auto c = new FileCurl("http://example.com/", f); c.perform();
 }
 
 Note, it doesn't call super() in the constructor, which is bad form, but
 the current default constructor is overkill for just downloading a file.
 (It's not a production-quality curl binding.)
 
 Rather than putting my module in 'fawcett/curl.d', you might just want
 to remove the 'module' line in 'curl.d' and put it in the same directory
 as your program.
 
 Best,
 Graham


Thank you very much, it's very interesting.

Best regards.

++ fredg
Nov 02 2011