www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - curl: catching exception on connect.

reply "Suliman" <evermind live.ru> writes:
I can't understand why I am getting exception on next code:

	void downloadFile()
	{
		foreach(link; links)
		{
			try
			{
				writeln(connect(link));
			}

			catch(Exception msg)
			{
				writeln(msg);
			}
		}
	}
	
Exception text:

std.net.curl.CurlException C:\DMD\dmd2\windows\bin\..\..\src\phobos\std\net\curl
.d(779): HTTP request returned status code 400
----------------
0x0040A536 in pure  safe bool
std.exception.enforceEx!(std.net.curl.CurlExceptio
n).enforceEx!(bool).enforceEx(bool, lazy immutable(char)[],
immutable(char)[], u
int) at
C:\DMD\dmd2\windows\bin\..\..\src\phobos\std\exception.d(529)
0x004073EC in char[]
std.net.curl._basicHTTP!(char)._basicHTTP(const(char)[], co
nst(void)[], std.net.curl.HTTP) at
C:\DMD\dmd2\windows\bin\..\..\src\phobos\std\
net\curl.d(784)
0x0040720C in char[]
std.net.curl.connect!(char).connect(const(char)[], std.net.
curl.HTTP) at
C:\DMD\dmd2\windows\bin\..\..\src\phobos\std\net\curl.d(694)
0x00402386 in void app.simpleDownload.downloadFile()
0x0040206D in _Dmain at D:\code\imgDownload\source\app.d(20)
0x00437C08 in void rt.dmain2._d_run_main(int, char**, extern (C)
int function(ch
ar[][])*).runAll().void __lambda1()
0x00437BDB in void rt.dmain2._d_run_main(int, char**, extern (C)
int function(ch
ar[][])*).runAll()
0x00437AF4 in _d_run_main
0x00432FEC in main
0x0047529D in mainCRTStartup
0x7729336A in BaseThreadInitThunk
0x77A59F72 in RtlInitializeExceptionChain
0x77A59F45 in RtlInitializeExceptionChain
std.net.curl.CurlException C:\DMD\dmd2\windows\bin\..\..\src\phobos\std\net\curl
.d(779): HTTP request returned status code 400

By the code App should simply write the server response like 400
or 200. But it's simply crash.

It's look like I do not right understand exception level and way
of use them. Could anybody help me?
Nov 30 2014
next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
This is turning out to be a common problem. :)

On 11/30/2014 11:24 AM, Suliman wrote:

 I can't understand why I am getting exception on next code:

      void downloadFile()
      {
          foreach(link; links)
          {
              try
              {
                  writeln(connect(link));
              }

              catch(Exception msg)
              {
                  writeln(msg);


In capitals to make a lasting effect: DON'T EVER OUTPUT THE EXCEPTION 
OBJECT!

Otherwise, you will fool yourself and others as if the exception is not 
being caught.

What is happening is that you are actually catching the exception, 
printing on screen and exiting cleanly.

There is no point of printing the exception object itself. Instead, 
print a meaningful message or something else:

         } catch(Exception msg) {
             import std.string;
             writeln(format("Failed to download '%s'.", link));
         }

Ali
Nov 30 2014
parent reply "Suliman" <evermind live.ru> writes:
Am I right understand all exception are derived from 
"assertThrown"
http://dlang.org/phobos/std_exception.html

So "msg" in "catch(Exception msg)" is from function 
"assertThrown"?

Could you show me example of how to handle type of exception?
Nov 30 2014
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 11/30/2014 10:38 PM, Suliman wrote:


 Am I right understand all exception are derived from "assertThrown"
 http://dlang.org/phobos/std_exception.html


No, all exceptions are derived from Throwable. It has two descendants:

- Error, representing conditions that are irrecoverable, and

- Exception, representing conditions that are recoverable.

Those two have many descendants themselves.

assertThrown() is useful in unit testing. It is a function that checks 
whether an expression indeed throws an exception.


 So "msg" in "catch(Exception msg)"


'catch' catches an exception object. You should name it as 'exc' or 
something similar. 'msg' makes it sound like "message" which it is not.


 is from function "assertThrown"?

 Could you show me example of how to handle type of exception?


Your example works: You are actually handling the exception. What you 
did with it later was confusing to you and to us.

The following program realizes that a connect() call failed by catching 
an exception. It simply outputs a message about it.

import std.stdio;
import std.net.curl;

pragma(lib, "curl");

const links = [ "ddili.org/non_existent_link" ];

void downloadFile()
{
     foreach(link; links)
     {
         try
         {
             connect(link);

         } catch(Exception exc) {
             import std.string;
             writeln(format("Failed to download '%s'.", link));
         }
     }
}

void main()
{
     downloadFile();
}

Ali
Dec 01 2014
parent reply "Suliman" <evermind live.ru> writes:
Big thanks Ali!

Only the small last question:

why:
	string link = "dlang.org";
	writeln(connect(link));

cause crash:

std.net.curl.CurlException C:\DMD\dmd2\windows\bin\..\..\src\phobos\std\net\curl
.d(779): HTTP request returned status code 400
----------------
0x00404263
0x0040226A
0x00402090
0x00402030
0x00421E6C
0x00421E3F
0x00421D58
0x0041A177
0x7641336A in BaseThreadInitThunk
0x77069F72 in RtlInitializeExceptionChain
0x77069F45 in RtlInitializeExceptionChain

I remember that there was some issue in docs (if I not mistaken) 
but I can't google about it.
Dec 01 2014
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 12/01/2014 10:25 AM, Suliman wrote:


 why:
      string link = "dlang.org";
      writeln(connect(link));

 cause crash:


(To be pedantic: An unhandled exception is not a crash. ;) )


 


std.net.curl.CurlException C:\DMD\dmd2\windows\bin\..\..\src
phobos\std\net\curl 


 .d(779): HTTP request returned status code 400


Indeed, the following program does behave in the same way:

import std.stdio;
import std.net.curl;

pragma(lib, "curl");

void main()
{
     string link = "dlang.org";
     writeln(connect(link));
}

I have very little experience with the HTTP protocol and its methods but 
I've just learnt that connect() is for HTTP's CONNECT method, which is 
most likely supported on port 443.

So, I tried

     string link = "dlang.org:443";

and got

std.net.curl.CurlException std/net/curl.d(3684): Server returned nothing 
(no headers, no data) on handle 198F860

So, that's an improvement I guess. (It was the same result at 
"google.com:443") Then I tried HTTPS explicitly:

     string link = "https://dlang.org:443";

Now the error message is more promising:

std.net.curl.CurlException std/net/curl.d(3684): Peer certificate cannot 
be authenticated with known CA certificates on handle 1653860

My guess is that you have to use HTTPS for CONNECT and that you have to 
have credentials for it. (?)

Ali
Dec 01 2014
parent reply "Suliman" <evermind live.ru> writes:
 My guess is that you have to use HTTPS for CONNECT and that you 
 have to have credentials for it. (?)

 Ali


dlang.org should work on HTTP, but not HTTPS.

Also I do not think that when I connect to HTTPS I should have 
any credentials. It's mostly like issue with curl...
Dec 01 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 1 December 2014 at 19:44:56 UTC, Suliman wrote:

 My guess is that you have to use HTTPS for CONNECT and that 
 you have to have credentials for it. (?)

 Ali


 dlang.org should work on HTTP, but not HTTPS.

 Also I do not think that when I connect to HTTPS I should have 
 any credentials. It's mostly like issue with curl...


connect() sends a "CONNECT" request to the server, as defined by 
HTTP [1]. This method is only used when you're working with 
proxies and the like. What you most likely want, however, is a 
"GET" request. Use get() for that.

[1] 
https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Request_methods
Dec 02 2014
parent reply "Suliman" <evermind live.ru> writes:
 connect() sends a "CONNECT" request to the server, as defined 
 by HTTP [1]. This method is only used when you're working with 
 proxies and the like. What you most likely want, however, is a 
 "GET" request. Use get() for that.


So what is the best way to check status server response (400, 404 
etc) to send get request and try to parse response, or there is 
any better way (probably with another lib?)?
Dec 02 2014
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 12/02/2014 05:19 AM, Suliman wrote:

 connect() sends a "CONNECT" request to the server, as defined by HTTP
 [1]. This method is only used when you're working with proxies and the
 like. What you most likely want, however, is a "GET" request. Use
 get() for that.


 So what is the best way to check status server response (400, 404 etc)
 to send get request and try to parse response, or there is any better
 way (probably with another lib?)?


connect() uses the curl.HTTP struct, which is available to us as well:

   http://dlang.org/phobos/std_net_curl.html#HTTP

I added two lines to the first example there:

import std.net.curl, std.stdio;

pragma(lib, "curl");

void main()
{
     // Get with custom data receivers
     auto http = HTTP("dlang.org");
     http.onReceiveHeader =
         (in char[] key, in char[] value) { writeln(key ~ ": " ~ value); };
     http.onReceive = (ubyte[] data) { /+ drop +/ return data.length; };
     http.perform();

     const HTTP.StatusLine statusLine = http.statusLine();
     writefln("GET request completed with code %s", statusLine.code);
}

Sample output:

date: Wed, 03 Dec 2014 18:19:15 GMT
server: Apache/2.2.22 (FreeBSD) PHP/5.3.15 with Suhosin-Patch 
mod_ssl/2.2.22 OpenSSL/0.9.8x DAV/2
last-modified: Sun, 02 Nov 2014 06:38:55 GMT
etag: "3b438-4f9a-506da7be81dc0"
accept-ranges: bytes
content-length: 20378
content-type: text/html
GET request completed with code 200

Ali
Dec 03 2014
prev sibling parent "Meta" <jared771 gmail.com> writes:
On Sunday, 30 November 2014 at 19:24:39 UTC, Suliman wrote:

 I can't understand why I am getting exception on next code:

 	void downloadFile()
 	{
 		foreach(link; links)
 		{
 			try
 			{
 				writeln(connect(link));
 			}

 			catch(Exception msg)
 			{
 				writeln(msg);
 			}
 		}
 	}


Change this here:

catch (Exception msg)
{
     writeln(msg);
}

to:

catch (Exception msg)
{
     writeln(msg.msg);
}

The problem is that you are catching the exception and then 
printing it out, which prints a string representation of the 
exception. This string representation includes the exception's 
message and its stack trace. What you probably want is to print 
out its .msg member, which is the error message.
Nov 30 2014