www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - URI parsing

reply John C <johnch_atms hotmail.com> writes:
What packages do people use when they want to parse URIs? I 
rolled my own but it's incomplete and as it's a fairly common 
need there must be one out there? In fact, I'm surprised there 
isn't one in Phobos yet.
Oct 05 2016
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 05/10/2016 11:19 PM, John C wrote:

 What packages do people use when they want to parse URIs? I rolled my
 own but it's incomplete and as it's a fairly common need there must be
 one out there? In fact, I'm surprised there isn't one in Phobos yet.


I developed[0] with the hopes that at some point in the future I could 
bring that over to Phobos. But before that I'm waiting for allocators to 
be moved out of experimental first.

[0] 
https://github.com/rikkimax/alphaPhobos/blob/master/source/std/experimental/uri.d

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
Oct 05 2016
parent John C <johnch_atms hotmail.com> writes:
On Wednesday, 5 October 2016 at 10:28:44 UTC, rikki cattermole 
wrote:

 I developed[0] with the hopes that at some point in the future 
 I could bring that over to Phobos. But before that I'm waiting 
 for allocators to be moved out of experimental first.

 [0] 
 https://github.com/rikkimax/alphaPhobos/blob/master/source/std/experimental/uri.d


Looks good, thanks.
Oct 05 2016
prev sibling next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Wednesday, 5 October 2016 at 10:19:08 UTC, John C wrote:

 What packages do people use when they want to parse URIs? I 
 rolled my own but it's incomplete and as it's a fairly common 
 need there must be one out there? In fact, I'm surprised there 
 isn't one in Phobos yet.


It's not easy to make a true URI implementation, by "true" I mean 
"conform" with https://tools.ietf.org/html/rfc3986.

But for the URL subset there's url.d: 
https://code.dlang.org/packages/urld.
Oct 05 2016
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 06/10/2016 12:12 AM, Basile B. wrote:

 On Wednesday, 5 October 2016 at 10:19:08 UTC, John C wrote:

 What packages do people use when they want to parse URIs? I rolled my
 own but it's incomplete and as it's a fairly common need there must be
 one out there? In fact, I'm surprised there isn't one in Phobos yet.


 It's not easy to make a true URI implementation, by "true" I mean
 "conform" with https://tools.ietf.org/html/rfc3986.

 But for the URL subset there's url.d: https://code.dlang.org/packages/urld.


Yeah the spec is quite nasty.
Luckily nobody uses Gopher anymore!

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
Oct 05 2016
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thursday, October 06, 2016 00:13:03 rikki cattermole via Digitalmars-d 
wrote:

 On 06/10/2016 12:12 AM, Basile B. wrote:

 It's not easy to make a true URI implementation, by "true" I mean
 "conform" with https://tools.ietf.org/html/rfc3986.


 Yeah the spec is quite nasty.


Yeah, like _far_ too many of the RFCs relating to the web, it's a mess. From
what I recall, it's not even possible to parse a URL in a completely
unambiguous manner. *sigh*

- Jonathan M Davis
Oct 05 2016
parent ketmar <ketmar ketmar.no-ip.org> writes:
On Wednesday, 5 October 2016 at 12:14:06 UTC, Jonathan M Davis 
wrote:

 a mess. From what I recall, it's not even possible to parse a 
 URL in a completely unambiguous manner. *sigh*


yeah. everything after scheme is highly dependent of scheme 
itself, so your parser should know about various schemes and 
their syntax. and even with "http:" it sometimes a guesswork.
Oct 05 2016
prev sibling next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 5 October 2016 at 10:19:08 UTC, John C wrote:

 What packages do people use when they want to parse URIs?


I did one based on the regex found in the RFC for my cgi and http 
libs.

cgi version (more complete)
https://github.com/adamdruppe/arsd/blob/master/cgi.d#L2163

http version (just what i needed for handling relative links in 
html)
https://github.com/adamdruppe/arsd/blob/master/http2.d#L114


If you just need to parse a complete uri though, go ahead and use 
the regex from the RFC. My thing is more about making uris based 
on other uris for handling those links.
Oct 05 2016
prev sibling next sibling parent Eugene Wissner <belka caraus.de> writes:
On Wednesday, 5 October 2016 at 10:19:08 UTC, John C wrote:

 What packages do people use when they want to parse URIs? I 
 rolled my own but it's incomplete and as it's a fairly common 
 need there must be one out there? In fact, I'm surprised there 
 isn't one in Phobos yet.


I also wrote a function for extended url parsing based on php 
source code. it is in dlib: https://github.com/gecko0307/dlib
dlib.network.url
Oct 05 2016
prev sibling next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 10/05/2016 06:19 AM, John C wrote:

 What packages do people use when they want to parse URIs? I rolled my
 own but it's incomplete and as it's a fairly common need there must be
 one out there? In fact, I'm surprised there isn't one in Phobos yet.


IIRC there's one in vibe I used not too long ago.
Oct 05 2016
prev sibling parent John Carter <john.carter taitradio.com> writes:
On Wednesday, 5 October 2016 at 10:19:08 UTC, John C wrote:

 In fact, I'm surprised there isn't one in Phobos yet.


Different John C here :-)

I'm sadly forced to do a bit of C++ recently and found they don't 
have one in the standard library either...

That feels archaic and out of the 1980's compared to the 
ecosystem around Ruby or Perl.

The downstream fallout of refusing to just "pick one", is that 
packages depending on such functionality each "roll their own", 
and users have to provide compatibility shims all over the place.

Part of the success of Ruby has been it comes "With Batteries 
included".
Oct 05 2016