www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - A web server with D

reply Mengu <mengukagan gmail.com> writes:
Hi,

I have been interested in and learning D for a while and currently developing
a web development IDE with it. I can say that I have a middle level knowledge
that I have been trying to increase. Anyway. I want to develop a web server
for Python and Ruby web applications some months later from now,
after completing the IDE. But until then I want all my research to be
completed and I have enough knowledge that I need to develop this web server,
in addition to D and within D.

So, my two questions are these:

  1) What do I need to know in order to develop a web server, generally?
  2) What do I need to know within D to make this good enough?

Thanks everyone in advance.
Jun 20 2010
next sibling parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 20/06/10 12:54, Mengu wrote:

 Hi,

 I have been interested in and learning D for a while and currently developing
 a web development IDE with it. I can say that I have a middle level knowledge
 that I have been trying to increase. Anyway. I want to develop a web server
 for Python and Ruby web applications some months later from now,
 after completing the IDE. But until then I want all my research to be
 completed and I have enough knowledge that I need to develop this web server,
 in addition to D and within D.

 So, my two questions are these:

    1) What do I need to know in order to develop a web server, generally?


A good starting point would be taking a look at current web servers and 
seeing what they do well etc. Apache httpd is the most widely used one 
which would be a good starting point, particularly feature wise, but 
it's also worth taking a look at others such as lighttpd, nginx and 
cherokee. Note that the main selling points for the latter are speed 
rather than functionality (although I believe this is changing).

 From doing quite a bit of server sided development myself I'd also 
recommend reading the HTTP RFC[1] thoroughly, a good few times and 
making sure you understand it completely. Understand its downfalls and 
intricacies, places where you could encounter issues, make sure you've 
thought about how you're gonna get around them before you start. A lot 
of web servers have already got them covered, so again taking a look at 
them would be a good starting point.

It would also be worth taking a look at best practices server-wise. I 
can't think of any off the top of my head, I know there are some things 
which are better done one way than another - you should take a look at 
these before you start too.

As an afterthought, it could be a good idea to take a look at security 
issues that have been reported for other httpd's, and making sure such 
vulnerabilities don't exist in your server in the first place! There 
will be some that are very httpd specific, I have no doubt there will be 
others that crop up from time to time relating to similar parts of the 
httpd.


    2) What do I need to know within D to make this good enough?


Why settle with good enough when you can settle for great? I don't think 
there's anything you specifically need to know within D for making a 
webserver, but you should make sure you know your way around the 
concurrency/threading code and also the sockets interface so you can 
make good use of them. Concurrency support continues to evolve in D, so 
it'd be worth keeping an eye on.


 Thanks everyone in advance.


Thanks,

Robert

[1] http://www.w3.org/Protocols/rfc2616/rfc2616.html - yes, I know it's 
a bit of a monster read, if you're gonna do this you need to do it 
properly though :)
Jun 20 2010
parent sybrandy <sybrandy gmail.com> writes:
 1) What do I need to know in order to develop a web server, generally?


 A good starting point would be taking a look at current web servers and
 seeing what they do well etc. Apache httpd is the most widely used one
 which would be a good starting point, particularly feature wise, but
 it's also worth taking a look at others such as lighttpd, nginx and
 cherokee. Note that the main selling points for the latter are speed
 rather than functionality (although I believe this is changing).


I haven't looked at it too deeply, but Mongrel2 looks to be very 
interesting.  IIRC, it's written in pure C and supports several 
different protocols.  I believe it leans heavily on the performance 
side.  However, it's not finished yet.

Since D's concurrency is going to use message passing, you may want to 
check out Yaws which is written in Erlang to see how they handle 
connections and what not.  However, do keep in mind that creating 
threads in D is much more expensive than creating processes in Erlang.

Casey
Jun 20 2010
prev sibling next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
Mengu Wrote:


 Hi,
 
 I have been interested in and learning D for a while and currently developing
 a web development IDE with it. I can say that I have a middle level knowledge
 that I have been trying to increase. Anyway. I want to develop a web server
 for Python and Ruby web applications some months later from now,
 after completing the IDE. But until then I want all my research to be
 completed and I have enough knowledge that I need to develop this web server,
 in addition to D and within D.
 
 So, my two questions are these:
 
   1) What do I need to know in order to develop a web server, generally?
   2) What do I need to know within D to make this good enough?


Read the HTTP RFC, tear your hair out as you realize that the grammar isn't
context-free, and console yourself with the knowledge that it's still one of
the most solid and well-followed internet protocol RFCs.  There are a few RFCs
on cookies you'll need to look at too.  After that, if you want a
high-performance server I'd suggest looking into libevent.
Jun 20 2010
parent reply "Nick Sabalausky" <a a.a> writes:
"Sean Kelly" <sean invisibleduck.org> wrote in message 
news:hvlf5u$18ae$1 digitalmars.com...

 Mengu Wrote:


 Hi,

 I have been interested in and learning D for a while and currently 
 developing
 a web development IDE with it. I can say that I have a middle level 
 knowledge
 that I have been trying to increase. Anyway. I want to develop a web 
 server
 for Python and Ruby web applications some months later from now,
 after completing the IDE. But until then I want all my research to be
 completed and I have enough knowledge that I need to develop this web 
 server,
 in addition to D and within D.

 So, my two questions are these:

   1) What do I need to know in order to develop a web server, generally?
   2) What do I need to know within D to make this good enough?


 Read the HTTP RFC, tear your hair out as you realize that the grammar 
 isn't context-free


Really? It seems so simple (but of course, XML seems really simple at first 
glance, too). Have an example?
Jun 20 2010
parent Sean Kelly <sean invisibleduck.org> writes:
Nick Sabalausky Wrote:


 "Sean Kelly" <sean invisibleduck.org> wrote in message 

 Read the HTTP RFC, tear your hair out as you realize that the grammar 
 isn't context-free


 
 Really? It seems so simple (but of course, XML seems really simple at first 
 glance, too). Have an example?


I may have exaggerated a tad, but the issue I was thinking of is that the
header values can contain pretty much anything--particularly cookies (no one
follows the cookie RFC).  So it's not uncommon to see:

    Cookie: myval=thing1:thing2

And then of course the body is just a blob of whatever.  So you can't just
tokenize an HTTP request the same way you would a D module, for example.  You
have to know whether you're in the control line (GET /thingy HTTP/1.1), header
key, header value, or body part.  What I've found so far is that it's easiest
to simply break a request into control line, set of header key/value pairs (of
which there can be duplicates of the same key), and body blob and then parse
individual headers or the body separately as needed.  This is more efficient
anyway I suppose.  It's just irksome that it seems necessary.
Jun 20 2010
prev sibling parent "Rory McGuire" <rmcguire neonova.co.za> writes:
On Sun, 20 Jun 2010 13:54:26 +0200, Mengu <mengukagan gmail.com> wrote:


 Hi,

 I have been interested in and learning D for a while and currently  
 developing
 a web development IDE with it. I can say that I have a middle level  
 knowledge
 that I have been trying to increase. Anyway. I want to develop a web  
 server
 for Python and Ruby web applications some months later from now,
 after completing the IDE. But until then I want all my research to be
 completed and I have enough knowledge that I need to develop this web  
 server,
 in addition to D and within D.

 So, my two questions are these:

   1) What do I need to know in order to develop a web server, generally?
   2) What do I need to know within D to make this good enough?

 Thanks everyone in advance.


read the web server related RFCs, not just HTTP (e.g. also MIME).
try to keep everthing re-usable, e.g. make a library and a web server.
make sure you allow re-writing of urls according to rules (see apache mod  
rewrite) and:
SSL,
Virtual hosts (both named and IP based) (ssl doesn't support name based  
virtual hosts),
access control.
Logging is also very important.
Jun 21 2010