www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DSource

reply Kyle Furlong <kylefurlong gmail.com> writes:
Okay, so we just got dug, and DSource has been down for more than a 
week. Does anyone else see the problem here? People will be turning away 
as I write this because the main repository of open source development 
in D has a "FATAL: connection limit exceeded for non-superusers" message 
on the front page.

While I appreciate the work done to create and maintain dsource.org, a 
week is not acceptable. I would be happy to help in any capacity to get 
even a mirror up asap.
Apr 25 2006
next sibling parent reply Walter Bright <newshound digitalmars.com> writes:
Kyle Furlong wrote:

 Okay, so we just got dug, and DSource has been down for more than a 
 week. Does anyone else see the problem here? People will be turning away 
 as I write this because the main repository of open source development 
 in D has a "FATAL: connection limit exceeded for non-superusers" message 
 on the front page.
 
 While I appreciate the work done to create and maintain dsource.org, a 
 week is not acceptable. I would be happy to help in any capacity to get 
 even a mirror up asap.


digitalmars.com saw roughly a 6 fold increase in traffic yesterday.
Apr 25 2006
parent reply Kyle Furlong <kylefurlong gmail.com> writes:
Walter Bright wrote:

 Kyle Furlong wrote:

 Okay, so we just got dug, and DSource has been down for more than a 
 week. Does anyone else see the problem here? People will be turning 
 away as I write this because the main repository of open source 
 development in D has a "FATAL: connection limit exceeded for 
 non-superusers" message on the front page.

 While I appreciate the work done to create and maintain dsource.org, a 
 week is not acceptable. I would be happy to help in any capacity to 
 get even a mirror up asap.


 
 digitalmars.com saw roughly a 6 fold increase in traffic yesterday.


Thats exciting, but what does that have to do with the fact that dsource 
has been down for much longer than just yesterday? It would have been 
nice to have some of that increased traffic funneled to a working 
dsource. What we got is in essence a PR disaster.
Apr 25 2006
parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
Kyle Furlong wrote:


 Walter Bright wrote:

 Kyle Furlong wrote:

 Okay, so we just got dug, and DSource has been down for more than a
 week. Does anyone else see the problem here? People will be turning
 away as I write this because the main repository of open source
 development in D has a "FATAL: connection limit exceeded for
 non-superusers" message on the front page.

 While I appreciate the work done to create and maintain dsource.org, a
 week is not acceptable. I would be happy to help in any capacity to
 get even a mirror up asap.


 
 digitalmars.com saw roughly a 6 fold increase in traffic yesterday.


 
 Thats exciting, but what does that have to do with the fact that dsource
 has been down for much longer than just yesterday? It would have been
 nice to have some of that increased traffic funneled to a working
 dsource. What we got is in essence a PR disaster.


The problems Dsource is having is not really traffic related (although high
traffic don't help), it is a technical problem in Trac wherein it don't
release it's db connections the way it should. You might help resolve the
problem by finding out where in the Trac/Python code the problem is, I know
Brad would be very happy to find it.
Apr 25 2006
next sibling parent reply =?ISO-8859-1?Q?Jari-Matti_M=E4kel=E4?= <jmjmak utu.fi.invalid> writes:
Lars Ivar Igesund wrote:


 The problems Dsource is having is not really traffic related (although high
 traffic don't help), it is a technical problem in Trac wherein it don't
 release it's db connections the way it should. You might help resolve the
 problem by finding out where in the Trac/Python code the problem is, I know
 Brad would be very happy to find it.


Has this already been fixed in the 0.10dev version of trac?

-- 
Jari-Matti
Apr 25 2006
parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
Jari-Matti Mäkelä wrote:


 Lars Ivar Igesund wrote:
 

 The problems Dsource is having is not really traffic related (although
 high traffic don't help), it is a technical problem in Trac wherein it
 don't release it's db connections the way it should. You might help
 resolve the problem by finding out where in the Trac/Python code the
 problem is, I know Brad would be very happy to find it.


 
 Has this already been fixed in the 0.10dev version of trac?
 


AFAIK, DSource is one of very few Trac enabled sites using Postgres as it's
DB backend. Apparently there are no similar problems with the default
SQLite backend. Therefore it has been somewhat difficult getting those who
know the code best to look at it.
Apr 25 2006
parent reply =?UTF-8?B?SmFyaS1NYXR0aSBNw6RrZWzDpA==?= <jmjmak utu.fi.invalid> writes:
Lars Ivar Igesund wrote:


 AFAIK, DSource is one of very few Trac enabled sites using Postgres as it's
 DB backend. Apparently there are no similar problems with the default
 SQLite backend. Therefore it has been somewhat difficult getting those who
 know the code best to look at it.


Sad to hear. Postgres might be more elegant, but I use sqlite myself for
various reasons. It's very easy to dump the db, for example. Is there a
good reason to use Postgres in the first place?

-- 
Jari-Matti
Apr 25 2006
parent reply Brad Anderson <brad dsource.org> writes:
Jari-Matti Mäkelä wrote:

 Is there a
 good reason to use Postgres in the first place?
 


I was getting ready for Digg-type traffic, and I have seen Trac db's stored in
SQLite lock during write (yes, the whole db locks).  So I chose Postgres.  I
have heard of using one database and separating the Trac tables for a project
into different schemas - the problem that was observed separate of dsource's
actually went away doing this.  This is a big task and I don't have a lot of
time right now.  A few weeks ago when I had some time, the problem wasn't as
pronounced...  Murphy.

I have tried some settings in postgresql.conf to limit connection time-outs
and they're not working yet.

Plus, I didn't see nearly the traffic Walter talks about getting on his site,
because the link to dsource is reasonably obscure.  Some of that lack of
traffic was the brief periods when we were down, but I would have seen hits in
the logs, just with errors.

Also, we weren't down for an entire week as the OP suggests.  Apache resets
itself, so we were down no longer than 2 hours at a time, and then were fine
for most likely 12 hrs, until it happened again.

BA
Apr 25 2006
next sibling parent Kyle Furlong <kylefurlong gmail.com> writes:
Brad Anderson wrote:

 Jari-Matti Mäkelä wrote:

 Is there a
 good reason to use Postgres in the first place?


 
 I was getting ready for Digg-type traffic, and I have seen Trac db's stored in
 SQLite lock during write (yes, the whole db locks).  So I chose Postgres.  I
 have heard of using one database and separating the Trac tables for a project
 into different schemas - the problem that was observed separate of dsource's
 actually went away doing this.  This is a big task and I don't have a lot of
 time right now.  A few weeks ago when I had some time, the problem wasn't as
 pronounced...  Murphy.
 
 I have tried some settings in postgresql.conf to limit connection time-outs
 and they're not working yet.
 
 Plus, I didn't see nearly the traffic Walter talks about getting on his site,
 because the link to dsource is reasonably obscure.  Some of that lack of
 traffic was the brief periods when we were down, but I would have seen hits in
 the logs, just with errors.
 
 Also, we weren't down for an entire week as the OP suggests.  Apache resets
 itself, so we were down no longer than 2 hours at a time, and then were fine
 for most likely 12 hrs, until it happened again.
 
 BA


I guess I got unlucky in that every time I looked in on the site in the 
past week it was down.
Apr 25 2006
prev sibling parent reply Walter Bright <newshound digitalmars.com> writes:
Brad Anderson wrote:

 Plus, I didn't see nearly the traffic Walter talks about getting on his site,
 because the link to dsource is reasonably obscure.


That's fixed now.
Apr 25 2006
parent Brad Anderson <brad dsource.org> writes:
Walter Bright wrote:

 Brad Anderson wrote:

 Plus, I didn't see nearly the traffic Walter talks about getting on
 his site,
 because the link to dsource is reasonably obscure.


 
 That's fixed now.


sweet, thanks
Apr 26 2006
prev sibling parent Kyle Furlong <kylefurlong gmail.com> writes:
Lars Ivar Igesund wrote:

 Kyle Furlong wrote:
 

 Walter Bright wrote:

 Kyle Furlong wrote:

 Okay, so we just got dug, and DSource has been down for more than a
 week. Does anyone else see the problem here? People will be turning
 away as I write this because the main repository of open source
 development in D has a "FATAL: connection limit exceeded for
 non-superusers" message on the front page.

 While I appreciate the work done to create and maintain dsource.org, a
 week is not acceptable. I would be happy to help in any capacity to
 get even a mirror up asap.


 digitalmars.com saw roughly a 6 fold increase in traffic yesterday.


 Thats exciting, but what does that have to do with the fact that dsource
 has been down for much longer than just yesterday? It would have been
 nice to have some of that increased traffic funneled to a working
 dsource. What we got is in essence a PR disaster.


 
 The problems Dsource is having is not really traffic related (although high
 traffic don't help), it is a technical problem in Trac wherein it don't
 release it's db connections the way it should. You might help resolve the
 problem by finding out where in the Trac/Python code the problem is, I know
 Brad would be very happy to find it.


I understand all this. I never thought that dsource was down due to the 
digg effect. I wish that had been the case! At least some of them would 
have seen the site. I was well aware of the connections issue and the 
length of down time which had been going on previous to the whole digg 
thing.
Apr 25 2006
prev sibling parent reply Hasan Aljudy <hasan.aljudy gmail.com> writes:
Kyle Furlong wrote:

 Okay, so we just got dug, and DSource has been down for more than a 
 week. Does anyone else see the problem here? People will be turning away 
 as I write this because the main repository of open source development 
 in D has a "FATAL: connection limit exceeded for non-superusers" message 
 on the front page.
 
 While I appreciate the work done to create and maintain dsource.org, a 
 week is not acceptable. I would be happy to help in any capacity to get 
 even a mirror up asap.


Has it been down for a week? I remember navigating dsource a few days 
ago and it was working!
Apr 25 2006
parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
Hasan Aljudy wrote:


 Kyle Furlong wrote:

 Okay, so we just got dug, and DSource has been down for more than a
 week. Does anyone else see the problem here? People will be turning away
 as I write this because the main repository of open source development
 in D has a "FATAL: connection limit exceeded for non-superusers" message
 on the front page.
 
 While I appreciate the work done to create and maintain dsource.org, a
 week is not acceptable. I would be happy to help in any capacity to get
 even a mirror up asap.


 
 Has it been down for a week? I remember navigating dsource a few days
 ago and it was working!


No, it hasn't been down at all, in the correct way of speech, but the Trac
part of the site that use the underlying database, causes connection leaks
that makes those pages inaccesible until they are cleaned up. At that point
it also affects the forums. The connections also cause some thrashing,
meaning that svn may or may not be somewhat accesible too, but in general,
svn has been mostly unaffected.
Apr 25 2006
parent reply lightoze <lightoze_member pathlink.com> writes:
Lars Ivar Igesund says...

No, it hasn't been down at all, in the correct way of speech, but the Trac
part of the site that use the underlying database, causes connection leaks
that makes those pages inaccesible until they are cleaned up. At that point
it also affects the forums. The connections also cause some thrashing,
meaning that svn may or may not be somewhat accesible too, but in general,
svn has been mostly unaffected.


May be this can be temporary solved by setting low connection timeout...?
Apr 25 2006
parent reply clayasaurus <clayasaurus gmail.com> writes:
lightoze wrote:

 Lars Ivar Igesund says...

 No, it hasn't been down at all, in the correct way of speech, but the Trac
 part of the site that use the underlying database, causes connection leaks
 that makes those pages inaccesible until they are cleaned up. At that point
 it also affects the forums. The connections also cause some thrashing,
 meaning that svn may or may not be somewhat accesible too, but in general,
 svn has been mostly unaffected.


 
 May be this can be temporary solved by setting low connection timeout...?
 
 


or use the SQLite backend?
Apr 25 2006
parent James Dunne <james.jdunne gmail.com> writes:
clayasaurus wrote:

 lightoze wrote:
 

 Lars Ivar Igesund says...


 No, it hasn't been down at all, in the correct way of speech, but the 
 Trac
 part of the site that use the underlying database, causes connection 
 leaks
 that makes those pages inaccesible until they are cleaned up. At that 
 point
 it also affects the forums. The connections also cause some thrashing,
 meaning that svn may or may not be somewhat accesible too, but in 
 general,
 svn has been mostly unaffected.



 May be this can be temporary solved by setting low connection timeout...?


 
 or use the SQLite backend?


In my experience, SQLite has huge problems with concurrency.  I've tried 
developing for it and it's really difficult to get multithreaded 
database access code to work.

-- 
-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GCS/MU/S d-pu s:+ a-->? C++++$ UL+++ P--- L+++ !E W-- N++ o? K? w--- O 
M--  V? PS PE Y+ PGP- t+ 5 X+ !R tv-->!tv b- DI++(+) D++ G e++>e 
h>--->++ r+++ y+++
------END GEEK CODE BLOCK------

James Dunne
Apr 26 2006