website goes down after handful of submissions

A place for Doghouse related posts.
Post Reply
vjl
Addicted to Bruji
Addicted to Bruji
Posts: 52
Joined: Sat Jun 09, 2007 6:05 pm
Contact:

website goes down after handful of submissions

Post by vjl »

I'm trying to submit a good portion of my Bookpedia collection and I am noticing that when I select a collection to submit, after a few minutes, the website front end goes down [just a blank white page] - this is for the main Books page, as well as the books I've submitted page [as well as the rest of the site].

Any idea what's happening?

Thanks!
User avatar
Conor
Top Dog
Posts: 5343
Joined: Sat Jul 03, 2004 12:58 pm
Contact:

Re: website goes down after handful of submissions

Post by Conor »

Thank you for the bug report. It's one that I am aware of as I have an computer program monitoring Doghouse.

What happens is that Doghouse has become so popular I am starting to hit the limits of ClearDB (The MySQL database provider I use). I have it on the to do list to migrate to PostgreSQL on Heroku that does not have the limits, but it's not an easy change. It also happens not very often, when there are lots of uploads from several users and lots of searches and then I start reaching the limit. Luckily I did set it to back down and then it comes back up a few minutes later and the new entries get processed later, when searches are not so high.

The website goes down, cause I have not added something to detect that there no results from the database and show an error message. I plan to fix the issue and then not need that error message at all.
vjl
Addicted to Bruji
Addicted to Bruji
Posts: 52
Joined: Sat Jun 09, 2007 6:05 pm
Contact:

Re: website goes down after handful of submissions

Post by vjl »

Hi Conor,

Thanks for the info - glad it wasn't me that was doing any kind of crashing! I am unsure they would be a good match, but I have a DreamHost account and have used their mysql database as a backup for some stuff, and the service and speed is good for my needs. Maybe you can give them a try, as migrating from one install of MySQL to another is a lot simpler than going to a different db!

https://wnc.li/z has more info if you're interested.

I've been slowly working on uploading a 2,000 item collection [contributing number 289 so far]. When I notice Bookpedia stops displaying the contributing text at the bottom, I just wait a while and try again. Hopefully some of the years of using Bookpedia will be useful for the Doghouse! :)
User avatar
Conor
Top Dog
Posts: 5343
Joined: Sat Jul 03, 2004 12:58 pm
Contact:

Re: website goes down after handful of submissions

Post by Conor »

Thank you. I hear good things about Dreamhost. Our regular infrastructure is at Site5 and I could use a MySQL database there. Doghouse runs and scales on Heroku on bigger plans. But I do think you are right that I could make a quick move to JawsDB MySQL without a migration. The postgre was just always an option as it's the native database to Heroku and not part of the add-on market place. The Thresher plan looks interesting and I shall investigate, I like the idea of SSD.
User avatar
Conor
Top Dog
Posts: 5343
Joined: Sat Jul 03, 2004 12:58 pm
Contact:

Re: website goes down after handful of submissions

Post by Conor »

You may all now go wild and submit simultaneous requests. I have moved the database on to a dedicated infrastructure with no query limits. Lets see how it performs.
User avatar
sitenoise
Addicted to Bruji
Addicted to Bruji
Posts: 227
Joined: Fri Mar 21, 2008 12:42 pm
Contact:

Re: website goes down after handful of submissions

Post by sitenoise »

Wow that was fast. I don't often submit a bunch at the same time but I had noticed it was a little up and down lately. So far, so good. :)
User avatar
Conor
Top Dog
Posts: 5343
Joined: Sat Jul 03, 2004 12:58 pm
Contact:

Re: website goes down after handful of submissions

Post by Conor »

Last two weeks have been upgrades to Doghouse, with more planned for April. When I am working on it, sometimes I break it for a few minutes, every couple of hours. :oops: I do have a pipeline for bigger changes, but for small changes, it's better to test them live against a big search load, being created by users. But then I always miss that one edge case that brings the server crashing. :D Luckily for every error I get an email and I hate getting error emails, so Doghouse stays quite stable.
vjl
Addicted to Bruji
Addicted to Bruji
Posts: 52
Joined: Sat Jun 09, 2007 6:05 pm
Contact:

Re: website goes down after handful of submissions

Post by vjl »

Hey Conor,

Just a quick heads up that while the Music and Movies sections seem to be working okay, the Books front end just produces a white page [it's been like this for a few days, as I've been spot checking every few hours]. https://doghouse.bruji.com/books/doghouse
User avatar
Conor
Top Dog
Posts: 5343
Joined: Sat Jul 03, 2004 12:58 pm
Contact:

Re: website goes down after handful of submissions

Post by Conor »

Thank you, looking into it.
User avatar
FineWine
Site Admin
Posts: 852
Joined: Wed May 28, 2008 2:41 am
Location: Tauranga, New Zealand

Re: website goes down after handful of submissions

Post by FineWine »

Problem confirmed
User avatar
Conor
Top Dog
Posts: 5343
Joined: Sat Jul 03, 2004 12:58 pm
Contact:

Re: website goes down after handful of submissions

Post by Conor »

Problem squashed.
Post Reply