website goes down after handful of submissions

A place for Doghouse related posts.

website goes down after handful of submissions

Postby vjl » Wed Feb 06, 2019 12:40 pm

I'm trying to submit a good portion of my Bookpedia collection and I am noticing that when I select a collection to submit, after a few minutes, the website front end goes down [just a blank white page] - this is for the main Books page, as well as the books I've submitted page [as well as the rest of the site].

Any idea what's happening?

Thanks!
vjl
Addicted to Bruji
Addicted to Bruji
 
Posts: 50
Joined: Sat Jun 09, 2007 6:05 pm

Re: website goes down after handful of submissions

Postby Conor » Thu Feb 07, 2019 2:03 pm

Thank you for the bug report. It's one that I am aware of as I have an computer program monitoring Doghouse.

What happens is that Doghouse has become so popular I am starting to hit the limits of ClearDB (The MySQL database provider I use). I have it on the to do list to migrate to PostgreSQL on Heroku that does not have the limits, but it's not an easy change. It also happens not very often, when there are lots of uploads from several users and lots of searches and then I start reaching the limit. Luckily I did set it to back down and then it comes back up a few minutes later and the new entries get processed later, when searches are not so high.

The website goes down, cause I have not added something to detect that there no results from the database and show an error message. I plan to fix the issue and then not need that error message at all.
User avatar
Conor
Top Dog
 
Posts: 5235
Joined: Sat Jul 03, 2004 12:58 pm

Re: website goes down after handful of submissions

Postby vjl » Thu Feb 07, 2019 6:21 pm

Hi Conor,

Thanks for the info - glad it wasn't me that was doing any kind of crashing! I am unsure they would be a good match, but I have a DreamHost account and have used their mysql database as a backup for some stuff, and the service and speed is good for my needs. Maybe you can give them a try, as migrating from one install of MySQL to another is a lot simpler than going to a different db!

https://wnc.li/z has more info if you're interested.

I've been slowly working on uploading a 2,000 item collection [contributing number 289 so far]. When I notice Bookpedia stops displaying the contributing text at the bottom, I just wait a while and try again. Hopefully some of the years of using Bookpedia will be useful for the Doghouse! :)
vjl
Addicted to Bruji
Addicted to Bruji
 
Posts: 50
Joined: Sat Jun 09, 2007 6:05 pm

Re: website goes down after handful of submissions

Postby Conor » Fri Feb 08, 2019 10:34 am

Thank you. I hear good things about Dreamhost. Our regular infrastructure is at Site5 and I could use a MySQL database there. Doghouse runs and scales on Heroku on bigger plans. But I do think you are right that I could make a quick move to JawsDB MySQL without a migration. The postgre was just always an option as it's the native database to Heroku and not part of the add-on market place. The Thresher plan looks interesting and I shall investigate, I like the idea of SSD.
User avatar
Conor
Top Dog
 
Posts: 5235
Joined: Sat Jul 03, 2004 12:58 pm

Re: website goes down after handful of submissions

Postby Conor » Mon Feb 11, 2019 7:01 am

You may all now go wild and submit simultaneous requests. I have moved the database on to a dedicated infrastructure with no query limits. Lets see how it performs.
User avatar
Conor
Top Dog
 
Posts: 5235
Joined: Sat Jul 03, 2004 12:58 pm

Re: website goes down after handful of submissions

Postby sitenoise » Tue Feb 12, 2019 3:36 pm

Wow that was fast. I don't often submit a bunch at the same time but I had noticed it was a little up and down lately. So far, so good. :)
User avatar
sitenoise
Addicted to Bruji
Addicted to Bruji
 
Posts: 199
Joined: Fri Mar 21, 2008 12:42 pm

Re: website goes down after handful of submissions

Postby Conor » Wed Feb 13, 2019 3:04 pm

Last two weeks have been upgrades to Doghouse, with more planned for April. When I am working on it, sometimes I break it for a few minutes, every couple of hours. :oops: I do have a pipeline for bigger changes, but for small changes, it's better to test them live against a big search load, being created by users. But then I always miss that one edge case that brings the server crashing. :D Luckily for every error I get an email and I hate getting error emails, so Doghouse stays quite stable.
User avatar
Conor
Top Dog
 
Posts: 5235
Joined: Sat Jul 03, 2004 12:58 pm

Re: website goes down after handful of submissions

Postby vjl » Mon Feb 18, 2019 10:29 pm

Hey Conor,

Just a quick heads up that while the Music and Movies sections seem to be working okay, the Books front end just produces a white page [it's been like this for a few days, as I've been spot checking every few hours]. https://doghouse.bruji.com/books/doghouse
vjl
Addicted to Bruji
Addicted to Bruji
 
Posts: 50
Joined: Sat Jun 09, 2007 6:05 pm

Re: website goes down after handful of submissions

Postby Conor » Tue Feb 19, 2019 5:20 pm

Thank you, looking into it.
User avatar
Conor
Top Dog
 
Posts: 5235
Joined: Sat Jul 03, 2004 12:58 pm

Re: website goes down after handful of submissions

Postby FineWine » Tue Feb 19, 2019 5:39 pm

Problem confirmed
User avatar
FineWine
Addicted to Bruji
Addicted to Bruji
 
Posts: 476
Joined: Wed May 28, 2008 2:41 am
Location: Tauranga, New Zealand

Re: website goes down after handful of submissions

Postby Conor » Wed Feb 20, 2019 12:31 pm

Problem squashed.
User avatar
Conor
Top Dog
 
Posts: 5235
Joined: Sat Jul 03, 2004 12:58 pm


Return to Doghouse