Thursday, May 26, 2016 • Midnight Edition • "Better than Bleeding Cool by that much."

The Outhouse - The Greatest Comic Book Forum

Comics news, comic book reviews, feature articles about comics, interviews with comic creators, plus the greatest comic book and pop culture discussion in the Outhouse forums!


Database Maintenance

Hey you! Reader! Want to be a part of the GREATEST COMIC BOOK AND GEEK COMMUNITY on the web?! Well, they're not accepting new members, but we'll take anyone here, so why not sign up for a free acount? It's fast and it's easy, like your mom! Sign up today! Membership spots are limited!*

*Membership spots not really limited!

User avatar

S.F. Jude Terror

OMCTO

Postby S.F. Jude Terror » Mon Feb 11, 2013 2:29 am

In a little bit, I'm going to run an optimization on the posts table on the database. The maintenance page will be up with a live chat while it runs. It shouldn't take more than an hour at most. The regular forum will come back as soon as it finishes automatically.

Trying to get something together that can run automatically at some late hour on a weekend each week to keep the tables running smoothly. This will be the first test run.
User avatar

covalesky

Expert Post Whore

Postby covalesky » Mon Feb 11, 2013 5:40 am

Don't know if its related or not, but I've been getting the search not allowed atm error throughout the weekend most of the times I try to see the newest posts.

Also getting the Thread does no exists when I click the go to last post icon in the threads list.
User avatar

sdsichero

2k11 Outhouse People's Champion

Postby sdsichero » Mon Feb 11, 2013 10:38 am

Yeah I can't search either. That's a bummer.
User avatar

S.F. Jude Terror

OMCTO

Postby S.F. Jude Terror » Mon Feb 11, 2013 10:55 am

Search is disabled when server load is too high. I have a replacement for this coming next week hopefully that will speed everything up. The server is a fucking nightmare right now, mostly because of all the big threads at this point, but in general because there are 2.5 million posts. It's completely overloaded trying to sort through them.
User avatar

Tintin Quarantino

Rain Partier

Postby Tintin Quarantino » Mon Feb 11, 2013 10:58 am

Let go of the past and erase it all.
User avatar

S.F. Jude Terror

OMCTO

Postby S.F. Jude Terror » Mon Feb 11, 2013 11:01 am

I actually have a cool solution for handling. Real quick explanation then I gotta go:


In order to have search enabled, the board keeps an index on post subject, post text, and both together.

This essentially duplicates the post table twice.

Every time a post is made, both indexes must be updated.

Means we're altering 2.5 million rows every time.

I plan to replace search with a custom search engine.

It will separate those indexes, reducing the posts table by 2/3, making posting much faster.

With posting faster, the tables won't be locked when trying to read from it.

Search will also go faster, being its own table.

It's just gonna be some work to get that going.
User avatar

S.F. Jude Terror

OMCTO

Postby S.F. Jude Terror » Mon Feb 11, 2013 11:03 am

Another thing I'm seeing is brutal is the jump to new post link on large threads. Looking at a way to disable it.

It requires, instead of a query that loads 50 or so posts, going through 115,000 posts or something (on the twenty-somethings thread for instance) in order to make the pagination.
User avatar

Keb

<( ' . ' )>

Postby Keb » Mon Feb 11, 2013 11:25 am

I say we prune the forums!
User avatar

IvCNuB4

Staff Writer

Postby IvCNuB4 » Mon Feb 11, 2013 11:29 am

Victorian Squid wrote:Let go of the past and erase it all.


:lol:
Seriously. How much additional space can we also free up if we delete all the fucking endless "Legion of Super Heroes reboot is an abortion" and "it just gets worse" sales doom-n-gloom posts ? :lol:
User avatar

superfictious

Humuhumunukunukuapuaa

Postby superfictious » Mon Feb 11, 2013 11:33 am

IvCNuB4 wrote:
:lol:
Seriously. How much additional space can we also free up if we delete all the fucking endless "Legion of Super Heroes reboot is an abortion" and "it just gets worse" sales doom-n-gloom posts ? :lol:


Are you saying this "maintenance" is a secret ploy to get rid of Nac and Herald? :o
User avatar

Keb

<( ' . ' )>

Postby Keb » Mon Feb 11, 2013 12:04 pm

Man has a point. Mega threads don't help andthe good ones we can always recreate. A fresh start would be good.
User avatar

sdsichero

2k11 Outhouse People's Champion

Postby sdsichero » Mon Feb 11, 2013 12:23 pm

Victorian Squid wrote:Let go of the past and erase it all.


Jude is going to do the Superboy punch/Flashpoint??? :shock:
User avatar

S.F. Jude Terror

OMCTO

Postby S.F. Jude Terror » Mon Feb 11, 2013 12:33 pm

History is important. :P


I have disabled search and I am importing all the posts into a new database table, one with less indexes, which is INNODB, which supports row level locking (making a post won't prevent reading from the table).

This will probably take most of the day. After that, we will begin rebuilding the search engine, and search will return (this could take several days). After that, everything should be a fuck of a lot faster.
User avatar

S.F. Jude Terror

OMCTO

Postby S.F. Jude Terror » Mon Feb 11, 2013 1:59 pm

Test...


Posts table is now INNODB. Let's see if the awful slowness goes away now.
User avatar

S.F. Jude Terror

OMCTO

Postby S.F. Jude Terror » Mon Feb 11, 2013 2:03 pm

Next I will be rebuilding search.

leave a comment with facebook


Who is online

Users browsing this forum: acousticgorilla, Mac E Tero, MSNbot Media, sdsichero, Stephen Day and 20 guests