The Outhouse - The Greatest Comic Book Forum

[S.F. Jude Terror]

In a little bit, I'm going to run an optimization on the posts table on the database. The maintenance page will be up with a live chat while it runs. It shouldn't take more than an hour at most. The regular forum will come back as soon as it finishes automatically.

Trying to get something together that can run automatically at some late hour on a weekend each week to keep the tables running smoothly. This will be the first test run.

[covalesky]

Don't know if its related or not, but I've been getting the search not allowed atm error throughout the weekend most of the times I try to see the newest posts.

Also getting the Thread does no exists when I click the go to last post icon in the threads list.

[sdsichero]

Yeah I can't search either. That's a bummer.

[S.F. Jude Terror]

Search is disabled when server load is too high. I have a replacement for this coming next week hopefully that will speed everything up. The server is a fucking nightmare right now, mostly because of all the big threads at this point, but in general because there are 2.5 million posts. It's completely overloaded trying to sort through them.

[Victorian Squid]

Let go of the past and erase it all.

[S.F. Jude Terror]

I actually have a cool solution for handling. Real quick explanation then I gotta go:


In order to have search enabled, the board keeps an index on post subject, post text, and both together.

This essentially duplicates the post table twice.

Every time a post is made, both indexes must be updated.

Means we're altering 2.5 million rows every time.

I plan to replace search with a custom search engine.

It will separate those indexes, reducing the posts table by 2/3, making posting much faster.

With posting faster, the tables won't be locked when trying to read from it.

Search will also go faster, being its own table.

It's just gonna be some work to get that going.

[S.F. Jude Terror]

Another thing I'm seeing is brutal is the jump to new post link on large threads. Looking at a way to disable it.

It requires, instead of a query that loads 50 or so posts, going through 115,000 posts or something (on the twenty-somethings thread for instance) in order to make the pagination.

[Keb]

I say we prune the forums!

[IvCNuB4]

Let go of the past and erase it all.

Seriously. How much additional space can we also free up if we delete all the fucking endless "Legion of Super Heroes reboot is an abortion" and "it just gets worse" sales doom-n-gloom posts ?

[superfictious]

Seriously. How much additional space can we also free up if we delete all the fucking endless "Legion of Super Heroes reboot is an abortion" and "it just gets worse" sales doom-n-gloom posts ?

Are you saying this "maintenance" is a secret ploy to get rid of Nac and Herald?

[Keb]

Man has a point. Mega threads don't help andthe good ones we can always recreate. A fresh start would be good.

[sdsichero]

Let go of the past and erase it all.

Jude is going to do the Superboy punch/Flashpoint???

[S.F. Jude Terror]

History is important.


I have disabled search and I am importing all the posts into a new database table, one with less indexes, which is INNODB, which supports row level locking (making a post won't prevent reading from the table).

This will probably take most of the day. After that, we will begin rebuilding the search engine, and search will return (this could take several days). After that, everything should be a fuck of a lot faster.

[S.F. Jude Terror]

Test...


Posts table is now INNODB. Let's see if the awful slowness goes away now.

[S.F. Jude Terror]

Next I will be rebuilding search.