Well that was fun! page 1

oblivion
27th December 2011, 03:28 PM
Sorry about the downtime and all the database too busy errors last night. It was a MySQL problem that started on TR (and we're still having problems over there). It looks like the TR post table has crashed.
oblivion
28th December 2011, 05:12 AM
It was way more fun than even I anticipated.

TR is still down, so there may be some more rocky moments ahead while the hosting service gets everything ironed out. I thought they were restoring the database from their Sunday night backup. It apparently hasn't happened yet.
charlou
28th December 2011, 05:19 AM
hey ob ... no worries .. these things happen. Thanks for the Email update earlier.
oblivion
28th December 2011, 05:38 AM
Well, TR appears to be up, but very, very slow. Somehow they were able to repair the databases after all, rather than restore a backup. We're not sure why the database is so busy still, but it seems to be slowly moving in the direction of normal, so I'm crossing my fingers we won't lose a day's worth of posts after all.
Magicziggy
28th December 2011, 06:49 AM
I can sense the perspiration...

The worst is over...

Chicken soup and an early night.
Hermit
28th December 2011, 07:44 AM
Best of luck with getting TR back up to speed.

This malfunction gives me a nudge to ask something I meant to ask earlier: What are our backup arrangements? How often are they done, and are there any being stored off-site? The latter will be useful if there is a hardware failure on the internal ones, and also if defective tables overwrite good ones before anyone notices that there is something wrong with them.
oblivion
28th December 2011, 10:07 AM
backups are being done twice a week. I don't know the details of the offsite aspect - how often a backup goes offsite. They have an option for continuous backup/recovery with offsite storage for applications where the loss of a couple days of data would be unacceptable, but of course you pay for it. The Guardian continuous backup and recovery would add $100/month to the overall server costs.
nostrum
28th December 2011, 10:28 AM
We're back! Poor TR I'll have a mosey on over to see how it's doing.
Adenosine
28th December 2011, 10:41 AM
It's doing all a gangly. Posts in raven's test thread are being replaced with posts from other parts of the forum. It's quite awesome.
oblivion
28th December 2011, 10:56 AM
Yeah. It looks like a combo of an index problem and the post cache holding posts that were lost in the restoration.
oblivion
28th December 2011, 10:22 PM
ok...full database restore for TR this time. I'm hoping it won't have any effect on mindromp this time, other than possibly a bit of a slowdown.
nostrum
29th December 2011, 02:00 AM
Bugger... is this the worst mishap to hit TR yet?
oblivion
29th December 2011, 02:39 AM
by far. by very, very far. but I've seen it (and worse) happen on other sites once a database gets to a certain size. I didn't think we had hit that certain size, though. I am going to have some work to do next week!
nostrum
29th December 2011, 03:02 AM
Blerk. Luckily you like that geeky work!

Sounds like what happened to RDF once... a few days' posting was lost and it was down for quite a while. I guess the silver lining is that TR is now all growed up and big..
oblivion
29th December 2011, 03:16 AM
Yay!!!!!!!!!!!!!!
Hermit
29th December 2011, 03:50 AM
by far. by very, very far. but I've seen it (and worse) happen on other sites once a database gets to a certain size. I didn't think we had hit that certain size, though. I am going to have some work to do next week!At least you had backups. A few years ago another forum I am a member of experienced a major database meltdown. It turned out to be beyond repair. Then it was discovered that no arrangements for backups had been made since the forum's move to another host. Several months' worth of posts and private messages disappeared irretrievably into the digital abyss. We were not amused.
oblivion
29th December 2011, 03:58 AM
that's horrible. :(

were the public posts at least available via the wayback machine?
Hermit
29th December 2011, 05:03 AM
that's horrible. :(

were the public posts at least available via the wayback machine?I found a few dozen forum pages that were cached by Google and nothing at all on the wayback site. Many thousand posts just went into - dare I say it? - oblivion. Even though I am not a prolific poster, nevertheless managed to lose about 550 posts, only a few of which I regret not having kept a locally backed up copy of. What I miss a lot more are many of the private messages that were also wiped out.

Nhận xét

Đăng nhận xét

Bài đăng phổ biến từ blog này

I don^t wanna go to work tonight! D: page 1

Gonzo 14th January 2012, 04:38 PM because then I would have to sleep. >:[ oblivion 14th January 2012, 04:39 PM ((((gonzo)))) triple expresso is thy friend. Daphne Girl 14th January 2012, 04:43 PM Gonzo- Do you work overnight shifts? Gonzo 14th January 2012, 04:44 PM yes nostrum 14th January 2012, 04:53 PM Condolences ((gonzo)) Gonzo 14th January 2012, 04:55 PM IT'S NOT THAT BAD... BUT i NEVER HAVE TIME TO READ ANYMORE, PLUS ALSO POST ON THE INTERWEB, I AM JUST LEAVING MY CAPS LOCK ON NOW BECAUSE IT DOESN'T EVEN MATTER. Gonzo 14th January 2012, 04:55 PM whoa :stare: nostrum 14th January 2012, 05:07 PM :staregonk: oblivion 14th January 2012, 05:08 PM :shockcorn: Mantisdreamz 14th January 2012, 10:57 PM whoa :stare: ((Gonzo)) I've got to work overnight too. and yes, coffee/espressos, etc help. I have yet to try one of those 'energy drinks' that so many ppl i work with like to drink. Apparently it's like having 4 espresso shots in one drink. :stare: Fuz...
Đọc thêm

Railroad tracks in the sky page 1

Mysturji 8th February 2012, 07:01 PM When I was a little kid, I used to like to imagine that high-flying jets leaving vapour trails behind them were laying railroad tracks in the sky, and that behind every such plane was a sky-train. Now, decades later, I still think that is a remarkably poetic image to spring from the mind of a little boy. In the interest of balance however, I should admit that I also liked to imagine - when taking a piss - that any remnant of toilet paper (or anything else) in the toilet was an enemy marine base, and that my penis was an enormously powerful space-based orbital laser with orders to destroy the base. What childhood imaginings do you remember? divagreen 8th February 2012, 07:08 PM I used to pretend my crayons were people and had whole cities made out of cardboard boxes and wore paper clothes. They even had a solid disco gold show and price is right games and a geneology chart. Pretty sure my parents thought that I was dumb and that I just did not kn...
Đọc thêm

Feed Students Semen = Collect Full ,000/mth Pension page 1

Jerome 3rd February 2012, 01:17 AM Feed Students Semen = Collect Full $4,000/mth Pension (http://www.huffingtonpost.com/2012/02/01/mark-berndt-lausd-responds-to-teacher_n_1247138.html) Los Angeles Unified School District (LAUSD) Superintendent John Deasy confirmed with Fox 11 News (http://www.myfoxla.com/dpp/news/local/lausd-reacts-to-teachers-arrest-20120131) that he has seen some of the more than 400 photos of teacher Mark Berndt allegedly spoonfeeding his semen (http://www.huffingtonpost.com/2012/02/01/excalif-teacher-in-court-_n_1246675.html?1328110095&ref=los-angeles) to blindfolded and gagged students, some with cockroaches on their faces. As the Los Angeles Times reports (http://latimesblogs.latimes.com/lanow/2012/02/la-teacher-suspected-of-lewd-conduct-keeps-benefits.html), Berndt will receive $4,000 a month in pension for life from the California State Teachers' Retirement System as well as lifetime health benefits from the district. Poor oppressed underpaid t...
Đọc thêm