You are at netAirspace : Forum : NOTAMs - Site Related Forums : About netAirspace

Perfect

Problem with sorting on "photos in detail" view. When I try to sort on "Reg", "Operator" or "type", pop-up window with the following message appears on the screen:

"Sorry, there was an error on the server. The page will reload when you click OK."

and after that the next pop-up window with code appears. Sorting on the rest seven columns from "Airpt" to "Uploaded" works correctly.

WalAndPl wrote:Problem with sorting on "photos in detail" view.

Thanks, I think I've fixed that now.

Nightly jobs: One choked on duplicate dates in a table, which existed as a result of recovery from previous crashes and needed cleanup. All three nightly scripts ran to completion when triggered manually, and I got photo PMs.

In about 24 hours we'll know whether they complete automatically now.

It all locked up again this morning. It didn't when the nightly jobs didn't run, which is a concern. I've just scheduled the backup to run earlier and the jobs later, in case they're interfering with each other, and we'll see what happens tomorrow.

ShanwickOceanic wrote:

WalAndPl wrote:Problem with sorting on "photos in detail" view.

Thanks, I think I've fixed that now.

I checked today sorting on each column and it works without any problem. Thank you very much!

I hope you will find the failure reason during the back up time, that netAispace isn´t offline for a few hours.

But nevertheless many thanks for your hard working in the background!

Just for info.

The last days the site was offline before 6 am. Today for a short period online around 8 am and then offline the next for 1,5h.

Thanks for that. At the moment I'm restarting the database server every morning as soon as I wake up, which is clearly unsustainable (especially if I want to have a lie in at the weekend!). On Sunday morning I was up early enough to watch it all go down, in real time, which was informative.

Despite my efforts to prevent it, several automated jobs kicked in while the database dump was ongoing, including the 5-minutely homepage refresh. Each of these needs disk and database access, while the database is both locked out and hammering the disk. What should happen is that the automated jobs either exit immediately (for the 5-minute job) or sleep until the backup is complete, and that didn't happen. I'm investigating that at the moment.

Once things are stable again, I can look at eliminating the nightly downtime for backup, but that's a significantly harder problem.

FYI
Images opening since early afternoon slow.

Apologies for an extended outage while we ran some analysis.

Yesterday from noon till late evening or longer was the site offline or maybe very, very slow.

Today since 11:30 slow after a few minutes impossible to open images.

Sorry to say this, but it´s very frustating to use netAirspace. Yesterday I thought wow all problems were solved, it was running fast and stable. But today till 11:00 the site was still slow or mostly dead.

I know, you do everything for solving the problems but for almost two weeks it´s a gamble if the site is online, slow or offline.

For my part, I will stop uploading and wait for better times.

Nevertheless thank you so much for your commitment.

I know, it's frustrating on this end too. I thought we'd nailed it.

Apologies for many random short outages this evening. Several configuration changes needed a server restart.

We've done some work to further reduce the impact of being crawled by search-engine robots, and to resolve some issues around SSL/HTTPS. Even though these are now in effect on our side, caching by other parties may cause a delay of up to 24 hours before they take full effect.

As with most of the other things we've done, I'm by no means certain that they are the root cause, but they should definitely help. In particular, the random nature of the hangs, and the specific database queries being run when they happen, strongly suggest a runaway bot running rampant through database-intensive parts of the site that it should never have been able to see. The changes made assume that the bots involved are friendly and will do what they're told.

Very early days but things are looking a lot better today than they have for a while. The site's the fastest that I've seen it in a long time, and we've managed a full 24 hours without going down completely. (This should not seem like an achievement.)

Some nightly jobs are still not completing, but for now, I want to be sure we're stable.

Yes, it´s fantastic how stable and fast the site working.

Thank you so much!

The only thing what isn´t correct working is the view counter. For the first hours it´s working then no more views are coming and after several days it´s working again. That´s very strange.

The view counter for individual photos shouldn't be affected, except perhaps by caching for an hour or two.

I moved several nightly tasks, including some user-view-counter-related stuff, into a third job file to get them several hours away from the other DB- and disk-intensive tasks, but they have been unreliable since then.

Now things appear to be stable, I'm going to put them back where they were (albeit delayed by half an hour). If the site stays up tomorrow morning, and the jobs run, we're winning.

It seems that we're out of the worst of this. My deepest apologies to all of you who've had to put up with the disruption.

For anyone who had First Class membership while all this was going on, be that introductory, bought with money, or bought with Miles: I've gone through the logs and added a month of First to your accounts. If this is you, and you don't see "Compensation for site performance" in your statement, shoot me a PM.

I've had a good play around with a lot of the site's functions and we appear to be running better tan we have for a long while.

Apologies that I wasn't much help - between the outages and Wales' idea of an internet connection (which I'm fairly sure was coal powered) there wasn't much I could do.