We are performing scheduled maintenance on the EventEspresso.com database. EventEspresso.com will be in maintenance mode (unavailable to view content or access your account) while we backup the database. If this scheduled maintenance on the database should cause interruptions, it should be relatively brief for approximately 2 hours or less starting at 11:30ET.
We spent the majority of today putting in place some measures to prevent DDOS attacks in the future and implemented some rate-limiting on the api endpoints for our plugin update engine. While doing this, we also noticed an inefficient query that compounded the server problems and we fixed that.
Long story short, automatic updates have been re-enabled. However, there is the possibility that we may still have to fine tune some of the measures we’ve put in place. If any of the following happen to you..
- You have multiple EE plugins installed and get update notifications for some that have updates but not others.
- You are browsing our site, interacting with forums and get a cannot connect message seemingly randomly at intervals.
…please contact us and we’ll look into things and see if there’s a problem with some preventative measures we put in place on our server.
Again, thanks for the patience during this experience folks!
Hi folks, as the developer responsible for finding out the cause of our recent server interruptions I just wanted to give you an update on what we’ve discovered and what we’re doing to hopefully prevent this from happening again.
As you are no doubt aware from some of the high profile sites that have gone down over the years, no matter how big or small you are in the internet worlds, there are always people out there who think its okay to cause harm to others on the internets (for whatever lame reason they may manufacture).
What was the cause?
What we discovered, is that there was a ddos attack against eventespresso.com and the attack vector they used was the api we have in place for recording uxip stats and our plugin update engine (used for autoupdates of plugins, and key verification).
How are we addressing this?
Temporarily, we have added site-wide rate limiting to the api. This means that for many of our clients, auto updates and uxip stats (if you opted in) will not function as normal. This will not impact your EE code hosted on your site. You just won’t be able to auto update until we get the fix in place.
Within the next 48 hours, we will be implementing a more permanent solution that will hopefully prevent this from happening again, we’ll update here and on our blog once that is in place and you’ll be able to resume normal auto-updates again.
Please Note: The changes we’ve put in place temporarily should not affect your ability to interact with EventEspresso.com and download plugin files from your account or post on the forums. Again, all this will affect (temporarily) is the ability to do one-click updates via your site dashboard.
Thanks for your patience while we’ve been trying to resolve this. The timing of this attack was unfortunate as much of our team (including myself) was traveling back home from WordCamp San Francisco this weekend and we were unable to address the problem sufficiently until we arrived home.
We are experiencing unexpected service interruptions and are working hard to bring the EventEspresso.com back online. Please stay tuned for updates.
We’ve successfully moved eventespresso.com and are currently in testing. It’s accessible however we are still monitoring and testing.
Eventespresso.com may be unavailable between the hours of 10 PM and 12 AM EST to a migrate to a new hosting provider. Stay tuned for further updates.
The database optimization maintenance has been completed with far less downtime than expected.
We’ll be taking EventEspresso.com offline for about two hours this evening to perform some database optimization maintenance. The EventEspresso.com website is scheduled to go offline at 10PM MT. Stay tuned for updates.
Thanks for your patience folks. Just one of those things we do to kEEp things runnEEng 🙂
We have some tests we usually run after an update that we run through. But things should be good to go soon.