Fixed the Paypal Donate button

I guess the Paypal donate button (to the right on the Flaming Family Web Page). It is now fixed.

Email Up and Running...

In case you didn't notice, Email is back up. The following is from the service provider.

I figured I'd share this as I could definitely feel their pain in this situation - having been in it myself.

Another Outage, Really? Yes, really. I regret having to write to you again so soon, but the latest outage of our IMAP/POP3 email was a very severe one that lasted for over 30 hours. After the last outage, we thought we had fixed the issues, but there were obviously some lingering, unforeseen bugs.

What Really Happened? Let's start from the beginning... A few months ago, we migrated all of our hardware out of a data center in San Diego. San Diego was where email was originally housed. Upon moving, we changed the structure and hardware that was handling IMAP/POP3 in order to make it more reliable and redundant.

We added new redundant switches, raid controllers, storage servers. Even the Ethernet links that connected these were redundant. After much testing, we decided on Oracle's OCFS2 filesystem, but that is where the trouble came, the one part of the system that could not be redundant: the filesystem. Bugs cropped up after the structure was put into production. We worked with Oracle to have them fixed, but their patches were no help. After the Nov 2nd's failure, they then proceeded to update our case to "Severe", but we never heard from them again.

After some trial and error, we were successful in getting the filesystem mounted in a read-only state. Instead of waiting for Oracle to figure out how to allow writes again, we began transferring the email to alternate hardware with a known reliable filesystem. The time estimates were made on how long it would take to move the data across the network. But email is a funny thing that way, the hundreds of gigabytes of data was not the problem, it was the shear number of tiny files being created on the filesystem. Some accounts had over 200,000 files that were only a few hundred bytes long. This is why our time estimates were so grossly incorrect.

We were in the dark just as much as you were as far as a time frame for when the data transfer would be completed. It was a long and painfully slow process.

Is IMAP/POP3 stable now? Yes, it is now structured the same way that it has been for the past 8 years. During those 8 years, we have never had any major outages, so we are confident that this solution is the right one.

So, what happens now? We have come to realize that managing email has become far more costly and time consuming for our small team to manage in house. We could hire more staff and purchase bigger, more expensive hardware, but this would in turn drive up our low prices and our customers would suffer. So, we realize that we should ultimately focus on what we do best, Managed DNS.

In light of this, we are joining forces with an email provider that has the time and resources to manage millions of email accounts effectively. They also have an uptime history of 99.999%, so outages are out of the question.

Does this mean No-IP will no longer manage my email? Not exactly. We will still manage your email, but another company will be maintaining the infrastructure of your mail. All support calls for IMAP/POP3 will still be directed to us. Mail forwarding will also be handled by the new company.

But what about my other Email Services: Alternate-Port SMTP, Backup MX and Mail Reflector? No changes will be happening to these services. We will still be managing all of them in-house and will continue to provide support for any issues that you have concerning these products.

What will the transition be like? Most importantly, will I still have all of my data? All of your emails, contacts, and calendar items will be transitioned as part of the migration process. The only difference you will probably notice is in the webmail interface, which will be different.

Will there be any downtime associated with this transition? Although we are still in the planning phase of this transition, we are hoping to have little to no downtime during this transition, and if there is downtime, it will be kept a minimum, during off peak hours.

We hope that you stick around with us. Email is going to be the same low price that it has always been, and it will now be backed up by a solid 99.999% uptime guarantee and honestly, you won't even notice a difference in your email, well except for the uptime ; ).

Email is important and that is ultimately why we have reached this decision.

The transition to the new service will be as painless as possible. There will be little effort on your end and we will be doing most of the legwork. At most, you will probably have to update a record or two.

You will be receiving information in the coming weeks with information about account migration.

In light of the recent outage, we are offering you a free month of email service, and have already added it to your account.

We thank you for your continued patience and for being a valued customer of No-IP. If you have any questions about anything, please do not hesitate to reach out.

Email Down

It seems email has been down by the service provider... No-IP has had several issues with email over the last day or so. For updates, see @NoIPStatus on Twitter.com or the No-IP.com Support page.

Blog Updates

If you're wondering why the multiple re-posts, it's because I updated a number of their settings on the blogs and accidentally unpublished and republished those articles.

Apologies...

Servers Decommissioned - Now Using Email Service

What Happened?

Well, the primary server (host to all the virtual servers within) had a glitch in it's drive array.

Some Background First...

To explain, there's a drive array module which controls the computer's access to the hard drives. With this card, I can pair a number of drives together to logically so they appear to the operating system (i.e. Microsoft Windows, Apple Mac OS, VMware ESX/vSphere, Novell Netware, etc.) as one big drive or a couple of big drives.

When a drive array spans 3 or more drives, the drive array starts to use parity technology for redundancy. This is, in short, a mathematical calculation where a block of data is stored on any of the drives and it's mathematical checksum is stored on the other drives. What this does is provide a high level of redundancy in the event a hard drive within the array of drives goes bad. Very cool so long as fewer than half of the drives in the array go bad at one time.

Back to the Issue...

None of my drives are reporting any issues.

In my case the array glitched (similar to losing power or power spike) and lost the drive configuration. In higher end array modules (and even in this one) I'm supposed to be able to pull the array configuration from the drive back to the card. That failed due to something with the glitch.

What About Backups?

I very likely could have restored from backup if I had backups of each virtual machine - remember though, each virtual machine is essentially a complete computer in and of itself (and I have - correction, had - quite a few). So, unfortunately, I do not have the resources (technically or financially) to do backups of these systems on that level. And I learned long ago with email and database systems, if you don't go all the way with backups (and backup everything) it's best to just do what you can to minimize your losses and downtime.

You did maintenance on the servers the other day. Did that have a play or possibly cause this issue?

Short answer, No. I was done with maintenance on Saturday and the server crashed while I was out buying presents on Tuesday around noon. Plus, what I did had no relation to the drive array system.

My Analogy of the Issue...

Probably the best way to explain this is, if you drive a full stretch Cadillac and go to the grocery store and fill up that huge trunk with groceries, and then 30 miles down the road simultaneously all four tires blow and the transmission breaks, do you think it's because the trunk is full of groceries or is it possible something else which may have happened (like running over glass in the roadway)? :-)

As for backing it up, we could probably have four spare tires in the trunk as well as a spare transmission - but it's unlikely anyone would go to that length of protection on a personal basis.

What Now?

For anyone using my email server, I've transferred it out to a service (provided by No-IP.com) so it will no longer go down (at least - not because of something I've done, or my Internet connection, or my server glitching). I've gone ahead and paid the first year service fees to get the transfer going and email is already flowing in. This service was already providing basic anti-spam services (I didn't have them set too high) so it will continue to do that - and we can also increase the spam protection services as well as the settings for individuals. I will also be re-setting up the reflection addresses which existed in my own server (things like group addresses which forward/reflect email to all members of the group).

What are the differences?

Here are the differences which you'll notice:

• No server based calendar. Personal calendar within your email client (i.e. Outlook, Apple Calendar, etc.) will be useable.
• No server based address book. I think everyone had their own address book on their computer anyways. This will still work the same.
• Web Email will have a different look and feel.

If you wish to use another email service (such as Comcast or Verizon) and want your "@flaming.ws" or "@feely.ws" email forwarded to it, please let me know.

Also, the reverse is true - if you wish to use the "@flaming.ws" or "@feely.ws" email service and currently have another service, please let me know. Either way can be accommodated and this email service is guaranteed to be up 99.99% of the time.

What Else?

For those of you who are using my Corporate Antivirus license, that server is also gone. Your Antivirus client will be fine for a couple of weeks if you wish to buy another (I would highly recommend it). I would recommend Trend Micro or AVG. I've not used AVG but have heard it is good from others. For those who have multiple computers (I think everyone does) and want to purchase Trend Micro, I've provided a link to the online store which allows you to purchase Trend Micro for multiple computers.

A couple little tricks on purchasing Antivirus software - specifically Trend Micro

1. Purchase it ONLINE from Trend Micro. Don't go to the store! You'll pay more at the store!!


2. Buy the best one. It's worth it!




3. I really like Trend Micro, but when purchasing Trend Micro or any other software, be aware of the "Download Protection Service". It's a waste of money! You'll always be able to download a copy of Trend Micro's software - no matter the situation!



Hit the "Trash Can" icon to remove the "Download Protection Service" and save yourselves about $8.


4. Don't skimp on virus protection! Remember everything you have on your computer - and what a malicious person could do with it. Trend Micro is having a serious sale right now so now's the time to purchase for your Windows pc.

Is that everything?

Surprisingly, that's about it. Amazing how a complete system wipe can bring you back to a clean slate.

The only other thing is an apology for any inconvenience or issues this may have caused to anyone. Know that your email is now in better hands now and will not go down.

Intermittent Outages This Week

Just a reminder....All servers are back online (flaming.ws and feely.ws email as well as other - FTP, etc.)! Currently I'm trying to fix the Antivirus server for those of you who use my corporate AV license for your Windows systems.

However, please don't be surprised if there are temporary outages this week while I fix issues (the AV issue and others), perform maintenance and cleanup. And if my new server arrives this weeks, there will be some longer outages while I install it and move systems around to it.

So for now, all is well. As always, I will keep the Flaming Family web site updated so you can always go to www.flaming.ws for any updates and post a comment if you need anything.

All Servers Are Up

All servers are back online!

Turns out the new Intel PRO/1000 Dual Port Network Adapter
was not the correct item needed for the additional capacity necessary to support the move of the firewall. I ended up reconfiguring my VMware ESX virtual host server to accommodate the virtual appliance firewall with the hardware configuration I was already running. In the process, I also upgraded the firewall to a newer system as well.

I won't be able to fully load it until I get the new server in to offload a few things to it, but the new virtual firewall does run better than my old Dell Dimension (a Pentium Pro 400Mhz system with 650MB of RAM and about 10GB in hard drive space - archaic by today's standards).

So for now, all is well. As always, I will keep the Flaming Family web site updated so you can always go to www.flaming.ws for any updates and post a comment if you need anything.

Servers are Down for Hardware Updates

I am bringing the servers down at this time (email) to update hardware in preparations for the firewall migration from the physical white-box firewall into a virtual appliance firewall.

In case anyone is interested, I will be installing a new Intel PRO/1000 Dual Port Server Adapter
network adapter for the additional capacity necessary to support the virtual appliance firewall.

During this time, I will keep the Flaming Family web site updated so you can always go to www.flaming.ws for any updates and post a comment if you need anything.

Computer Updates for Thanksgiving Week

Just a reminder, during the week of Thanksgiving, there will be some server outages to those of you using my email server (@flaming.ws and @feely.ws).

I will be conducting at least one upgrade if not more on the following:

Firewall Move:

• Migration of the physical white-box firewall into a virtual appliance.
  
• Physically installing a couple of Network Ports.

Additional maintenance may also occur.

1. Server Operating System Updates.
   
2. Server Reboots.
   
3. Service Migrations (License Servers, Log Servers, Web Servers) to other systems.
   
4. Service Maintenance (cleanup of aforementioned services and logs).
   

During this time, I will keep the Flaming Family web site updated so you can always go to www.flaming.ws for any updates and post a comment if you need anything.

Bad Drive?

So my very first laptop has finally died - or, at least, the hard drive within it died (my Dell Inspiron 8600 I bought 6 or 7 years ago). I only had it running a web cam, a license server, and a network attached drive which was backup to some other things so no huge loss (had all the data copied off elsewhere too). Still...annoying as I now have to recreate the services elsewhere.

Anyways...I thought I'd share this picture as even my diagnostic tools are indicating a "bad drive".

Click picture for larger image.


To help interpret this screen, in 27 hours, 21 minutes, and 42 seconds, it's only scanned two sectors of the drive and the second is not only bad but unrecoverable. It's working on the third segment - so less than 0.3% complete. To bring perspective here, this tool should only take about 30 minutes to scan a 350GB drive (the size of drive here). It's estimating another 11,142 hours, 37 minutes, and 4 seconds. In case you can't do math that quickly, that's 464 days (or 1 year and 99 days), 6 hours, 37 minutes, and 4 seconds. So...yeah...bad drive. :-)

Oh well....that's why we have a technology called "Backup".

I.T. Rule #1 - Backup, Backup, Backup!
I.T. Rule #2 - When all else fails, revert back to rule #1.


- Posted using BlogPress from my iPad

Upcoming Server Outages

Just a forewarning, during the week of Thanksgiving, there will be some server outages to those of you using my email server (@flaming.ws and @feely.ws).

I will be conducting at least one upgrade if not more on the following:
Firewall Move:
a. Migration of the physical white-box firewall into a virtual appliance.
b. Physically installing a couple of Network Ports.

Additional maintenance may also occur.

1. Server Operating System Updates.
2. Server Reboots.
3. Service Migrations (License Servers, Log Servers, Web Servers) to other systems.
4. Service Maintenance (cleanup of aforementioned services and logs).

During this time, I will keep the Flaming Family web site updated so you can always go to www.flaming.ws for any updates and post a comment if you need anything.

- Posted using BlogPress from my iPad

Email is Up!

Flaming.ws/Feely.ws Email is back up and working.

Maintenance was successful.  I moved the virtual machine which is the email server to another virtual server host and was able to power off one of my hosts.

So now my office is much quieter and I don’t have to pay the electricity for that host to run just one virtual machine.

Now, everything comfortably resides on one host.

Next move is to see if I can get Zimbra to replace Exchange for email services.

Note in the above picture how much memory and CPU Exchange (FlameMail1 VM) uses vs. how much Zimbra uses.  Both VMs have been allocated 2GB of RAM (Memory).  Exchange uses nearly all of it (note the bar graph in the right column is in the red), Zimbra uses almost none.

But, that’s another project for another time. 

Happy emailing for now and thanks for your patience!

Email Outage

Flaming and Feely Email is currently down for maintenance.

Upcoming Email Outage

Sometime over the next couple of days, I will be attempting to move my email server around for maintenance.  No email will be lost and any new incoming emails will be queued up for delivery as soon as the server is back up.

The email server doesn’t run too well (it’s Microsoft...what else can I say) and I’d like to ensure it runs solidly before leaving on extended vacation to Italy this fall.

I’ll likely start the move tomorrow (Thur) so please be aware you will be without email while the move is occurring.

For updates, please check the Flaming Family website!

Computers Hate Me - I Hate Computers (Sometimes)

I think the little Computer Bugs are on the loose.  No sooner did I finally bring my email server up from the 4 failed attempts at moving it to a new Virtual Server Host, then my firewall's intrusion protection system went haywire and knocked me off the Internet until now.

It took me awhile to figure out where the issue was (only about another 8 hours) - and after multiple times rebooting the Comcast router and my firewall, I found it was my firewall not talking to the Comcast router instead of the other way around (usually the case).

Sometimes I just want to pull one of my brother's stunts from our childhood and throw a Rubik's Cube into the wall.

Wouldn't you just love to watch this thing blow up into a million pieces?  :-D

Email Server Move Cancelled - For Now

Well, apologies for the email system being down for the last 24 hours.  In short, I was hoping to move the flaming email server from one of my hosts to another and the move has been giving me grief.

So, after the fourth failure, I terminated the move and brought everything back online.  Until I figure out why the grief on the move, it'll remain up...and I'll warn everyone next time.

Also, during the down time (about 24 hours), no data was lost and all incoming email was stored so nothing was dropped.

Computer Updates

Well....the mail server move goes slowly....(as my funds and resources are limited, I cannot afford a super fast data center).  It is about 95% through the move right now.  It timed out earlier due to the slow host I'm moving it from...so hopefully it will not do so again (as I can only hit retry).

Dad's computer is looking much better as well.  I was able to import the image backup into a virtual machine, tweak on the settings a bit, and then get the system to load up.  Again, thank goodness I have a MacBook Pro - and VMware Fusion to create a virtual machine and dump the image backup of the system into it so I can tweak on it virtually without messing up the hard drive.
☺

I've been cleaning it up and updating the OS with patches and all the apps with their respective updates.  To do this, I've been using Secunia and CCleaner by Piriform (short for "Crap Cleaner").  Secunia updates all of the non-Microsoft (and MS apps as well) apps in much the same manner as Windows Updates does - but better!

And CCLEANER removes all of the "crap" left over from updates, runs, etc.

I love technology....but I love it even better when it just works!  I guess this is why I have a Mac!

Flaming Computer Work

Well...the Flaming family email server is down for the moment while I move it to another host. I had moved it to a separate host last week while working on some lab stuff but it's poor performance demands I put it back.

So, please be patient while it is moved back to it's original host.  It should be done in a couple of hours.


In the mean time, I'm attempting to repair my Dad's laptop - turns out something hosed up the drive partitions.  Fortunately, I was able to get in and backup all of the data (so it seems).  Right now, I'm attempting to restore the backup of the data to a virtual machine so I can work at repairing it and then image that repaired virtual machine back over to the original hard drive.

All repairs are underway....but just moving slowly.  Thank goodness I run a Mac!  ;-)

Anyways, please be patient while repairs/maintenance is in progress.

Thanks!

Flaming Family Server Upgrades

This past weekend we went through the process of backing up and upgrading our primary server from Citrix XenServer 4.1 to a VMware ESX version 3.5!

All primary Flaming Family servers are currently up and accessible!

Why did we do this? Well...the license was expiring on the primary server and it was WAY more expensive than we could afford.

Any additional questions can be directed to Support.



Details:

• We backed up the existing virtual machine servers using UltraBac UBDR Gold 4.0.
  


• We replaced the virtual server host operating system.
  Existing virtual server host operating system is Citrix XenServer 4.1.
  New virtual server host operating system will be VMware ESX 3.5.
  
  
  


• All incoming email was stored by No-IP and made available as soon as the guest virtual machines (VMs) were loaded on to the new virtual server host operating system.