INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: 7 TB DEGRADED!! Ahh!!  (Read 2392 times)

benn600

  • Citizen of the Universe
  • *****
  • Posts: 3849
  • Living: Santa Monica CA Hometown: Cedar Rapids IA
7 TB DEGRADED!! Ahh!!
« on: September 15, 2007, 10:43:48 am »

With my system, arrays go through (RAID 6):
Online (no problems)
Degraded (1 drive failure)
Critical (2 drive failure)
Offline (3 drive failure)  NO!!

Well, a drive failed last night!  I woke up, grabbed my iPhone, and started viewing my email as I walked down stairs.  A few emails from my Promise Raid Card!  They are somewhat common from time to time as it reports bad sectors on drive 4 once in a while.

So drive 16 had failed!  Oh great!  Luckily, the rad was working properly and it was simply Degraded and everything was still working.  I was almost going to try to use the same hard drive again -- maybe it was wrong!  But I decided I needed a new drive NOW.  I had just been thinking a DAY ago that I should order a new drive to keep on hand just in case.  Well, I didn't and it's Saturday which means it wouldn't ship till Monday or Tuesday and could be Wed, Thur, or Friday before I got it.  It's not worth the risk of waiting.

So it happened to be almost 10 AM and Best Buy was almost open.  I ran and got a WD 500GB SATA drive.

I stuck it in and Promise told me I had to restart the server to kick start the rebuild -- not a good policy!  Since this is a home server nobody really cared at all.  But in a business that's the whole point of all this hot-swappable stuff!

Strangely enough, when I put the new drive in it goes through a cycle of saying all the drives on that port are unplugged and then finally says they're plugged back in.  That scared me.  It went from Degrade to Critical to OFFLINE!  Luckily it came back up right away.

So I rebooted and it's now rebuilding.  A family member is watching a streaming DVD from the server and I'm accessing some other data.

So all in all, the process is going fine so far.  I knew that waiting was not a possibility.  On my way to Best Buy I started thinking why didn't I make some initial backups of a few things that I didn't have backups of.  Then I realized it would just add extra wear and tear that could further harm the array.

I just now started a 90 GB backup of some data.  I hope I don't regret doing it.

I'll obviously report back in good or bad fortune.

Addition: I just checked my records and it is one of the drives that I took out of an external enclosure.  The drive is at least 6 months old but up to 15 months old.  That still isn't very good odds, I guess.
Logged

Mr ChriZ

  • Citizen of the Universe
  • *****
  • Posts: 4375
  • :-D
Re: 7 TB DEGRADED!! Ahh!!
« Reply #1 on: September 15, 2007, 11:45:16 am »

Aren't Media Systems supposed to be relaxing?  ;)

benn600

  • Citizen of the Universe
  • *****
  • Posts: 3849
  • Living: Santa Monica CA Hometown: Cedar Rapids IA
Re: 7 TB DEGRADED!! Ahh!!
« Reply #2 on: September 15, 2007, 11:49:20 am »

Well, other than about 30 minutes of work, I'm currently listening to my favorite song while another family members is watching a favorite television DVD.  And I'm copying some data from the server!

It isn't relaxing when something goes wrong but it's rebuilding now and is at around 7%!

Even if I was using what other people had suggested--more, smaller arrays, I would still be at this point with a failed drive.

Besides, if I was using all drives independently, I'd be out 500GB!  Yikes!  Now, I am led to believe I haven't lost any data at all!
Logged

Mr ChriZ

  • Citizen of the Universe
  • *****
  • Posts: 4375
  • :-D
Re: 7 TB DEGRADED!! Ahh!!
« Reply #3 on: September 15, 2007, 11:57:24 am »

I'd be out 500GB!  Yikes!  Now, I am led to believe I haven't lost any data at all!

Ah alright, that can't be bad  :)

KingSparta

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 20063
Re: 7 TB DEGRADED!! Ahh!!
« Reply #4 on: September 15, 2007, 02:30:08 pm »

Quote
I stuck it in and Promise told me I had to restart the server to kick start the rebuild -- not a good policy!

I think it is more for the computer and OS to recognize the new drive
Logged
Retired Military, Airborne, Air Assault, And Flight Wings.
Model Trains, Internet, Ham Radio, Music
https://MyAAGrapevines.com
https://centercitybbs.com
Fayetteville, NC, USA

benn600

  • Citizen of the Universe
  • *****
  • Posts: 3849
  • Living: Santa Monica CA Hometown: Cedar Rapids IA
Re: 7 TB DEGRADED!! Ahh!!
« Reply #5 on: September 15, 2007, 07:11:38 pm »

But in a business isn't the whole point to not have to reboot?  I was thinking it was the card.  Once I inserted the new drive, it didn't even show up in the configuration area for the controller.  Once I rebooted, the rebuild process started automatically.

Now I will, once rebuilding is finished, see if the 500GB drive that "failed" still works.  It almost seemed like it would work fine because it sounded like it started up fine.  I wonder how likely it is for it to say a drive failed even though it didn't?  Anyway, I'll just add it to my backup routine if it still works--then I'll get another 500 GB backed up!

And I really need to order another to keep on hand!  I over paid $30 or so at Best Buy but would have saved even more with OEM online.  So it's a good idea to keep one on hand so I don't have to rush out to get one!  And there was only one on the shelf so they could have been almost out (unless more in the back)...and then there are other stores to try.
Logged

benn600

  • Citizen of the Universe
  • *****
  • Posts: 3849
  • Living: Santa Monica CA Hometown: Cedar Rapids IA
Re: 7 TB DEGRADED!! Ahh!!
« Reply #6 on: September 16, 2007, 05:39:04 pm »

The card keeps reporting bad sectors (BSL) on Disk 4.  However, since I'm notified by email, I get emails saying that but it's always about 4 emails each time.  It says error.  Then error fixed.  Then BSL logged.  Then BSL cleared...lol.  So it must not be sure.  I'm tempted to replace the drive.

And the 16th drive that failed, I have it back in a WD My Book case right now (it originated in one) and it's formatting on my desktop.  It's up to 80% so far.  I have a feeling that the drive is fine.  It would be nice to know why it was flagged as unusable and disconnected.  I often wonder what if it got brilliant and started randomly disconnecting drives!  It could easily destroy the array if it's algorithm for checking for failed drives malfunctioned!

I'm not very upset because now I may have this extra drive for backup.  So now I'll have two 500GB drives to backup to.

Now I'm tempted to want to build a small 3-5 drive enclosure with some drives in RAID5 so I can have a single drive letter to backup to (and one device).

I called Promise to ask about the BSL issue and the individual I spoke with couldn't stop recommending I immediately add a spare drive.  That really isn't possible, though!  I'd have to backup the data and re-create the array.  Besides, I'd lose 500GB.  So this caused me to go on this crazy research kick to figure out what kind of costs I would incur to build a 24-drive system.

Lol.  Wow.  Prices multiply very quickly.  Promise doesn't even make a 24-port card so I would be paying the higher premium for the other brands.  For an Areca or 3ware card, it is about double the price of my 16-port card at around $1,400.

Plus, my 16-port case was $680.  For 24-ports, the only good one I found has an included redundant PSU (no complaints!) but it was around $2,400!  Three times the price!  Oh, and 24 drives at $110 each, $2,700.

I knew I settled for 16-ports for a reason.

But at that point, I could, ideally, use RAID 6+ (my Promise card has it) so it would have a third parity disk plus probably two hot spares.  This would mean a 2.5 TB addition in space with enhanced reliability and auto-correction features (spares).

Without the core computer components (mobo, cpu, memory, OS HDDs), I'm already $2K higher.  I guess I just went for the sweet spot on price/capacity/redundancy.  The case I got really was a good deal compared to many other ones.  16 hot swappable bays can be much more expensive than $680.
Logged
Pages: [1]   Go Up