Building a storage server...

[Squeetard]

...or a software one

No. no no no. Software de-duplication systems are crap. Very unreliable. Lots of corruption issues. They spend more time repairing themselves than they do backing up your data.

 

[Cerb]

They? No, it. BTRFS supports deferred dedupe, but not real-time dedupe, so the number of available reliable file systems to use is just one.

I have read of very few corruption problems not related to the underlying hardware, and it's been working for years out in the wild. In fact, it tends to be the opposite: ZFS tends to help find storage and network problems not otherwise detected.

But, depending on time needed (such as getting into *n*x and Samba, if not already familiar with them), more HDDs might be a cheaper option, by far.

 

[imagoon]

Server 2012 dedup also works great.

 

[Cerb]

Server 2012 dedup also works great.

But crap, that's expensive! You're right, though, they did add that with CSVFS, didn't they?

 

[imagoon]

But crap, that's expensive! You're right, though, they did add that with CSVFS, didn't they?

Not nearly as expensive as the time I waste on linux / samba boxes. It isn't even a GUI / CLI thing as I use powershell exclusively to build most of it.

It works on native NTFS, no need for CSVFS or clustering services to be installed at all.

 

[Cerb]

I'm consistently in the opposite boat, with most everything working pretty well in *n*x-land, but feeling like I'm running up a muddy hill fussing w/ Windows .

 

[imagoon]

I'm consistently in the opposite boat, with most everything working pretty well in *n*x-land, but feeling like I'm running up a muddy hill fussing w/ Windows .

It works great until it doesn't. Then you are looking through forums posts for random error messages because writing actual documentation isn't "cool." I respect MS for technet. Normally everything is there even if google pops up random blogs for the first page.

I really try to give Linux a chance but I have never had it just work. Then I am roaming around on forums trying to find out how to fix issue "x" on distro "y" because no one can decide how they want to do simple things like system services. PS I don't want this to go as Linux bashing. I have no ill will for it, just that it seems like I waste much more time on getting it to do things.

 

[Cerb]

I think a lot of it is mindset. Neither are remotely unreliable, but they are very different, including how they're going to break. OTOH, I also spend more time in some variety of Linux than Windows, when on my own time, which I'm sure helps.

 

[OVerLoRDI]

I'd recommend raid 10 as well. Depending on the performance need you might not need to go for a full SAN. You could use something like a poweredge R720xd and get plenty of storage, redundant power, ethernet, etc, plus in a package that isn't a proprietary SAN, so you have a decent shot of servicing it yourself (or throw on dell's mission critical support package with whatever response time you need)

 

[Cerb]

There's no need for a SAN here at all, just a NAS. A SAN is a can of worms I prefer to stay far away from. I can't, as one is not far from my desk, but it's an overly-complicated single point of failure that I just hope doesn't fail before it gets taken down (not SAN tech, but a single SAN server on a non-redundant network). Since consolidated storage is not a necessity (performance of the storage is not a SAN v. NAS issue, so much as performance improvements from not having cross networks all over the place to different servers, and/or block images stuck on a regular file system v. real block access), a NAS should be the option. A big name for quick parts turnaround is a good idea, but a NAS is a much simpler way to do it, and will handle the work.

 

[imagoon]

It doesn't sound like he needs a SAN. Just a NAS. If you have no need for shared storage or block level replication then most of the time you don't need a SAN. There is nothing wrong with either tech and the term is starting to blend anyway. SAN generally means "block level" where NAS is file system level. What you need varies on what you want to do. For example 2012 NTFS dedup won't work unless it is block level access. (there is more to this really, it is more than most systems not running Windows NTFS emulate NTFS poorly but that is another subject.) NAS might do file level deduplication however.

Since system images rarely have complex permissions a NAS normally works well.

 

[EricFiskCGD]

Good day guys,

i was tasked to build a storage server for the large amount of disk images my department does and being a bit pressured to find the best buck per terabyte i come to the AT forum to ask assistance.

would anyone be generous as to lay out a spec compilation for a system that would be high on storage yet light on the pocket ? taking all suggestions

My advice is to keep your boss in the loop with regular updates in your search to cover your butt.

This reminds me of everything I despise about members of upper management who are experts on topics they know nothing about. The best analogy I can think of is the restaurant owner I worked for in college back in the early 1990’s who thought that because he had the only restaurant in town that was “successful” he was an expert on everything. He would ask for advice, ignore the advice he was given, then when he would run into problems he would ask…

“Hey, Do you know anything about computers?”

This boss of yours wants something that’s powerful, versatile and reliable system that’s also cheap?

Whenever I need something for my small business, I research the funk out of it. I can spend a couple of hours each day obsessing over what to purchase and where to buy it. Why isn’t your boss researching that with you?

To me it sounds like you’re being set up, because if this thing fails it’s going to be your head on the chopping block.

 

[smitbret]

My advice is to keep your boss in the loop with regular updates in your search to cover your butt.

This reminds me of everything I despise about members of upper management who are experts on topics they know nothing about. The best analogy I can think of is the restaurant owner I worked for in college back in the early 1990’s who thought that because he had the only restaurant in town that was “successful” he was an expert on everything. He would ask for advice, ignore the advice he was given, then when he would run into problems he would ask…

“Hey, Do you know anything about computers?”

This boss of yours wants something that’s powerful, versatile and reliable system that’s also cheap?

Whenever I need something for my small business, I research the funk out of it. I can spend a couple of hours each day obsessing over what to purchase and where to buy it. Why isn’t your boss researching that with you?

To me it sounds like you’re being set up, because if this thing fails it’s going to be your head on the chopping block.

As I follow this thread, this keeps running through my head. If OP is having to get these answers here, then he probable doesn't have the tech/networking know-how to properly implement this. When the system is slow or fails his head is gonna be on the chopping block.

OP, telling your boss "Thanks, but no thanks" right now is far less risky than taking this project.

 

[Cerb]

Or, if that's not the case, a simple setup like a 4 or 5 bay Synology box. Getting back on to the OP's issues, after the URE derailment, a Synology or Drobo box isn't hard to set up and use, and is cheaper than a Windows server. But, if Thor--or a random factory defect, for that matter--strikes, you're on your own for a week, and might end up with a broken array, to boot.

You get what you pay for. But, you need to know what needs paying for, both to not buy more than you need, and also to not buy less, and/or to be able to tell management that they do not value their data highly enough.

 

[EricFiskCGD]

As I follow this thread, this keeps running through my head. If OP is having to get these answers here, then he probable doesn't have the tech/networking know-how to properly implement this. When the system is slow or fails his head is gonna be on the chopping block.

OP, telling your boss "Thanks, but no thanks" right now is far less risky than taking this project.

It also sounds to me that this boss is LAZY and is expecting someone else to do his job for him!

I forgot to add that while his boss is looking for something that’s powerful, versatile and reliable system that’s also cheap – there is no such animal. From my own personal experience (and please… bring my life meaning by learning from my mistakes!) you’re either going to spend the money to get something high-end and have it perform or you’re going to buy something cheap and remain frustrated.

Or worse, you’re going to keep replacing the same thing over and over again because your tools keep breaking until you come to your senses and buy the better made and more expensive product. In short, by buying the same crap over and over again before buying the unit you should have bought first, you’ll be spending at least twice as much. It’s called being “Penny wise but dollar foolish.”