Starwind 4 0 Serial Podcast
Hi there Spiceheads,I was wondering if anyone here has any experience setting up VMware's vSAN on a 45drives chassis? I know they have their list of 'Ready Nodes' and preferred vendors, but we were already looking at the 45drives and just needed to figure out which software we were going with. FreeNAS is nice but when we saw vSAN we figured we might be able to make a much better device out of it.I assume it wouldn't be supported by VMware in this configuration, so are there any other pitfalls we need to watch for? Any known (or potential) performance issues? Wingzfan99 wrote:Hi there Spiceheads,I was wondering if anyone here has any experience setting up VMware's vSAN on a 45drives chassis?I've got quite a few VSAN deployments under my belt (going back even to the private beta).
I am not aware of anyone using the 45 drive chassis. Generally we see Dell (R7xx or FX) SuperMicro (SuperServers, or Twin/FatTwin) and Cisco (C240). I've had conversations about using a UCS C3160 but there are some challenges with this (beyond having a failure domain that large means re-sync's can get pretty brutal if you do not have a very large cluster of these, and potentially 40Gbps ethernet to handle re-sync).wingzfan99 wrote:but we were already looking at the 45drives and just needed to figure out which software we were going with.This is a really really odd design for an IT/Business solution.Normally the process is1. Establish Business need.2. Establish software that will handle need.3. Establish hardware that will support software handling business need.4.
Profit.What your describing is1. Establish hardware.2. Find software the will make hardware solve the need.3. Check if the above has met the need.4.???wingzfan99 wrote: FreeNAS is nice but when we saw vSAN we figured we might be able to make a much better device out of it.Wildly different platforms here. One is a GUI on a BSD Unix system that has an 'alright' NAS product (I'd still rather use Windows for features like FSRM, or a system that supports clustered NAS heads like a VNXe/BluArc in a lot of use cases for SLA/management reasons).Can you state what business challenge/problem it is you are trying to solve? Maybe then we can help advise you on finding a solution.
@ 4:0 Facebook
Trying to discuss components in a vacuum is kind of hard (Its like asking a crowd if you should buy Ford, or Ferrari without stating what your commute is, what your budget is etc).wingzfan99 wrote:I assume it wouldn't be supported by VMware in this configurationVMware WILL still support non-ready node configurations that use HCL components (Scroll down on the ready node page, the link is at the bottom). They are directing people to ready nodes so they don't buy things that are not compatible that will have problems (like what you are talking about doing). I'd strongly advise anyone wanting to go into more custom waters to engage with a VMware partner who knows VSAN in and out. I've done this (Working for a VSAN elite partner) for the past year and had great success.wingzfan99 wrote:so are there any other pitfalls we need to watch for?The RocketRAID is an AHCI SATA controller. I suspect that it has a standard hilariously anemic low queue depth (something like 31 across all ports).
As such for non-streaming workloads (VSAN) and high random workloads (Flash drives) I would expect it to perform poorly. Attempts at using AHCI (Which has a very serial structure/nature) for not just VSAN but flash drives at scale I would not expect a positive outcome.Also I don't think HighPoint RocketRaid's have controller drivers in ESXi so I'm not even sure if this would work.wingzfan99 wrote:Am I crazy?Not crazy. When I first started I did crazier/sillier things (build my own DRDB NFS, Hyperconvered system back in 2008). As you spend time in the industry you realize that what sounds like a fun weekend lab project isn't always the best to run production on (and you learn how to speak to accounting/finance people and explain what you need to spend on budget to make things work)..
If you're going to be putting a virtual SAN into production I highly recommend not using anything not on the HCL Virtual SAN gives you a lot of flexibility and you definitely don't have to just look at the ready nodes. I liked the idea of it VSAN for this very reason. But I wouldn't use unsupported hardware or you're likely to run into problems.I'm not running 45 disk hosts but I am running hosts with 18 disks a piece.
And there is room to add up to 8 additional disks if more capacity is needed on each host.. Wingzfan99 wrote:While I appreciate your experience and in-depth reply, that was rather condescending.Sorry if I come off as an ass, just in my day job I've been the guy people call in after they spend 20-200K on hardware that isn't going to do what they need, or get asked to 'make it work' and have to explain how the costs are sunk, and your going to end up having to spend a lot of money re-buying things to get out of it. I've seen a lot of people go down this path, and it ends up with the same disappointment in the data center when you can't easily refactor the project.
I've also seen some home brew storage disasters get people fired (again, very early in my career I had a DRDB system split brain on me, and because we didn't have budget or experience to properly fence it. Luckily we were able to recover the data but I still want those 3 days of my life back).wingzfan99 wrote:Simply because we're not following the perfect project planning method does not indicate inexperience.
Different organizations have different methods and budgetary constraintsThis is why I ask what the business is trying to accomplish. There are times where non-supported risky hacks are worth it (where the cost of failure is low, and the price for success is very high). This is not a common situation to be in.Also if all your are needing is a cold/deep archive target, you may actually have better economics going with SAS/Shelfs modular design (reduces power). The 45 drive coldish/scale out controller anemic low MTBF servers are really only BEST suited for very niche requirements (systems that are application redundant not primary storage, and have streaming write inbound workloads and very low change of read/mixed with a fill/slow read back/clean up type IO pattern). While they can be forced into duty for other needs they are likely not the best design/choice.I'm not being pedantic here I promise.
Just trying to understand what it is you need to accomplish so maybe we can steer you to a more appropriate solution. If budget is a concern SuperMicro is very cost effective.Another issue with the 45drive type solutions is that they are generally used with ultra-capacity 4KN drives. These drives will not perform consistently with VMware (not just VSAN, but VMware). The snapshot and other IO patterns are not optimized for it (at a guest HBA, as well as a kernel basis).
Starwind 4 0 Serial Podcast Online
512E drives are not supported either (this is why you don't see 8TB drive on the HCL currently). If you are also looking for any cost-efficient vSAN alternatives, I may advise you to try StarWind Virtual SAN Free and configure 2 servers as “shared nothing” fault-tolerant NAS by “mirroring” their internal storage between them. Resulting solution exposes continuously available SMB 3.02 shares and failover NFS v4.1 mount points and targets in such a use cases:.Shared storage for Microsoft Hyper-V VMs and SQL Server DBs (CA SMB3).Shared storage for VMware vSphere & ESXi, Citrix XenServer and various Xen VMs (NFS).Failover file server (common data, VDI profiles, backups and so on) (SMB3 & NFS)By doing this way you can consolidate all available storage in 2 instances which will definitely let you to easily manage your infrastructure as well as achieving HA shared storage for your client servers. Such configuration requires at least 3 servers: 2 StarWind Nodes and 1 (or more) Compute server(s).I may also advise you to look at xbyte’s hardware in case if you do not have any servers in your inventory yet.
They sell refurbished Dell servers and have really good pricing.Your configuration would look like this.