Fun with Tintri Part 2

In my last post I talked a bit about the hardware and getting set up with our Tintri VMstore T540 system which went very smoothly. I had thought about writing my next entry about going through the initial system setup and configuration but to be honest it's so easy I didn't think it was necessary. Part of me wanted to start throwing some workloads on this thing and see how they performed but I needed to collect at least some performance data before I started moving my existing stuff around. I've never really done any serious I/O testing before and I wasn't entirely sure where to begin, but eventually I stumbled upon a fling provided by VMware called the I/O Analyzer and this thing is a fantastic tool. You download and deploy a small OVA template that creates a lightweight Linux VM. The cool thing is that you can deploy multiple copies of the appliance and control them all from a single instance. There's a great video by Chris Wahl that shows you how to quickly get up and running with this appliance and I highly recommend anyone looking to learn how to use it check it out.

So after watching the video that Chris created I dived right in and got started running a number of different tests against my current storage system and the Tintri T540. There's a few notes/caveats for my setup here that I did want to share:

• The Tintri system did not have any active workloads other than the test workers on it at the time of testing. My current storage was running an active production workload but it has enough additional capacity to allow for the testing to run concurrently.
• The current storage system I'm using does have some flash onboard but it is only servicing read operations whereas the Tintri's flash is serving all of the I/O operations.
• Both my current storage and the Tintri are serving storage to my VMware environment via NFS.
• Testing Equipment consisted of 3 Cisco UCS B200 M2 blades which have Dual 6-core Intel Xeon processors and 96 GB of RAM. All of my networking is running at 10 Gig.

My goals with this testing were twofold: One, determine what certain workloads would deliver in terms of throughput on each system. Two, see just how far I could push the Tintri to get an idea of just how much performance it can deliver. Without any further delay, let's take a look at some of the results:

VMware I/O Analyzer Tests

Test 1 - Single Worker Process OLTP 4k - 5 Minute Duration

Storage	Workload	Total IOPS	Read IOPS	Write IOPS	MBPS	Read MBPS	Write MBPS
Current Storage	4k 70% Read 100% Random	8355.75	5850.52	2505.23	32.64	22.85	9.79
Tintri	4k 70% Read 100% Random	20947.37	14664.28	6283.08	81.83	57.28	24.54

So this first test was supposed to mimic the I/O patterns of a typical OLTP database workload. You can see that while both systems handled it relatively well, the Tintri pushed more than double the IOPS of my current array in the same 5 minute duration. My company does run a decent sized OLTP workload so this test was of significant interest to me to get an idea of how one of my current systems might operate on a Tintri system vs. what I can deliver today.

Test 2 - Single Worker Process Exchange 2k7 - 5 Minute Duration

Storage	Workload	Total IOPS	Read IOPS	Write IOPS	MBPS	Read MBPS	Write MBPS
Current Storage	8k 55% Read 80% Random	3338.28	1833.16	1505.12	26.08	14.32	11.76
Tintri	8k 55% Read 80% Random	12837.28	7057.01	5780.27	100.29	55.13	45.16

In this test I used an Exchange I/O pattern simulation which was of interest as I run Exchange in the environment as well, but we really don't have any major performance issues with it. Even so, it was good to see how this workload would be handled on each system. You can see that the Tintri didn't push as many IOPS as it did with the OLTP simulation but it makes sense given that the I/O was more evenly split between read/write operations and the block size went up to 8k. Regardless of that it still did about 4 times the IOPS as my current storage so that was great to see.

Test 3 - Multiple Workers w/ Varying Workloads - 5 Minute Duration

Storage	Workload	Total IOPS	Read IOPS	Write IOPS	MBPS	Read MBPS	Write MBPS
Current Worker 1	4k 70% Read 100% Random	1509.71	1057.44	452.27	5.9	4.13	1.77
Current Worker 2	8k 55% Read 80% Random	5005.66	2749.87	2255.79	39.11	21.48	17.62
Current Worker 3	8k 95% Read 75% Random	3225.18	3062.74	162.44	25.2	23.93	1.27
Tintri Worker 1	4k 70% Read 100% Random	13036.61	9132.6	3904	50.92	35.67	15.25
Tintri Worker 2	8k 55% Read 80% Random	12288.17	6759.12	5529.05	96	52.81	43.2
Tintri Worker 3	8k 95% Read 75% Random	5944.5	5645.89	298.6	46.44	44.11	2.33

Current Storage Total IOPS: 9740.55
Tintri Total IOPS: 31269.28

For the next test I decided to bring in some more worker VMs to run some concurrent testing. I reused the OLTP/Exchange workload simulations from Tests 1 and 2, and I also added another workload which I believe was the web server option in the I/O Analyzer. Once again this test showed me that both storage systems handled the workloads fairly well but the Tintri simply has a lot more horsepower than my current system. It really shows you the power of flash storage and the impact that it can have on your major applications. My current storage system is a full blown traditional SAN taking up a full 42U server rack and it's probably got about 150ish SAS disks and this 3U Tintri system with 16 disks is blowing right by it on every test.

At this point I had a pretty good idea of what the Tintri was capable of and I certainly showed that it has a lot more power than the storage I'm using today. My next objective was to try a few tests to see just how much throughput I could push through the Tintri system because why the heck not?

Test 4 - Tintri Stress Test w/ 5 workers - 5 Minute Duration

Storage	Workload	Total IOPS	Read IOPS	Write IOPS	MBPS	Read MBPS	Write MBPS
Tintri Worker 1	4k 70% Read 100% Random	8389.25	5872.5	2516.75	32.77	22.94	9.83
Tintri Worker 2	8k 55% Read 80% Random	9139.71	5024.52	4115.19	71.4	39.25	32.15
Tintri Worker 3	8k 95% Read 75% Random	3719.73	3533.8	185.93	29.06	27.61	1.45
Tintri Worker 4	16k 66% Read 100% Random	8548.35	5636.2	2912.15	133.57	88.07	45.5
Tintri Worker 5	8k 20% Read 0% Random	3923.99	784.34	3139.66	30.66	6.13	24.53

`

Total Tintri IOPS: 33721.03

So the IOPS count on this test was a little lower than I expected but still very very good. I re-used a few of the same workload patterns and threw in a few other types from the I/O Analyzer to run 5 concurrent workloads against just the Tintri. Since I only had three hosts to work with I figured that some of the workloads were likely competing with one another so that may account for why I didn't see a significantly larger total number than I did in Test 3.

After posting my first blog entry on twitter Justin Lauer recommended that I try an appliance called Tingle that is provided by Tintri and available for customers on their support site. Much like the VMware I/O Analyzer, this is an OVA template that you deploy into your environment and it can run I/O simulations against your storage. I already had enough good data from the I/O Analyzer but I figured it might be fun to give Tingle a shot too and see how hard it would push the Tintri. I deployed 5 of the appliances and ran their "IOPS" test which executes 20 threads doing random reads and 1 thread doing random writes for a read/write ratio of 71:29. Tingle does not give you the detailed reports that the I/O Analyzer does (you are able to watch the appliance console screen and read the output to get an idea of how each appliance is doing), but my I/O stabilized around 55-60k IOPS and for a very brief period I had the system pushing 75k concurrent IOPS:

I'm honestly not sure how accurate of a test this was and I'm tending to lean toward the numbers that I got from the I/O Analyzer as a bit more accurate as to what I would see in the real world, but mainly what I learned from all of this is that the Tintri's got some serious power boxed up in a relatively small appliance. I think that it's going to be hard for businesses to ignore the performance and ease of use that you can get out of something like a Tintri when comparing against more traditional storage options. The fact that I was able to get this system up and running in very little time and get this kind of performance is extremely appealing to me, and I think it will be to many others as well.

Now that I've got some good data for how my Tintri T540 is going to perform we're moving closer to putting some actual workloads on it! I'm also interested in finding out what kinds of alerting and information that I can get from the Tintri and how their support system works because that's something that can't be ignored when you're looking at new storage. Say what you will about how the support rates from company X or Y, but you know that the big players usually have pretty big support organizations behind their products and the phone gets answered when you really need it to be. I will dig deeper into this and more as my adventure with Tintri continues!