If you’re planning to load test your cloud app, the
comparison below might be handy.
The research I have done is on these service providers:
Soasta – CloudTest
Pros
- A complete environment to setup tests
- Add test users locations seamlessly
- a good coverage around the globe
- A comprehensive real-time interface to metrics
- Drill down analysis on web requests
- A Lite (Free) version is available with a limit of 100 concurrent users
Cons
- Requires some setup: you have to spin up a VM to load the test environment
Pricing
“CloudTest is sold based on server hours, including
support. With no limits on the number of testers or software access, you select
Plans based on how many test server hours you’ll run.
Plans are available for coverage of a few tests each month up to thousands of
tests each year” - http://www.soasta.com/wp/wp-content/uploads/2012/10/DS-CloudTest-100812.pdf
Blitz
Pros
- Capacity planning tools, Optimization, Performance monitoring are out-of-the-box
- Nothing to install/setup, all cloud-based
- Simulate up to 5,000 concurrent virtual users
- It has a chrome/firefox plugin to monitor app performance on develop environment
- Integrated with Google Analytics
- Performance Monitoring Alerts (Email, Twitter, SMS, PagerDuty)
Cons
- Only 8 test locations around the globe (Australia, Brazil, California Ireland, Japan, Oregon, Singapore, Virginia)
- Relatively less sophisticated interface (compared to Soasta)
Pricing
- Works with “Credits”
- 1 credit = 1 minute and 1,000 users - https://www.blitz.io/pricing
- 40 Credits = $40
- 150 Credits = $135
- 300 Credits = $240
BlazeMeter
Pros
- JMeter compliant
- Easy to use test creation interface
- Nice interface for test results
- You can export test results
- Compare two test runs
- Has a server in Australia (yes, we’re load-testing apps in Oz)
- Test scheduling
- Integrated with Google Analytics
- VPN is supported
- Chrome extension to record JMeter scripts and upload it to BlazeMeter
- It has a free version that can simulate up to 50 concurrent users
- A good customer base
Cons
- Limited Geo coverage (California, Oregon and Virginia USA, Tokyo, Ireland, Brazil, Singapore and Australia)
Pricing
- Monthly plans – Starting $199 (1,000 users, 20 hours) up to “Call Us”
- On-Demand (Per hour) – Starting $19 (1,000 users, 1 server) to $299 (40,000 users, 40 servers)
- http://blazemeter.com/pricing
LoadStorm
Pros
- Scalable up to 300,000 users
- A testing data centre in Sydney, Australia
- Supports automatic crawling (Spider)
- No scripting language is involved
- Nothing to install
Cons
- Currently they only cover Virginia, California, Oregon, Ireland, Sydney, Tokyo, Singapore, Sao Paulo
- No built-in feature to collect server performance metrics
Pricing
- Monthly plans – Starting Free (100 users, 25 users per test) up to “Call Us”
- On-Demand (Per hour) – Starting $39 (1,000 users per test) to “Call” (10,000+ users)
- http://loadstorm.com/load-testing-cost/
Microsoft Cloud-based Load test tool
Pros
- MS offers a free version of TFS services (up to 5 developers)
- Reuse on-premise tests
- Easy to set-up
Cons
- Requires Visual Studio™ Ultimate (Pricy)
Pricing
- It’s still vague - varies according to your MSDN subscription
- If you don’t have an MSDN subscription, you’ll have to buy VS Ultimate (around AU$19,274) and then subscribe for TFS Services
