Load testing your cloud app - Tools comparison

If you’re planning to load test your cloud app, the comparison below might be handy.

The research I have done is on these service providers:

1.          Soasta CloudTest,
2.          Blitz,
3.          BlazeMeter,
4.          LoadStorm,
5.          and MicrosoftCloud-based Load test tool.

Soasta – CloudTest

Pros

•          A complete environment to setup tests
•          Add test users locations seamlessly
•          a good coverage around the globe
•          A comprehensive real-time interface to metrics
•          Drill down analysis on web requests
•          A Lite (Free) version is available with a limit of 100 concurrent users

Cons

•          Requires some setup: you have to spin up a VM to load the test environment

Pricing

“CloudTest is sold based on server hours, including support. With no limits on the number of testers or software access, you select

Plans based on how many test server hours you’ll run. Plans are available for coverage of a few tests each month up to thousands of tests each year” - http://www.soasta.com/wp/wp-content/uploads/2012/10/DS-CloudTest-100812.pdf

Blitz

Pros

•          Capacity planning tools, Optimization, Performance monitoring are out-of-the-box
•          Nothing to install/setup, all cloud-based
•          Simulate up to 5,000 concurrent virtual users
•          It has a chrome/firefox plugin to monitor app performance on develop environment
•          Integrated with Google Analytics
•          Performance Monitoring Alerts  (Email, Twitter, SMS, PagerDuty)

Cons

•          Only 8 test locations around the globe (Australia, Brazil, California Ireland, Japan, Oregon, Singapore, Virginia)
•          Relatively less sophisticated interface (compared to Soasta)

Pricing

•          Works with “Credits”
•          1 credit = 1 minute and 1,000 users - https://www.blitz.io/pricing
•          40 Credits = $40
•          150 Credits = $135
•          300 Credits = $240

BlazeMeter

Pros

•          JMeter compliant
•          Easy to use test creation interface
•          Nice interface for test results
•          You can export test results
•          Compare two test runs
•          Has a server in Australia (yes, we’re load-testing apps in Oz)
•          Test scheduling
•          Integrated with Google Analytics
•          VPN is supported
•          Chrome extension to record JMeter scripts and upload it to BlazeMeter
•          It has a free version that can simulate up to 50 concurrent users
•          A good customer base

Cons

•          Limited Geo coverage (California, Oregon and Virginia USA, Tokyo, Ireland, Brazil, Singapore and Australia)

Pricing

•          Monthly plans – Starting $199 (1,000 users, 20 hours) up to “Call Us”
•          On-Demand (Per hour) – Starting $19 (1,000 users, 1 server) to $299 (40,000 users, 40 servers)
•          http://blazemeter.com/pricing

LoadStorm

Pros

•          Scalable up to 300,000 users
•          A testing data centre in Sydney, Australia
•          Supports automatic crawling (Spider)
•          No scripting language is involved
•          Nothing to install

Cons

•          Currently they only cover Virginia, California, Oregon, Ireland, Sydney, Tokyo, Singapore, Sao Paulo
•          No built-in feature to collect server performance metrics

Pricing

•          Monthly plans – Starting Free (100 users, 25 users per test) up to “Call Us”
•          On-Demand (Per hour) – Starting $39 (1,000 users per test) to “Call” (10,000+ users)
•          http://loadstorm.com/load-testing-cost/

Microsoft Cloud-based Load test tool

Pros

•          MS offers a free version of TFS services (up to 5 developers)
•          Reuse on-premise tests
•          Easy to set-up

Cons

•          Requires Visual Studio™ Ultimate (Pricy)

Pricing

•          It’s still vague - varies according to your MSDN subscription
•          If you don’t have an MSDN subscription, you’ll have to buy VS Ultimate (around AU$19,274) and then subscribe for TFS Services

Securing Web API

Download the mind map from here

SydneyAzure User group - BizTalk Services Session

Last night I attended SydneyAzure UG. The talk was about BizTalk Services, presented by Mick Badran.

here's a summary...

What are BizTalk Services?

      ·         Windows Azure Cloud-based integration service

o   B2B integration (Partners, agreements, ..)

o   rich message endpoints

     ·         BizTalk Adapter service pack

o   on-premise adapters designed to rapidly connect systems

o   It has management service (REST APIs)

BizTalk Services SDK

   §  VS2012 - Developer Surface

o   Project Type

o   Bridges: It's like a port (in BizTalk land) where it can connect to source and destination and in between a staged pipeline that can decode, enrich, transform, call external assemblies, .. Just like a normal BizTalk Pipline.

o   Connector: an arrow that connects a bridge to source or destination, it can be configured for message inspectors

o   Schemas: You cannot distinguish or promote properties as in BizTalk Server, since there’s no property schemas

o   Maps

§  New map designer

§  they added a new GetBizTalkContextProperty functoid,

§  map-level settings on how to handle mapping errors

·         Fail map

·         output null

o   all artefacts are compiled into a Workflow activity

o   No support for orchestrations yet. Maybe the next release there will be support for Workflow (Windows Workflows)

  §  You don’t have to have BizTalk Server installed on the machine, they’re totally independent.

  §  It’s still V1, so don’t expect unit testing frameworks are there to support your development.

  §  BizTalk Services Adapter Service + IIS Plugin

  §  Developer + Runtime components are supplied

   The conclusion I reached after attending the session is that the product is not yet mature enough to market it to clients, but it’s good as a start for moving BizTalk components to the cloud.

Why keep the balance?

I read many posts/articles about keeping the balance between personal and professional life. For starters, this one starts by giving some tips about to keep the balance.

However, I believe we're only living one life (Personal + Professional), at the end of the day, you're one human being that has one brain, one heart, etc.. So why do we bother keeping this virtual balance where there's no line, even fine line, between the so-called two lives?

The moment you start this separation, the same moment you stop the ideas flow throughout your day. You have only one life, don't restrict your mind to this separation of concerns where it doesn't exist!

M

Luxor - Reports Deployment Tool for Dynamics CRM 4.0

I posted a new tool on codeplex:

This tool helps developers copy reports (SSRS) files and its configuration from development to production environment. It's compatible with Dynamics CRM 4.0.

This release includes the following features:
Batch-Downloading Reports
Batch-Uploading Reports

The package includes source code and executable file.

http://luxor.codeplex.com/

Sakkara - CRM Customization Helper

Check out my new tool on Codeplex:
http://sakkara.codeplex.com/

WCF - Store and Forward

When designing WCF services, specifically, OneWay operation contracts, developers usually expect the service to be stable. One doesn’t consider network instability between the service and its clients.

That’s why, many messaging engine, such as BizTalk and Tibco EMS has a built-in feature called Store and Forward.  That is, the client doesn’t consume the service directly but rather publish the messages into a store, and then the messaging engine peeks on the message store and route it to pre-designated subscribers. This architecture pattern is called Pub/Sub. You can look it up here.

In WCF world, developers have the facility to build this store and forward capability using MSMQ binding (netMsmqBinding). However, MSMQ has a lot of networking issues, especially if it’s used as a network resource in a multi-forest or multi-domain environment.

I have worked on a small-scale solution to accomplish this task using a database message store. It’s built on the following components:

 

1.  Router: the router is called by the client. Currently it’s a stateless router based on WCF 4.0 routing service (http://msdn.microsoft.com/en-us/library/ee354381.aspx - Routing Service).
2. Persistence Service: a WCF service with a single Operation Contract (Publish). It’s used to publish WCF messages into the message store.
3. Message Store: a database to store WCF messages.
4. Message Processor: Peeks the message store for new messages based on subscriptions and route them to subscribers.
5. Subscription Manager: a WCF service used to subscribe or unsubscribe. It has two Operation Contracts. Subscribe and Unsubscribe.

Known Issues:

Ø  It works only with basicHttpBinding.

Ø  Subscription manager is not testedJ

Ø  It uses the SOAP action to route messages. That means that you’ll have to subscribe for each SOAP action. I’m planning to use Regex to match subscriptions instead.

Ø  It’s not tested with large messages (> 4mb). I believe it will need some work to adjust the readerQuotas on the channel binding.

The solution code and database script can be found here.