Load Testing with VSTS FAQ

As I mentioned in my bio, in addition to .Net consulting, I do some web and load test consulting. Here is a handful of questions that come fairly often.

Q) Connections per user or Connection Pool – Which is recommended?

A) Connection per user is ideal. It requires the most resources on the agent. If agent CPU < 75% and memory usage < 90% you should be ok. Note: the number of connections may be 2X the number of users as dependent requests may get another connection and run in parallel.
B) Connection pool means that one Web test may have to wait before issuing a request when another Web test is using the connection. The average time that a Web test waits before submitting a request is tracked by the load test performance counter Avg. Connection Wait Time. This number should be less than the average response time for a page. If it is not, the connection pool size is probably too small.
More Info: http://msdn.microsoft.com/en-us/library/ms404664(VS.80).aspx#ConnectionModel

Q) What is the difference between having multiple scenarios in a load test and different tests in a single scenario.

A) You can and should use multiple scenarios. A scenario typically mimics different “groups” of users. For example a single website might have a couple admin type users that perform 1 set of tasks and regular users that perform another set of tasks. To simulate these users you would create a scenario for each group. The scenarios, of course, can contain multiple tests and if you choose some of these tests may be in both scenarios.


Q: How is load test execution affected by data source access methods?

A: When you add data source to a Web test, you can choose from three different access options.

• Sequential - If you set a data source to sequential, it uses the records from the data source in the order it reads them in. When it reaches to end of the records, it will loop back to the beginning and start again. It will continue to do this for the duration of the load test.
• Random - If you select random, then a random row will be used for each test iteration. This will continue to happen for the duration of the load test.
• Unique - If you select unique, the load test will access the data sequentially, but it will only execute 1 test iteration for each row in the data source regardless of what the test duration is set to. So if you have 20 rows, then the load test will execute 20 tests and then stop.

For more information, see About Data Binding in Web Tests.