At work we use JetBrains TeamCity to handle the deployment of all of our websites, both continuous integration and production. We have unit testing setup and it is part of the build steps in TeamCity. Since a lot of our sites have APIs, we were lacking real world API testing. Sure, you can have your unit testing project fire up an in-memory server and test APIs, but wouldn’t it be nice to test from external sources and different regions around the world. Run real world tests and experience the same results as your users would.

I introduce to you Runscope.com. Runscope offers automated API testing and monitoring from locations around the world. You can write assertions for your tests to ensure your APIs are on their best behavior. I won’t go into the full details of all that Runscope offers but you can read all about it on their documentation pages. The purpose of this blog is to show you how to integrate Runscope tests into your continuous integration process.

I’ve created a simple ASP.NET web api app for this post, created a bucket in Runscope and added a few tests.

alt

alt

The first test will assert that the GET all superhero call returns results with a 200 OK status code.

alt

The next test will POST a new superhero, assert that a 201 Created status code is returned, and will capture the newly created superhero’s Id as a Runscope variable to be used in later tests.

The third test uses the SuperheroId variable that was captured in the POST test and asserts the new superhero is returned in the GET all api call. Runscope offers the ability to add JavaScript snippets to handle more advanced variable extraction and assertion scenarios. Very cool!

Finally, I’ll delete the superhero and assert a 200 status code is returned.

A nice little feature Runscope offers when adding variables to your tests is auto-completion! A small detail that impressed me the first time I saw it. This really comes in handy when dealing with a large amount of variables.

In a continuous integration environment, I want to run these tests anytime changes are checked into source control to ensure the APIs still function as expected. Runscope offers trigger URLs to trigger a test run via a GET or POST request. You can either trigger an individual test or all tests within a bucket. In TeamCity, the quickest way to setup a Runscope trigger is to add a Powershell runner build step (Windows) or a command line runner (Linux) and invoke a web request to the test trigger.

Powershell (Windows)

Individual test trigger

Invoke-RestMethod https://api.runscope.com/radar/:trigger_id/trigger

Bucket trigger

Invoke-RestMethod https://api.runscope.com/radar/bucket/:trigger_id/trigger

Command Line (Linux)

Individual test trigger

curl https://api.runscope.com/radar/:trigger_id/trigger

Bucket trigger

curl https://api.runscope.com/radar/bucket/:trigger_id/trigger

In a normal TeamCity CI build configuration, I would set up version control settings to check my GitHub branch for changes. If changes are detected, the process would go as followed:

  • Build project source
  • Run unit tests
  • Deploy site to CI web server
  • Trigger Runscope tests

For the purpose of this demo, I only set up the Runscope trigger build step. Since my TeamCity server is Windows, I went with the Powershell runner.

Start the TeamCity build and looking at the log we can see that the Runscope trigger was called successful.

Heading over to Runscope, I can see that test was triggered via the Trigger URL

I set up these Runscope tests to email me when they are finished as well as post a message in a Slack channel to inform the entire team of the test results.

Integrating Runscope into our CI environment has been fun and very useful. I would encourage everyone to check them out. For your efforts they will hook you up with a free t-shirt!