Running UI tests in Internet Explorer with Jenkins on AWS

In the best conditions it can feel difficult to use Selenium WebDriver to run UI tests on Internet Explorer. Running on the same machine you develop on isn’t realistic as WebDriver for IE requires all of the window focus. So you move it into a virtual machine on your box and then find that your machine can’t give up enough resources to reliably run the UI test suite you’ve built out. Even opening a new application on the host machine can cause the tests running in the virtual machine to fail. All that being said you want to have testing done somewhere else, that you don’t have to see, where it won’t be interrupted and have it all run every time you commit to source control. Our team is starting with Jenkins and automatic provisioning of Elastic Compute Cloud (EC2) servers in Amazon Web Services (AWS).

Asking for help running UI tests on Internet Explorer in this setup seemed to result in short replies of “no”. I won’t tell you about all the wrong paths I took getting to this solution. This tutorial of sorts makes several assumptions about where you are in the setup and what you already know how to do. For example, I had no Jenkins administration experience before doing this, I didn’t sign up for AWS, and I never setup any security credentials before starting this task.

  • I will not go into any specifics of running UI testing against IE because you’ve already got tests passing when run locally.
  • You already have a Jenkins install running and are comfortable enough to break things.
  • This Jenkins install has the EC2 plugin and has been connected to AWS
  • You’ve been given access to an AWS account where you can spawn instances with wild abandon.
  • You’ve got a PEM file for said AWS account to decrypt your instance’s administrator password.
  • You aren’t afraid of a little command line work.
  • You’re ready to do some Windows configuration.
  • And finally, you’re prepared to have patience.

This process involves:

  1. Create a new EC2 instance
  2. Configuring that EC2 instance to automatically login as an administrator
  3. Setting up the EC2 instance to register itself on your Jenkins build server
  4. Creating an AMI of this EC2 instance
  5. Adding or using an existing Jenkins user to use on the slave instances
  6. Configuring the Jenkins EC2 plugin to connect to the AMI we created
  7. Adding our own User Data information to the Jenkins EC2 configuration
  8. Start a build, launching an EC2 instance and starting the local process
  9. See success as this instance launches Internet Explorer from your UI test suite and watch as your tests pass because Selenium WebDriver can interact with IE.

AWS Setup

After signing into AWS and navigating to the EC2 console screen you want to launch a new instance.2015-08-28 16_00_24-EC2 Management Console

This example test machine will be on Microsoft Windows Server 2012. I used an M3.Large instance but you might be able to get away with less machine.2015-08-28 11_50_39-EC2 Management Console

When you get to the step to configure the instance’s security group you’ll want to create a new group or select an existing group that has the RDP protocol open to a machine you’ll be connecting from. 2015-08-28 11_51_40-EC2 Management Console

Back on the EC2 instances console, when the machine state is green and running you’ll want to collect the administrator account password by right clicking on the instance and selecting Get Windows Password. Then after you’ve decrypted and copied the Windows admin password you need to right click on the menu again to connect to this instance. 2015-08-28 11_55_02-EC2 Management Console

Once you’ve gotten into Windows the first thing we’ll do is create another account and add it to the administrator security group. Open up the Control Panel, then User Accounts, then User Accounts again, then click on Manage User Accounts. Here you’ll click on Add a user account. 2015-08-28 12_03_32-ec2-54-161-109-120 - - Remote Desktop

Once you’ve created that account, navigate back to the Manage Accounts screen and select that new User. Then select change account type, and change it to Administrator. 2015-08-28 12_04_21-ec2-54-161-109-120 - - Remote Desktop

Now we need to set that user to automatically login and go to the desktop when Windows starts up. Use the shortcut Windows+R to open the Run dialog and enter the command control userpasswords2. 2015-08-28 12_06_07-ec2-54-161-109-120 - - Remote Desktop

In the User Accounts window that appears you’ll want to uncheck the first option requiring users to enter a password to use the computer. 2015-08-28 12_06_48-ec2-54-161-109-120 - - Remote Desktop

When you click OK you’ll want to enter the user name and password for that new administrator account you created. 2015-08-28 12_07_25-ec2-54-161-109-120 - - Remote Desktop

Test Environment Setup

At this point it’s time for you to setup this machine. Go through installing everything you need to build and test your project. Build and test your project, make sure IE UI tests run. You want every thing to be green here. Get it all prepared because we’re going to freeze this as an AMI to be used on demand by Jenkins. After you’ve done all that we need to download a couple of files that I wrote to configure this machine when it launches to connect to Jenkins and register itself as the slave it should be.

You’ll need to install Java, wget and Python 2.7 on the system. I used Chocolatey for my configuration. After that save this file to your users directory, ie: C:\Users\Test, and named it

Then download a batch file we’ll use to run this script and save it to the same directory, ie: C:\Users\Test, and named it slave.bat.

Run both files to make sure they work. They won’t really work right now we have other configuration to do but it will verify they are linked and you have Python installed.

What this Python script does is look at the EC2 User Data values that will be set by Jenkins later. It then connects to Jenkins as a slave, run by this administrator user and in an active session. This is the one weird trick in getting Internet Explorer UI testing to work. It demands an active and interactive session, which is why it was such a pain to run anywhere.

AWS Setup Continued

To finish setting up the instance we need to add a new task to run our startup slave script every time the system starts. If you’re unsure where the Task Scheduler is you can use the Run dialog again and enter the command %SystemRoot%\system32\taskschd.msc /s. When you click Create Task on the right of the Task Scheduler the first thing to do is make sure you’ve got Run with highest privileges checked. 2015-08-28 13_38_04-ec2-54-161-109-120 - - Remote Desktop

Select the Triggers tab and add a trigger to run this task at login.

Now select the Actions tab and create a new Start a program action pointed to our batch file. Be sure to use the format cmd /k “path/to/my/batch/file”2015-08-28 13_38_21-ec2-54-161-109-120 - - Remote Desktop

Go ahead and reboot the instance and then log back in with your custom admin account. You should be presented with a cmd prompt and a unsuccessful script launch. 2015-08-28 14_57_08-ec2-54-161-109-120 - - Remote Desktop

If you’ve gotten this far then go back to the EC2 console, right click on our running instance and select Image, then select Create Image2015-08-28 15_19_58-EC2 Management Console

Now navigate over to the AMIs page on the right navigation and you can see the new AMI pending. Save the AMI ID at this time we’ll need to use it later in Jenkins.

Jenkins Setup

Set aside AWS for a bit and lets take a look at Jenkins. We’re going to be collecting information to populate our instance’s user data property as well as configure a few other connections. If you don’t already have the EC2 Plugin installed do so at this time. After installing it take some time to look around and familiarize yourself with the setup. I’ve assumed you’ve already got this part connected to AWS. We need to collect a few pieces of information along the way to help get the slave system connected properly.

  1. The URL for our build server
  2. A user that has permission to start slave machines
  3. That user’s username
  4. That user’s API token
  5. A prefix by which our Jenkins’ slaves are named which is conventional

As I presumed earlier you already have the Jenkins build server that our instance will be able to access. Save that build server URL now as it completes our first requirement. If you don’t already have a user created or want to create a new user for launching these slaves do so at this time to take care of our second requirement. Now go to that users profile page and click the Show API Token button. 2015-08-28 13_40_31-User ‘Barrett Sonntag’ Configuration [Jenkins]

This User ID and API Token fulfill our third and fourth requirements so hang onto those values. 2015-08-28 13_40_50-User ‘Barrett Sonntag’ Configuration [Jenkins]

The fifth requirement can be found when configuring the EC2 plugin for Jenkins. Go to Manage Jenkins, then Configure System, then on the very bottom click Add a new cloud, then configure that cloud. After that click the Add button under this cloud. Here your Description field is what will set your slave name prefix. I suggest not using spaces, but if you go crazy just make sure to URL encode it and then save this value off to fulfill the fifth requirement. Set the AMI ID to the same ID we collected earlier for our EC2 AMI. Be sure to set the FS root, for my examples I used “c:\jenkins”. Set the remote user as the user you created earlier and the set the AMI Type to windows and enter the user’s password there. Set the boot delay to 180 as recommended. 2015-08-28 15_28_30-Configure System [Jenkins]

Now click Advanced under these options. It’s time to use all the variables you saved to build our User Data values.

  • JENKINS_URL is the build server URL
  • SLAVE_PREFIX is the AMI description
  • USER_NAME is the user’s username you saved
  • USER_TOKEN is the user’s token you saved

2015-08-28 15_30_17-Configure System [Jenkins]

Verify User Data Injection in AWS

At this point you should be able to spawn a new instance with Jenkins by starting a build and see that User Data. Go back to the EC2 console after starting a build and find the new instance that was provisioned. Right click on it and select Instance Settings, then select View/Change User Data.

2015-08-28 15_33_55-EC2 Management Console

The code there should have a USER_DATA value, but the value is base64 encoded so it looks nothing like what we entered before. 2015-08-28 15_35_41-EC2 Management Console

That is okay because the script we setup in the EC2 AMI is already configured to decode that. At this point your build should be working, to check it out you can connect into this instance and login using your custom user credentials we made at the beginning. You can see the command window open up from the scheduled task and start debugging as necessary or celebrate a successful build.

Launching the Search BETA

Screen Shot 2012-12-24 at 4.22.03 PM

My major project of 2012, Search BETA, launched earlier this month, the morning of December 14th, and it was a journey worth documenting. It began back in May when the company decided to start our own labs or internal incubator groups. One began right away in New York City and the other started a couple weeks later in Austin. I had the opportunity to join the Austin group with a team of developers and product people that I highly respected. I was grinning the whole time with excitement. With the direction that we were independent from the company but had access to the data we sequestered ourselves in a room for a few weeks. Over pots of coffee and pints of beer after work we brainstormed our some good ideas for our brides to find inspiration for their weddings. Our group started out as Judy Galani, Product Strategist, Forrest Andrews and Matt Oehlers, Senior Software Engineers, Erin Bender, Freelance Art Director, and myself as Interactive Media Developer. We “sold” the idea and working prototype to our “investors” by October and it was then time to go from prototype to production.

We had to take the design in house from this point on, but after a fond farewell and thank you to Erin we started to add in more management to the group. It was a greenfield project so our back-end developers Matt and Forrest were excited to push forward into new territories. Within a day or two we were able to deploy projects straight from Visual Studio to the AWS servers, I was blown away. Forrest focused on our MongoDB NoSQL data store and the API that I would later pull all the front-end content from, ASP.NET MVC. Matt worked on massaging the data into Apache Solr and in a matter of days we had an impressive result set coming back for queries.

Screen Shot 2012-12-24 at 4.22.40 PM copy

The front-end technology was built to be a responsive two page website that used a combination ofjQuery and Underscore.js along with HTML and CSS to create a multi-device interface that would be familiar and easy to use. The main grid view uses Masonry to create the Pinterest-esque layout for search results and Underscore.js templates to load in a different layout for different data types. The single page view focuses on a low browser width touch interface to give a very image focused search experience. To combine and minify the page scripts down I used the BundleCollection library in ASP.NET MVC which worked great. There are still a large number of extra scripts on the page but I’ve tried to load them asynchronously and not let them interfere with using the interface immediately.

The site has been up and working great for a few weeks now and launched along side a new look and feel for Go take a look, and of course let me know if you find any bugs here or on the feedback tab on the bottom!

Continue reading

Post-mortem report of my free online dating service; Part 2 of 2

My free online dating site launch in May of 2009 after a little under 2 years of work. Now I just need to get people to come to or write about the site and the real work has begun. I post it up to all the social networks at the time telling people to come check out my site. “SPAMMER!” I hear. Crap, how can I tell you about the site but not tell you about the site?

I thought my solution to organic marketing would be to get other people to write about it. I searched for and made lists of blogs and sites that wrote about online dating or dating in general and started crafting individual emails for each site. For hundreds of sites I found I spent a few hours each night trying to first find a method to contact them via form or email. If I was able to find a contact method I wrote up an email about how awesome my new site was and the work I put into it and they should take a look or write about it. After a few months of this I was completely burnt out. I had gotten next to no responses from all my work, less than 1% feels generous. No one responding to me after the hundreds of communications I sent out was crushing.

I tried a couple of paid per post services but that investment didn’t pan out. The services themselves were OK, the posts went up pretty quick but the types of sites doing this weren’t exactly well respected. With all their other pay per posts it was like trying to hold a quiet conversation at a concert. A worthless avenue for marketing.

The only reliable method to get new visitors from day one that I used was online advertising through Google AdSense and Facebook Ads. I setup accounts for both services and right away I was getting click throughs and new members. Everyday I was checking my cost per registration from both services and quickly found Facebook Ads just wasn’t performing well. With just Google AdSense to focus on I created a dozen campaigns with different keywords targeting states, cities and the different sexes. I spent a good amount of money here and it was extremely addictive. I would not recommend pay per click campaigns for anyone without extremely deep pockets or free services to use. Without this service I would not have gotten any users and with it I ended up hating the project more and more. I was spending money everyday and making a little money but it was never increasing over time.

One of my biggest mistakes was not developing a proper site administration system. I underestimated the amount of profile removals I would need to make per day. I spent about 30-45 minutes on some days just cleaning up bad profiles. I finally setup a rudimentary system but even then I sometimes had to resort to my set of SQL scripts to do some tasks.

My largest problem I never resolved was spam. The only people that actually wanted to seek out and use my site were spammers. Mostly from Nigeria and it’s surrounding countries but also from Russia for mail order brides. I had ReCAPTCHA in place but they didn’t care. These weren’t robots I was keeping out but tons of people that sent out hundreds of messages a day. My final attempt was a manual message and user approval process. That meant that before any new users messages were sent I had to approve them. This was just too much work and something I never considered before launching but could never solve afterwards. They ruined the site for me and the legit users that signed up.

I’m going to shut down the service this year and will be open sourcing my code, I need to clean out any S3 key stuff. There was a lot of important lessons I learned during this experience. My product didn’t have an interesting angle. I didn’t solve any new problems or solve an existing problem so well people wanted to talk about it. My marketing was underbudget to compete against the larger players. I under developed my application and failed to think through the spam factors.

I’m happy to have done the project but I’d be lying to say I’m not even happier to have it in the past.

Check out Post-mortem report of my free online dating service; Part 1 of 2 if you didn’t see the ramp up to launch