If you kick off some tests in selenium or protractor chances are that they run smoothly in one browser but not even start in the other. The reason for that can be anything and more often than not, hard to solve. That is why I started this project on GitHub to demonstrate how you can use Selenium, Protractor or WebdriverIO with different test runners, assertion libraries and browsers running tests locally, remote or even in parallel.
But before I begin first follow the steps described in the project's README to get everything up and running.

Basic Authentication

To login to the demo website you have to deal with Basic Authentication first. As you might already know Selenium cannot control the native popup you get with basic auth. In my search to solve this I found two solution which overcome these shortcomings.

The most easiest solution is to add the username and password to the url

http://foo:bar@localhost

That's it, BUT not for Firefox, they have an addition popup

Fortunately this popup can be disabled with the config property network.http.phishy-userpass-length. For that to work with Selenium, you have to use Firefox profiles, which you can see in action here.

But it is also possible to handle the authentication with NodeJS, before Selenium takes over, but it requires a different setup. The site itself should be public and the authentication part protected. After you've authenticated a redirect will bring you to your public site, but with some kind of authentication token appended to the url. In such case you can do the authentication with NodeJS followed by Selenium which uses this redirect url as its entry point. You can see the NodeJS part in action here.

Local testing

There is only one test case in this project, which looks more or less like this

expect(HomePo.title).toEqual('Hello world!');

And there is also a chai/should and cucumber variant. I admit, not very spectacular, but it serves its purpose very well :)

NodeJs

To run this test with pure nodeJs

$> ./node_modules/.bin/babel-node ./src/selenium.js

To simplify things a bit most commands are added to package.js. The above command can be shortened to

$> yarn start

Code

Selenium

To run selenium you need to define the browser you want to use for testing

$> yarn start -- --browse=chrome

This will only work if you have the browser driver available. If you want to use a selenium standalone server run the command as follows

$> yarn start -- --browse=chrome --selenium-standalone

Code.

Mocha

The Mocha tests can be run as follows

$> yarn mocha -- --browse chrome
$> yarn mocha -- --browser firefox --browser-version 51

I run mocha programmatically which gives you more control over what is happening as can be seen here.

Protractor

With protractor things get easier, it can start the selenium server for you. So the only thing you have to type is

$> yarn protractor -- --browse chrome
$> yarn protractor -- --browser firefox --browser-version 51 --cucumber

The first example runs the jasmine specs while the second runs cucumber tests

WebdriverIO

WebdriverIO supports the same options as Protractor

$> yarn wdio -- --browser firefox --browser-version 51 --cucumber

With the current version of Protractor and WebdriverIO you don't need to use firefox profiles anymore in order to solve the issue with basic auth, its all taken care of now!

WebdriverIO also comes with a handy tool called ./node_modules/.bin/wdio which can setup the configuration file and run the specs with a lot of interesting options. Furthermore, in contrast to selenium and protractor, WebdriverIO transforms all actions to act synchroniously, for example

browser.getText('h1') === 'Hello world!'

It is also available for angular with webdriverjs-angular. It waits for angular to be ready, but they haven't enhanced it with angular specific commands as the developers think that functional tests should be as framework-agnostic as possible.

Remote testing with Browserstack

With your Browserstack automation credentials you can now run your tests in the cloud. The only thing that is added to the configuration of Protractor and Webdriver IO is

export.config = {
    browserstackUser: myName,
    browserstackKey: myAccessKey,
    ...
    capabilities: {
        'browserstack.local': true,
        'browserstack.debug': 'true'
        ...
    }
    ...
}

Note that the configuration can change a bit depending on the tooling. To test the demo site on browserstack you need to add --browserstack-user name and --browserstack-key access-key argument to the above commands and the tests will auto-magically run on browserstack :) If it is a bit too much for you to add these to every command, you can also use environment variables

$> export BROWSERSTACK_USERNAME=name
$> export BROWSERSTACK_ACCESS_KEY=access-key

Here is an example how selenium can be started

$> yarn start -- --browser ie --browser-version=9
                --os=windows --os-version=7
                --browserstack-user barfoo --browserstack-key liufo7ag4

Checkout Browserstack to see what operating systems, devices and browsers are supported. Furthermore, as you might have noticed, running tests locally with Firefox is a true nightmare, but not on Browserstack, apparently they fixed all that. They even fixed that Firefox issue with Basic Authentication.

Parallel testing

In Protractor parallel sessions are controlled by a couple of arguments. First, with maxSessions you can define the total amount of browser used to run the tests. With maxInstances you can control the total amount of browsers per capability. Finally, the shardTestFiles property lets you share spec files between browser instances, which is good for performance

exports.config = {
    ...
    maxSessions: -1 // unlimited

   multiCapabilities: [{
           browserName: 'chrome',
           shardTestFiles: true,
           maxInstances: 10
       },
       {
           browserName: 'edge',
           shardTestFiles: true,
           maxInstances: 2
       }],
      ...
};

WebdriverIO has a slightly different configuration for parallel testing

exports.config = {
   ...
   maxInstances: 10

   capabilities: [{
           browserName: 'chrome',
           maxInstances: 10
       },
       {
           browserName: 'edge',
           maxInstances: 2
       }],
      ...
};

Available CLI options

During this project the amount of configuration options has increased to a serious amount. They are all defined inside config.js But I've listed the most important ones below

--browser name	Name of the browser to be used for testing
--browserstack-key access-key	The access-key of the Browserstack account
--browserstack-user username	The username of the Browserstack account
--browser-version number	Browser version
--cucumber	Run cucumber feature tests (only for WebdriverIO and Protractor)
--max-instances number	Max amount of browser instances for one capability
--max-sessions number	Total amount of browser instances for all capabilities
--os name	Name of the operating system (Browserstack only)
--os-version number	Version of the OS (browserstack only)
--selenium-standalone	Use selenium standalone (Note that you have to start the server yourself!)

Conclusion

Considering all the work you have to do to get testing on your local machine up and running, I advice to use Browserstack. They have and will solve all the issue that can possibly happen. Like they did with Basic authentication. Local testing is of course faster, and should probably be the only reason why you want to do it.

Furthermore, considering the testing frameworks I've used in this project, WebdriverIO is the big winner. They take away all the pain with their wdio command line tool, its awesome. But the biggest advantages is that everything is done synchronously.