Re-using existing browser session in selenium

Comparison between Selenium and UFT Behavior

For those of us who come from a QTP/UFT background, being able to test the same browser after a disconnect is usually a piece of cake

1. systemutil.run "iexplore.exe"
2. x = 2/0
3. Browser("index:=0").navigate ("http://www.google.com")

If we run the above script in QTP/UFT, the script will error out on Line #2 and we can re-run the script from Line #3 and it would still work and navigate inside the browser we had opened earlier.

UFT also works when the browser is launched manually and not through script. This is not possible in Selenium though because of the architectural differences.

Since the selenium developers don’t see this as a much needed feature, they are strongly against adding this to the selenium code.

There are solutions people have spoken about, which requires patching the webdriver.py file. I don’t recommend such solutions as it makes it difficult to update the selenium package to newer versions.

In this article we will see a much cleaner and effecient way to achieve this

Launching a browser in Selenium

To launch a browser in Selenium using python, we just import the webdriver and start a browser

from selenium import webdriver

driver = webdriver.Chrome()

If we run the above script, it will launch the browser and exit. Since the browser object we created got destroyed, we no more can control this browser using script. This is a zombie browser in terms of Selenium, though one can kill it manually

Now when we create a WebDriver (Firefox, chromer or any browser) object in Selenium there are few things that happen

The intended browser agent is started (chromedriver for chrome, geckodriver for Firefox and so on)
A command executor is created. This object is responsbile for sending commands to the agent
A new session is established with the agent, which in turn communicates with the browser. A session_id is also generated to identify the session.

The information from Step #2 and Step #3 is important for us to be able to re-create a driver object. So let us update our script to print this information

from selenium import webdriver
driver = webdriver.Chrome()

executor_url = driver.command_executor._url
session_id = driver.session_id

print session_id
print executor_url

driver.get("http://tarunlalwani.in")

A sample output of the above is as below

7397a385-7661-0449-9e0a-902355617485
http://127.0.0.1:51259

Creating driver using Session ID and Command Executor

So now we have the session_id and executor_url. Let’s try and recreate the session

My first take was to create the driver using the RemoteWebDriver or WebDriver driver

from selenium import webdriver

driver = webdriver.Chrome()
executor_url = driver.command_executor._url
session_id = driver.session_id
driver.get("http://tarunlalwani.in")

print session_id
print executor_url


driver2 = webdriver.Remote(command_executor=executor_url, desired_capabilities={})
driver2.session_id = session_id
print driver2.current_url

When we run the above code, it prints the url http://tarunlalwani.in at the end. Which means we were able to recreate the driver object for a Chrome browser.

Testing on Firefox browser

We tested our previous approach on Chrome browser. Let’s test our solution on Firefox as well. When we update our code and run it

from selenium import webdriver

driver = webdriver.Firefox()
executor_url = driver.command_executor._url
session_id = driver.session_id
driver.get("http://tarunlalwani.in")

print session_id
print executor_url


driver2 = webdriver.Remote(command_executor=executor_url, desired_capabilities={})
driver2.session_id = session_id
print driver2.current_url

This errors out at driver2 = webdriver.Remote(command_executor=executor_url, desired_capabilities={}) in our script

Traceback (most recent call last):
  File "/Users/tarun.lalwani/Desktop/tarunlalwani.in/tarunlalwani/content/code/selenium_enumerator.py", line 101, in <module>
    driver2 = webdriver.Remote(command_executor=executor_url, desired_capabilities={})
  File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 98, in __init__
    self.start_session(desired_capabilities, browser_profile)
  File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 188, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Session is already started

The error occurs at response = self.execute(Command.NEW_SESSION, parameters). The issue is that geckodriver doesn’t support firing newSession command again. So we need a way to override the execute function while creating the driver

We create a new function which returns a dummy response on newSession with our sessionId. Updated code is as below

from selenium import webdriver

driver = webdriver.Firefox()
executor_url = driver.command_executor._url
session_id = driver.session_id
driver.get("http://tarunlalwani.in")

print session_id
print executor_url

def create_driver_session(session_id, executor_url):
    from selenium.webdriver.remote.webdriver import WebDriver as RemoteWebDriver

    # Save the original function, so we can revert our patch
    org_command_execute = RemoteWebDriver.execute

    def new_command_execute(self, command, params=None):
        if command == "newSession":
            # Mock the response
            return {'success': 0, 'value': None, 'sessionId': session_id}
        else:
            return org_command_execute(self, command, params)

    # Patch the function before creating the driver object
    RemoteWebDriver.execute = new_command_execute

    new_driver = webdriver.Remote(command_executor=executor_url, desired_capabilities={})
    new_driver.session_id = session_id

    # Replace the patched function with original function
    RemoteWebDriver.execute = org_command_execute

    return new_driver

driver2 = create_driver_session(session_id, executor_url)
print driver2.current_url

This code works fine. The only caveat being that we loose the actual capabilities that the browser returned when it was launched. This should not be a big deal as such in most cases.

Why and where to re-use the session?

Now we have achieved something, but the question remains as to what do we do with it? And why do we need it even?

Well we have used it in few particular scenarios

Idle Browser

In one of our scraping project, we want to end our script and leave the browser idle. When the script re-runs it re-uses the last browser and continues the work.

This mimics how a human would actually behave. Which can become an important factor when scraping sites with higher security against scraping

Continuing exited script

In one of our automation scripts, we were going through different types of pages and had handling for most of them. But there were certain edge cases.

So we wanted our script to stop and fail. Once we patch and re-run it, it resumes the work from existing open url

Framework functionality

One of our automation framework, allowed users to execute the automation scripts from any point. For implementing such feature it was important to be able to recreate the last session driver.

TARUN LALWANI