How to use docker to execute 100 parallel selenium webdriver tests? | Do we need thread-local?

In today's fast‑paced development environment, ensuring software quality is crucial. Even when a feature is not used by end‑users but only…

Japneet Sachdeva

~5 min read · March 9, 2025 (Updated: March 9, 2025) · Free: No

In today's fast‑paced development environment, ensuring software quality is crucial. Even when a feature is not used by end‑users but only by internal teams, undetected bugs can lead to inefficiencies and unexpected issues.

In this guide, we'll discuss a detailed process of setting up an automation framework that can run 100 parallel tests using Selenium WebDriver, Java, Maven, and TestNG — all orchestrated in Docker. We'll also cover how to manage Maven dependencies, configure TestNG for parallel execution, handle different machine architectures, and safely manage driver instances using ThreadLocal.

1. The Importance of Robust QA

When QA fails to find bugs in a feature, it might indicate one of two scenarios:

A Truly Robust Feature: The tests have verified that the feature meets its acceptance criteria. This is ideal — even if it's only used internally, quality matters.
Testing Gaps: Alternatively, it could signal that your test cases haven't fully replicated real‑world admin usage, meaning edge cases could later emerge when non‑technical product team members begin using the feature. In such cases, establishing a strong feedback loop is crucial.

Even for internal tools, maintaining quality through comprehensive testing prevents inefficiencies and ensures a smooth workflow.

2. Maven Project Setup with Selenium and TestNG

Using Maven for dependency management greatly simplifies your project setup. Below is an example pom.xml that includes all necessary dependencies:

pom.xml

  <dependencies>
    <!-- Selenium Java -->
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>
    
    <!-- TestNG -->
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>${testng.version}</version>
      <scope>test</scope>
    </dependency>
    
    <!-- WebDriverManager to auto-manage browser drivers -->
    <dependency>
      <groupId>io.github.bonigarcia</groupId>
      <artifactId>webdrivermanager</artifactId>
      <version>${webdrivermanager.version}</version>
    </dependency>
    
    <!-- Optional: JUnit -->
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.13.2</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
  
  <build>
    <plugins>
      <!-- Maven Compiler Plugin -->
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.12.1</version>
        <configuration>
          <source>${maven.compiler.source}</source>
          <target>${maven.compiler.target}</target>
        </configuration>
      </plugin>
      <!-- Maven Surefire Plugin to run TestNG suite -->
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.0.0-M7</version>
        <configuration>
          <suiteXmlFiles>
            <suiteXmlFile>testng.xml</suiteXmlFile>
          </suiteXmlFiles>
        </configuration>
      </plugin>
    </plugins>
  </build>

3. Configuring TestNG for Parallel Execution

Create a testng.xml file to instruct TestNG to run tests in parallel. For 100 parallel sessions, configure the suite as follows:

testng.xml

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="ParallelSuite" parallel="methods" thread-count="100">
  <test name="ParallelTests">
    <classes>
      <class name="com.example.tests.SampleTest"/>
      <!-- Include additional test classes as required -->
    </classes>
  </test>
</suite>

4. Sample Test Class with ThreadLocal Management

When running tests in parallel, using a ThreadLocal for WebDriver ensures each thread has its own driver instance. Create a sample test under src/test/java/com/example/tests/SampleTest.java:

SampleTest.java

package com.example.tests;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.*;
import io.github.bonigarcia.wdm.WebDriverManager;
import java.net.URL;

public class SampleTest {
    // ThreadLocal to store WebDriver instances per thread
    private static ThreadLocal<WebDriver> driver = new ThreadLocal<>();

    @BeforeMethod
    public void setUp() throws Exception {
        // Set up ChromeDriver using WebDriverManager
        WebDriverManager.chromedriver().setup();
        ChromeOptions options = new ChromeOptions();
        // For remote execution, set the Selenium Grid hub URL
        URL hubUrl = new URL("http://selenium-hub:4444/wd/hub");
        WebDriver localDriver = new RemoteWebDriver(hubUrl, options);
        driver.set(localDriver);
    }

    @Test
    public void verifyTitle() {
        driver.get().get("https://www.example.com");
        String title = driver.get().getTitle();
        Assert.assertTrue(title.contains("Example"), "Title did not contain 'Example'");
    }

    @AfterMethod
    public void tearDown() {
        if (driver.get() != null) {
            driver.get().quit();
            driver.remove();
        }
    }
}

Note: I have used webDriverManager as it works with older selenium-webdriver versions. If you are using latest than skip using it!

Note: Using ThreadLocal ensures that each parallel test thread accesses its own WebDriver instance without interference.

5. Dockerizing the Framework

Dockerfile for the Test Runner

Create a Dockerfile in your project root to build and run the tests:

# Stage 1: Build the project using Maven with JDK 17
FROM maven:3.8.6-openjdk-17 AS build
WORKDIR /app
COPY pom.xml .
RUN mvn dependency:go-offline -B
COPY src ./src
RUN mvn clean package -DskipTests

# Stage 2: Run tests using the built project
FROM maven:3.8.6-openjdk-17
WORKDIR /app
COPY --from=build /app .
CMD ["mvn", "clean", "test"]

Build the image with:

docker build -t my-selenium-tests

Docker Compose Setup for Selenium Grid and Test Runner

Create a docker-compose.yml file to spin up a Selenium Grid and the test runner. This example is for Linux‑based images. For ARM‑based machines (like Apple M1/M2), substitute with ARM‑compatible images (e.g., seleniarm/standalone-chrome):

version: "3"
services:
  selenium-hub:
    image: selenium/hub:4.10.0-20230426
    container_name: selenium-hub
    ports:
      - "4444:4444"

  chrome:
    image: selenium/node-chrome:4.10.0-20230426
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_MAX_SESSIONS=5
    deploy:
      replicas: 20  # 20 nodes x 5 sessions each = 100 sessions

  test-runner:
    image: my-selenium-tests
    depends_on:
      - selenium-hub
    networks:
      - default

networks:
  default:
    driver: bridge

Windows/macOS Note: On Docker Desktop for Windows/macOS, host networking isn't fully supported. Using a shared bridge network (as above) ensures that the test runner container can resolve selenium-hub correctly.

Start the setup with:

docker-compose up --build

6. Multi‑Platform and Architecture Considerations

Windows & macOS: Docker Desktop on Windows and macOS runs a lightweight Linux VM; the provided Compose setup works similarly across both. Make sure your Docker Desktop is updated.
ARM vs. x86‑64: For ARM platforms (like Apple M1/M2), you must use images built for ARM. For example, in your docker-compose.yml, change the chrome service's image to an ARM‑compatible one:

image: seleniarm/standalone-chrome:4.10.0-20230426

7. Bringing It All Together

End-to-End POC Summary

Maven Project & Dependencies:

Use the provided pom.xml to manage Selenium, TestNG, and WebDriverManager dependencies.

TestNG & Parallel Execution:

Configure testng.xml for parallel test execution (100 threads).

Test Implementation:

Write tests with ThreadLocal-managed WebDriver instances.

Dockerization:

Create a Dockerfile to build and run the Maven project.
Use Docker Compose to spin up a Selenium Grid (hub plus multiple Chrome nodes) and a test-runner container.

Platform Considerations:

Adjust Docker images for Windows/macOS and ARM architectures as needed.

Execution:

Build the Docker image and bring up the grid with docker-compose up --build.

This complete setup acts as a robust proof-of-concept for executing 100 parallel Selenium tests in a containerized environment.

Conclusion

By following this detailed guide, you now have an end-to-end framework for:

Managing Selenium dependencies using Maven,
Running parallel tests with TestNG,
Orchestrating a scalable Selenium Grid with Docker, and
Ensuring thread safety via ThreadLocal for your WebDriver instances.

This framework is adaptable for various platforms and architectures, making it a solid foundation for live projects. If you have any questions or need further details, please leave a comment or reach out!

Job Guarantee Full Stack QA & Automation Course Get your next job as an SDET in 2025: Link

Subscribe to my YouTube Channel: https://www.youtube.com/@IamJapneetSachdeva

#japneetsachdeva

#test-automation #automation-testing #testing #software-testing #docker