At CHEQ, we block bots. Our mission is to fight the fake web, assisting businesses in directing their efforts toward authentic users, thereby elevating and scaling every aspect of Go-to-Market performance.
Over the last few years, the startup company has seen massive growth, with new users and new eyes constantly using and looking at our line of products. This required a new approach to quality — a shift from building quickly, a massively demanded product, to starting to build it right — with quality gates in place, which one of the most important ones as we know is — Testing Automation.
This is how we've done that.
When I arrived, the company had already developed and managed 2 products: CHEQ ClickCease, CHEQ Paradome, where the latter was targeting a large-scale enterprise-fortune-500 clients, and the former's target audience was small-to-medium businesses or advertising agencies looking to protect their Go-To-Market funnels. In addition, the company had already started working on a revamped CHEQ ClickCease product — we now call CHEQ Essentials, which is built in a more modern stack and designed to serve mostly medium businesses and digital marketing agencies. Its stack is Module-Federated React Frontends with a Nest.js backend and Postgres MySQL database, with Playwright as its testing automation engine.
My mission was to build the automation infrastructure for CHEQ Essentials from the ground-up which will be developed hand-in-hand with the product itself, and with the essential processes in place for the product to grow with quality gates in mind. During this process, we came to the realization that the technical part of it — is only half of the story. The other half is about people.
When starting to build a product from scratch, including its radar elements such as automation infra, I was looking at 3 main pillars:
- Technology & Tools — Choosing the right packages, putting the infrastructure in place, writing quality maintainable and readable code, connecting it seamlessly to the existing CICD deployment pipeline, and more.
- People — The team back then was not educated to even think about the concept of putting guardrails such as automated tests in place. This required a re-wiring of how to approach a development task with the developers.
- Processes — We were moving fast. Without proper and easy-to-grasp development processes, it would be very hard for the automation project to keep up with the Product team's requirements, and the fast-paced development of the product.
Technology & Tools
The testing framework that was chosen for the task was Playwright — a framework for Web Testing and Automation. Our product is a SaaS web-based app and it seemed like a great fit. There are other solutions out there and you should choose the right tool for the job.
Further questions like implementing the testing engine as part of the product's code in comparison to creating an external project were also debated quite a lot. We went with the latter for 2 main reasons:
- E2E tests are service-agnostic. They act as a user using the entire flow of your application. A specific test doesn't really know which service it sees at any given point in time — therefore it's cross-frontends (and backends).
- Decoupling the deployment process of the automation project from the services is much more convenient to handle. A small fix in a test (which occurs a lot) doesn't require a full application deployment, with all its branches and environments.
So now we have an external Playwright-based automation project ready to be populated with tests validating the flows of the application — but which flows?
We have decided that the terms for which an E2E test will be created is if the flow has a major impact business-wise or the flow is critical in the app. Examples for which would be: Signup (and payment), login, upgrade/downgrade packages, important application interactions such as onboarding flow, and more. The reason we only choose these and not create a massive suite of tests, is that E2E tests tend to be flaky, and there is nothing more important than keeping the suite stable, as if it's not — people tend to ignore it, with time.
I won't get into too much detail about the structure of the automation project itself as it has enough content for a separate article, but a partial list of things to keep in mind would be: Tests-per-Environments (not all tests should run for each environment — dev, stg, prod. Use Playwright configs inheritance), Tests-per-Frontend service (changing code on service-A doesn't have to invoke all tests in the suite, but rather only the relevant tests — use Playwright tags), dockerizing the project so that the CICD process will pull its image and run the tests as part of the deployment pipeline with input variables such as current env vars and invocation mode, timely run versus per-deployment run, reporting, reverting mechanism, and many more aspects.
I would add that in addition to our E2E tests suite, we have integrated API testing infrastructure validating our backend services running on each deployment. This article is focusing on the quality-integration journey in the company, so I won't elaborate too much on this, but that's an important part of automated testing railroads.
People & Processes
By now, we have faced the technology challenge, but that might be the easy part. Without proper education for the team/s, no line of code written would be worth it.
When I started, the entry point was that the developers' mindset was not in place for automated testing. We built a new product from scratch, we received tons of requirements from the Product team, and we were facing tight schedules. On top of that, the company's goal was to make a shift from a dedicated team that handles automation, to an integrated mode where all developers should take ownership of their feature, including the documentation, quality, and automation aspects.
The first thing I did was gather the teams and for a short 30-minutes presentation explained what all this deal is about automation, and about the importance of it. People have to be aligned with your vision so that when a task comes in, they'll have the proper motivation. Next, we went through a few sessions about how to actually write tests using Playwright, how to respond to a failing test, how a development process looks when automated tests phases are integrated and more. Don't forget that developers need to learn a whole new syntax and mindset, which is not easy in addition to their routine development tasks.
When an actual test task arrived to a developer for the first time, I created the test myself with a shared screen while he watched my steps. That is a gradual entrance to the world of E2E tests. Next, they created the tests based on my templated and sent me for a PR review, which was then fixed with comments and merged into the automation project.
With time, the integration of the automation processes into peoples' routines was complete. It took some supervision and control — as well as some nagging. I noticed people started asking me what is going on with the automation part for feature A or B. This made me realize the transformation is complete.
Final Thoughts
The relatively long process of transforming development teams to quality-in-mind with everything that has to go with it, is not mission impossible. With some degree of dedication from the teams, some planning, as well as hands-on supervision, this can be a fun process, leveling up the standard for the company you work for and for the team you are a part of. Hopefully this article gave some points that should be looked at when starting this process, at least based on my experience.
This article was focused on our E2E framework and infra. We have also implemented per-service integration (api) tests that cover the most important endpoints of our backend services that serve either our front-end applications, as well as our identification & blocking engine. The people & processes part for this side of the automation infrastructure is not very different, but the tech & tooling — is.