With so many considerations to be made, Salesforce test automation can be a trap for inexperienced players. But with expert knowledge and experience, these challenges can be overcome. Idea Science’s test automation guru Georg Fischer shares his secrets to navigating the automation of Salesforce testing.
Focus on minimising risk and tracking your risk coverage
It’s important to acknowledge that in most projects there is usually an insufficient amount of time to automate everything. And because testing typically gets crunched into a small timeframe, trying to automate every test case is risky. The good news is you don’t need to. By rethinking the testing approach, focusing on quality over quantity, and acknowledging that in most instances stakeholders don’t actually need perfect software, you can maintain quality with reduced test coverage. They do, however, need to provide a positive user experience (UX). The Pareto principle is useful here. It states that for many outcomes, roughly 80% of consequences come from 20% of the causes.
So, how do we achieve sufficient test coverage to optimise UX with less test cases? By focusing on business risk. Even with formally defined test cases, such as executing 1113 of 1140 tests and having 1000 pass, this provides little real value and context to the business. We need to know if the failed tests relate to business-critical processes and those that are used most frequently. The percentage of test cases automated or passed would not be a bad metric if all of them were equally important, but they are not.
If you want a quick assessment of the risks associated with promoting the latest Salesforce release candidate to production, below is a simple example to assess risk coverage.
Let’s assume your requirements are weighted using the following simple method:
If you fully cover the lead management requirement, you’ll cover 80% of your risk. Even if you only cover sales leads you will still cover 69% of your risk. This approach applies for testing Salesforce or any other system.
Choose the right tools and framework to go beyond Salesforce
This might be a bit controversial in an article that is talking about Salesforce test automation, but one of your main considerations should be that your organization’s business processes extend beyond Salesforce. As such, your test automation should also reflect this. This is not simply saying that all test automation should cover End-to-End business processes. In fact the majority of your tests should not be End-to-End tests, but more on that later. Ideally you want to use the same framework, the same approaches and best practices across your entire organisation. Choose a framework that works for your whole business and supports all the technologies that you are supporting. This allows you to more easily share resources, knowledge and automation engineers between teams. Don’t just expect change, embrace change. Between Salesforce releases, patches, third party apps, and your own customisations and integrations, your Salesforce solution/ecosystem is constantly changing. If you decide to leverage a code-based approach eg. Selenium, make sure you stay up to date with the latest and upcoming changes in Salesforce found in the Trailblazer Community.
If you are using a commercial test automation tool, aim for one that allows access to Salesforce Pre Release Business Scenario Testing (BST), such as Tricentis Tosca.
Tosca allows you to access Salesforce’s Pre-Release Business Scenario Testing, which provides the option to run automated tests against new Salesforce releases before they become available to the public.
It’s also useful to ensure that your Salesforce test automation framework is as agnostic as possible. If your functionality is available on Classic, Lightning and a community portal, your framework needs to allow you (without significant maintenance effort) to execute tests in all of them. Similarly, if your system is being used on mobile devices, make sure your test automation can run on mobile devices as well as your regular web browser.
It’s important, wherever possible, to decouple test cases from test data. Doing so allows the automation to run across multiple environments with different data sets and provides easy extensibility if you are adding additional record types and values later.
Another useful consideration is to leverage module driven test automation. This approach significantly reduces your maintenance efforts as you only need to maintain technical identification in one place. For example, when Salesforce’s Winter ’20 release occurred, you could no longer use ‘document’ or ‘document.body’ to access the shadow DOM of Salesforce Lightning web components. However, by using a module driven framework, the maintenance burden to update the automation to reflect this change is greatly reduced.
Shift to API testing to flip the traditional test pyramid
Test automation is not just about automating manual (E2E) tests.
One of the main issues for preventing defects and increasing delivery speed is that code is often delivered late, which is why you should embrace API test automation. API test automation allows for progression automation that can be conducted while development is still being completed. All you need is your web service definition and you can automate your test cases. This essentially flips the traditional testing pyramid on its head.
Your API test automation should focus on smaller more specific tests but still allow you to link these test cases into business scenarios. Essentially you should aim for something like this:
There’s plenty more to discuss about the testing pyramid and how to stop it from eroding. We’ll visit this in a future Idea Science blog post. Want to know more about optimising your Salesforce test automation? Idea Science can help.
