📣 Requestly API Client – Free Forever & Open Source. A powerful alternative to Postman. Try now ->

Mastering Selenium Assertions: Hard vs. Soft Asserts

Azma Banu
Learn the difference between hard and soft assertions in Selenium and how to use them for effective test automation
Mastering Selenium Assertions_ Hard vs Soft Asserts

Selenium automates browser interactions but lacks built-in assertion methods to validate test results. To fill this gap, testing frameworks like TestNG, JUnit, and unittest are used.

Assertions in Selenium are crucial for validating test outcomes. The two main types—Hard Assertions and Soft Assertions—differ in how they handle test execution when a condition fails.

  • Hard Assertions: Halt test execution immediately upon failure, used for critical validations, throwing an exception and marking the test as failed.
  • Soft Assertions: Let tests continue despite failures, useful for gathering multiple results, and require assertAll() to report all outcomes at the end.

This article explores the difference between assert and verify, and how to effectively use these methods in Selenium automation.

What is Selenium?

Selenium is a powerful tool for browser automation, enabling developers to simulate user interactions with web applications.

However, while Selenium facilitates interaction with web pages, it lacks built-in robust assertion methods that are critical for validating test outcomes. Assertions allow testers to verify if actual results meet expected outcomes, which is essential for determining the success or failure of test cases.

Since Selenium doesn’t natively provide an assertion mechanism, it relies on external testing frameworks such as TestNG, JUnit, or Python’s unittest to handle assertions. These frameworks enhance Selenium’s capabilities by providing a way to compare expected results against actual outcomes and report success or failure accordingly.

Types of Assertions in Selenium: Hard vs. Soft

Assertions play a key role in automating tests, enabling testers to confirm that applications behave as expected. The two primary types of assertions used in Selenium are Hard Assertions and Soft Assertions. Both are used to verify the expected outcome of a test, but they differ in how they handle test execution upon failure.

Hard Assertions

Hard assertions immediately stop the execution of a test case if the assertion fails. This is useful for cases where the test cannot continue meaningfully if a specific condition is not met. For example, when validating critical functionalities or business logic, a failed assertion should halt further execution to avoid running meaningless steps. In case of failure, a java.lang.AssertionError exception is thrown, and the test case is marked as failed.

Examples of common hard assertions include:

  • assertEquals()
  • assertNotEquals()
  • assertTrue()
  • assertFalse()

Soft Assertions

Soft assertions, unlike hard assertions, allow tests to continue even if an assertion fails. This means that all the checks in the test are executed, and the results are reported at the end, allowing the tester to view all errors at once.

Soft assertions are useful when you want to perform multiple checks in a single test without stopping the execution due to an assertion failure. To view the results of soft assertions, you need to call assertAll() at the end of the test.

Hard Assertions vs. Soft Assertions

Feature

Hard Assertions

Soft Assertions

Test Execution

Stops test execution if failed

Continues execution even if a failure occurs

Result Reporting

Immediate reporting

Results are reported after the test completes (via assertAll())

Method Invocation

No need for additional method calls

Must invoke assertAll() to view results

Exploring Hard Assertions in Selenium

Hard assertions are crucial when you need immediate validation of a specific condition within a test case. Let’s dive into some common methods of hard assertions in Selenium.

assertEquals()

This assertion compares two values (actual and expected) and ensures they are equal. If they are not, the test case is marked as failed.

Example of assertEquals():

				
					import static org.testng.Assert.assertEquals;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.Test;

public class BrowserStackTutorials {
    @Test
    public void testAssertEquals() {
        System.setProperty("webdriver.chrome.driver", "C:\\path\\to\\chromedriver.exe");
        WebDriver driver = new ChromeDriver();
        driver.navigate().to("https://www.browserstack.com/");
        String actualTitle = driver.getTitle();
        String expectedTitle = "Most Reliable App & Cross Browser Testing Platform | BrowserStack";
        assertEquals(actualTitle, expectedTitle, "Page title does not match the expected value");
        driver.quit();
    }
}
				
			

If the actual title doesn’t match the expected one, the test will fail, and the message “Page title does not match the expected value” will be displayed.

assertNotEquals()

This assertion checks that two values are not equal. If the values match, the test will fail.

Example of assertNotEquals():

				
					import static org.testng.Assert.assertNotEquals;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.Test;

public class BrowserStackTutorials {
    @Test
    public void testAssertNotEquals() {
        System.setProperty("webdriver.chrome.driver", "C:\\path\\to\\chromedriver.exe");
        WebDriver driver = new ChromeDriver();
        driver.navigate().to("https://www.browserstack.com/");
        String actualTitle = driver.getTitle();
        String expectedTitle = "Incorrect Title";
        assertNotEquals(actualTitle, expectedTitle, "Titles should not match, but they do");
        driver.quit();
    }
}
				
			

Here, if the actual title matches the incorrect title, the assertion will fail with the message “Titles should not match, but they do“.

assertTrue()

This method checks if the given condition evaluates to true. If the condition is false, the test will fail.

Example of assertTrue():

				
					import static org.testng.Assert.assertTrue;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.Test;

public class BrowserStackTutorials {
    @Test
    public void testAssertTrue() {
        System.setProperty("webdriver.chrome.driver", "C:\\path\\to\\chromedriver.exe");
        WebDriver driver = new ChromeDriver();
        driver.navigate().to("https://www.browserstack.com/");
        boolean verifyTitle = driver.getTitle().equalsIgnoreCase("Most Reliable App & Cross Browser Testing Platform | BrowserStack");
        assertTrue(verifyTitle, "The page title is not as expected");
        driver.quit();
    }
}
				
			

If verifyTitle is false, the assertion fails and the message “The page title is not as expected” is shown.

assertFalse()

The opposite of assertTrue(), this method checks if the given condition evaluates to false. If the condition is true, the test will fail.

Example of assertFalse():

				
					import static org.testng.Assert.assertFalse;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.Test;

public class BrowserStackTutorials {
    @Test
    public void testAssertFalse() {
        System.setProperty("webdriver.chrome.driver", "C:\\path\\to\\chromedriver.exe");
        WebDriver driver = new ChromeDriver();
        driver.navigate().to("https://www.browserstack.com/");
        boolean verifyTitle = driver.getTitle().equalsIgnoreCase("Incorrect Title");
        assertFalse(verifyTitle, "The title matched an incorrect value");
        driver.quit();
    }
}
				
			

If verifyTitle is true, meaning the actual title matches the incorrect title, the assertion fails and the message “The title matched an incorrect value” is displayed.

assertNull() and assertNotNull()

These assertions check if a value is null or not null, respectively. If the condition is not met, the test fails.

Example of assertNull():

				
					import static org.testng.Assert.assertNull;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.Test;

public class BrowserStackTutorials {
    @Test
    public void testAssertNull() {
        System.setProperty("webdriver.chrome.driver", "C:\\path\\to\\chromedriver.exe");
        WebDriver driver = new ChromeDriver();
        String verifyAssertNull = null;
        assertNull(verifyAssertNull, "Expected value is not null");
        driver.quit();
    }
}
				
			

Example of assertNotNull():

				
					import static org.testng.Assert.assertNotNull;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.Test;

public class BrowserStackTutorials {
    @Test
    public void testAssertNotNull() {
        System.setProperty("webdriver.chrome.driver", "C:\\path\\to\\chromedriver.exe");
        WebDriver driver = new ChromeDriver();
        driver.navigate().to("https://www.browserstack.com/");
        String actualTitle = driver.getTitle();
        assertNotNull(actualTitle, "Page title should not be null");
        driver.quit();
    }
}
				
			

Soft Assertions in Selenium

Unlike hard assertions, soft assertions do not stop the execution of a test when a condition fails. Instead, they allow the test to continue and report the results at the end of the test run.

This is useful when you want to check multiple conditions without halting the test due to a single failure. To gather results from soft assertions, the assertAll() method must be invoked at the end of the test.

Example of Soft Assertions:

				
					import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.Test;
import org.testng.asserts.SoftAssert;

public class BrowserStackTutorials {
    @Test
    public void testSoftAssert() {
        System.setProperty("webdriver.chrome.driver", "C:\\path\\to\\chromedriver.exe");
        WebDriver driver = new ChromeDriver();
        SoftAssert softAssert = new SoftAssert();
        driver.navigate().to("https://www.browserstack.com/");
        String actualTitle = driver.getTitle();
        softAssert.assertEquals(actualTitle, "Most Reliable App & Cross Browser Testing Platform | BrowserStack", "Title does not match");
        softAssert.assertNotEquals(actualTitle, "Incorrect Title", "Title matches an incorrect value");
        softAssert.assertNotNull(actualTitle, "Page title should not be null");
        softAssert.assertAll();  // Reports all assertion results
        driver.quit();
    }
}
				
			

Implementing Soft Assertions Without a Framework

While frameworks like TestNG and JUnit make soft assertions straightforward, it is also possible to implement them manually. This involves creating a custom mechanism to track assertion results and report them at the end of the test.

Manual Soft Assertion Implementation Example:

				
					import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.Test;
import java.util.ArrayList;
import java.util.List;

public class BrowserStackTutorials {
    static class AssertionResult {
        boolean passed;
        String message;
        AssertionResult(boolean passed, String message) {
            this.passed = passed;
            this.message = message;
        }
    }

    @Test
    public void testManualSoftAssert() {
        System.setProperty("webdriver.chrome.driver", "C:\\path\\to\\chromedriver.exe");
        WebDriver driver = new ChromeDriver();
        driver.navigate().to("https://www.browserstack.com/");
        List<AssertionResult> assertionResults = new ArrayList<>();
        String actualTitle = driver.getTitle();
        String expectedTitle = "Most Reliable App & Cross Browser Testing Platform | BrowserStack";
        
        if (actualTitle.equals(expectedTitle)) {
            assertionResults.add(new AssertionResult(true, "Title matches as expected"));
        } else {
            assertionResults.add(new AssertionResult(false, "Title does not match the expected value"));
        }

        for (AssertionResult result : assertionResults) {
            if (result.passed) {
                System.out.println("PASSED: " + result.message);
            } else {
                System.out.println("FAILED: " + result.message);
            }
        }

        driver.quit();
    }
}
				
			

Why choose BrowserStack for Testing Selenium Assert Tests?

BrowserStack Automate is a powerful cloud-based platform designed to simplify Selenium testing. It allows teams to run automated Selenium tests on real browsers and devices, eliminating the need for managing physical infrastructure.

BrowserStack Automate ensures that web applications are tested across a variety of environments to guarantee cross-browser compatibility, scalability, and reliability.

Here’s why you must choose to run your Selenium tests on BrowserStack:

1. Real Browsers and Devices: Run Selenium assert tests on over 3000 real browsers and devices, ensuring accurate results under real user conditions.

2. Cross-Browser Compatibility: Execute tests across multiple browsers (Chrome, Firefox, Safari, Edge) to validate assertions on various platforms without setup hassles.

3. Parallel Testing: Run tests in parallel on multiple configurations, reducing execution time and speeding up feedback on assertion failures.

4. CI/CD Integration: Integrate seamlessly with your CI/CD pipeline, automating the execution of Selenium tests and ensuring fast detection of assertion errors.

5. Real-Time Logs and Screenshots: Access real-time logs, screenshots, and video recordings to debug failing assertions quickly and efficiently.

6. No Infrastructure Overhead: Eliminate the need for local test infrastructure. BrowserStack manages the setup, scaling, and maintenance for you.

Conclusion

Assertions are an integral part of Selenium test automation, ensuring that tests are validated and behave as expected. Both hard and soft assertions offer unique advantages, depending on whether you want immediate test termination or wish to continue executing the test to gather all errors.

Selenium, combined with appropriate assertion methods, helps testers verify application behavior accurately and efficiently. For best results, it’s also recommended to run tests on real devices, which provides more realistic testing conditions and results.

With tools like BrowserStack Automate, you can execute Selenium tests across a wide range of real devices and browsers for comprehensive testing.

Written by
Azma Banu

Related posts