Automated Testing. For recent years, it has been constantly regarded as one of the potential tech frontiers filled with countless resources and investments. According to a recent report published by KPMG, 85% increment has been observed in test automation over a 2-year period across all industry domains. The automated testing is not the panacea, but definitely, one of the game-changers to enhance your business growth with a promising future.
The following report aims at guiding you through some major misconceptions between the manual and automated testing.
1) Test Automation? Automated Testing?
Before we compare two different testing methodologies, we should initially register due attention on differentiating these two confusing concepts. The most common explanations are: Test automation focuses on automating the whole process of trailing up and coordinating different tasks, while automated testing is served for specific tests (e.g. a set of regression tests) through automation as opposed to conducting them manually.
In this bustling era when DevOps becomes prevailing, automated testing is the glue where all this rapid delivery of value can exist. Meanwhile, test automation, as a theoretical term, seems fictional for its grandiose idea of sheer automation without any human efforts. In reality, no matter which phrase you are applying to, the automatic feature only serves as the supports to the manual testing.
2) Automated Testing for all goods?
Efficiency. Cost-effectiveness. Accuracy.
There 3 words are commonly mentioned in the article praising about the Automated Testing as if the Manual one can only perform when human observation is highly demanded to ascertain the user-friendliness. Based on our previous project, we absolutely cannot ignore its dominance in multiple testing, better reporting, saving the time & money and avoiding the simple human mistakes. Indeed, it is of great help to seemingly invest less in the development. So, is Automated Testing for all goods? The answer may not be as positive as what you think.
In fact, automated testing is still not reaching its full-blown phase. The dream of “set the frame and leave it alone” is not appearing as often as what we expect. Usually, the application updates are together with maintaining the source code and updating the automated testing scripts. Well, with being said, this also crashes the hope of entirely tool-based automation. As suggested by experts as an unspoken truth, a linear “record and playback” approach with non-technical resources is not maintainable.
So Automated Testing is great but it has its limitations. Basically, it’s not for all goods. Well, maybe nothing is. 🙂
3) When does the application to Automated Testing make sense?
It should be traced back to the very rudimentary purpose of automated testing, better Return on Investment (ROI). Simply put, the ROI formula pertaining to this matter can be oversimplified and calculated as follows:
ROI of Automated Testing = Number of Iteration * Manual Testing Cost – Cost of First Time Automated Testing – Number of Updates * Cost of Updates
If the Number of Iteration and Number of Updates are approximately similar or even the same, in some cases (For example, a completely new product):
ROI of Automated Testing = Number of Iteration * (Manual Testing Cost – Cost of Updates) – Cost of First Time Automated Testing
The key takeaways from these 2 formulas:
- The ROI is directly proportional to the Number of Iteration
- The ROI can be negative when Manual Testing Cost is cheaper than actual Automated Testing
- The Cost of Automated Testing is mostly not higher the Cost of Manual Testing, but the Cost of Updates does affect the formula.
The emphasis of oversimplified doesn’t take into consideration of the matters being out of quantitative scopes, such as depletion of the iteration mandate, possibility of testing manual blind spot and so forth. As the decision makers, it’s inappropriate to chase the outright resource-saving automation or calibrate the ROI of Automated Testing. Hence, it dares the decision makers to initially answer to themselves the succinct question: is the automation necessary in the testing process? Then, it requires them to obtain a deeper understanding and better instinct in both the automated testing and the general software development as well.
4) Yes, so… when for the Automated Testing?
Although there are no silver bullets in the Testing field, the Automated Testing is the preferred option in the following scenarios when multiple tests are highly needed and improvements (rather than changes) are needed:
- Regression Testing: It is a type of software testing that ensures that previously developed and tested software still performs the same way after it is changed or interfaced with other software. Changes may include software enhancements, patches, configuration changes, etc. Here, automated testing is suitable because of frequent code changes and the ability to run the regressions in a timely manner.
- Load Testing: Load testing generally refers to the practice of modelling the expected usage of a software program by simulating multiple users accessing the program concurrently. Automated testing is also the best way to complete the testing efficiently when it comes to the Load Testing.
- Repeated Execution: Testing which requires the repeated execution of a task is best automated.
- Performance Testing: Similarly, testing which requires the simulation of thousands of concurrent users requires automation.
5) Any tools to refer to?
(Source: Best Top 10 Automation Testing Tools for 2018)
In ITware, with one of our recent successful cases, we can proudly claim that we take continuous integration and deployment seriously in order to ensure the best level of quality assurance. And Selenium out of all the options is the one we are specialized in. As the most popular open-source framework for applications, it supports multiple system environments (Windows, Mac, Linux) and browsers (Chrome, Firefox, IE, and Headless browsers). The scripts can be written in various programming languages such as Java, Groovy, Python, C#, PHP, Ruby, and Perl.
There are myriads of guidelines, booklets, tutorials, forums and videos about the far-off but enticing Automated Testing on the Internet. Hopefully, this article has shown you the discrepancies between these 2 types of Testing. As suggested by our experience Tester, the manual testing should be initiated with automated one concurrently following up. Hopefully, this article has gleaned enough information for you to grasp some insights for making a better decision between Manual Testing or Automated Testing.
Should you have any further concerns or requests, please feel free to drop us a message. Do you want to read more use case about software testing?
