Very concerned about the quality of the products. This explains the worldwide availability of testers software. By providing, these people ensure its quality.
Many testers will never forget about negative testing, although not all programmers are happy about it. Such control is needed to protect against hackers, bots, Dos/DDos attacks.
What is the calling of testing specialists? They must find problems that are not visible to others. Do not delay with negative testing, otherwise you expose the system to danger.
Positive and negative testing
Let's start from the beginning. There are 2 types of control when test cases are included in testing: positive and negative. The latter has the advantage.
Positive testing is a process of checking for correct behavior according to technical requirements and documentation. Positive testing is performed to ensure that the system does exactly what is expected.
Negative testing is a process of checking for incorrect behavior. Through such testing, we can learn that the system can cope with unexpected situations.
Positive-negative testing
To perform software testing, you need to have intuition or a hunting instinct. A testing specialist is a versatile person who can perform both business analysis and testing.
Testers check whether the process is performed correctly: whether it complies with the technical requirements and test scenarios. Performing positive and negative testing separately will take longer than performing them simultaneously. This is because there are two test iterations.
After all, the closer hour X is, the faster time goes by the sooner you need to complete tasks, fix defects, apply business requirements (which may vary), and much more. Deadline is the hottest time!
The separation of negative and positive testing is simply contrary to the nature of the tester! Its task is to check the system for all possible actions of the end user.
People are generally illogical and can cause problems in software. Negative testing will help avoid problems.
In my courses for training novice testers, I suggest they write positive and negative tests for:
- The function of calculating the root in a calculator.
- Working with the cart (adding / deleting / editing) in the online store.
And that’s what I noticed - everything is fine with positive testing, the guys are coming up with ideas different types tests (because the task is to name a few, and not to list everything, all, all, therefore, even if I work in a team, I don’t have to repeat myself). But many people have problems with the negative, they ask for clarification, because “nothing comes to mind other than entering symbols into the numerical value of the number of goods in the basket and calculating the root of a negative number.”
Therefore, I decided to write an explanatory article.
Positive testing
A tester is a person who provides information about the product to the team. So we decided to make the same online store, thought through the concept, wrote the code, and now the task of testing is to tell whether everything works the way we need it.
And of course, positive tests are of great importance. If a user comes to our website and cannot put an item in the cart, then he will not care at all that when entering any special characters or SQL injections, we write beautiful error messages.
Therefore, when we are given something to test, we can happily rush to break new molds, but need to check the correct scripts first. First we satisfy loyal and competent users, and then we do the rest.
Thus, positive testing is aimed at making sure that the main functionality works. All scenarios for using our system are feasible and lead to the expected result, and not to errors.
Let's look at an example:
The main test case is to check that the root of the correct number is actually calculated.
It can be divided into the following equivalence classes:
- After calculating the root, an integer remains (root of 4 = 2)
- After calculating the root, a fractional number remains (root of 3)
Hmm, what if we not only have a fractional number after root calculations, but also before ? Can we take the root of 2.2? Positive test? Positive!
You can also divide numbers into small ones, up to 100, for example. Then take the interval from 100 to the size int and the third will be even larger, as much as fits into our calculator. 3 equivalence classes, we check one value from the interval.
Let's not forget about the boundary values, let's check 0. Positive test? But of course! The root of 0 is 0, not an error!
From the main thing, perhaps, everything.
Oh, that's where there is room for imagination!
The user can perform so many different scenarios!! But first of all, let's take the main, shortest ones. Because if they don’t work, then long chains (added - edited - deleted - added again - etc.) are definitely not worth checking. So:
Do you think it will work if it works separately? Nooooo, guys, you're testers! Never take programs at their word! Have you come up with a script? Check it out!
Moreover, such a scenario may well fail; we have already removed this product from the cart, right? So the system may well not allow us to add it again. Like “you’ve already given up, hello, I remember everything!” Is this behavior correct? No!
Is the script itself positive? Yes! Although already with notes of perversion, I must admit
Negative testing
We must remember that negative testing is no less important than positive testing. Because our users are people, not robots. And people tend to make mistakes. And we must always remember about this human factor.
If I come to the site, place an order and everything goes well, I will come there again. But if I come, place an order and accidentally make a mistake somewhere, for example, I paste a message copied from ICQ instead of a number, then I want to see a tactful remark, and not a crash of the entire system.
In general, nowadays there is usually a large selection of sites to solve a user’s problem (for example, you need to buy something). Having looked at them and realizing that the functionality he needs is everywhere, the user will choose the most beautiful and convenient site.
But, no matter how convenient such a site is, if it is not able to work under the influence of the human factor, the user will sooner or later leave. “Step left, step right - execution”, who will like that? I would like to be able to make mistakes and correct errors, and not get slapped with scary error messages all over the screen.
That's why we conduct negative testing. What is negative testing? This is entering obviously incorrect data. We enter and see how the program behaves, whether the error messages are clear...
But how to create such tests? Let's look at examples:
1. Function for calculating the root in the calculator.
The first thing that comes to mind is what happens if you calculate the root of a negative number?
But what else can you think of here?
- Root from the void - remember about boundary values, we cannot enter a string of negative length, but we can enter a boundary value (string of zero length)!
- Root of symbols - you need to check what the system will say if you enter or paste something symbolic into it. Moreover, we divide the symbols into Russian, English and special symbols!
- The root of the meaning "four" - symbols can also be divided into abracadabra and "type number". By the way, if we talk about such “type of numbers”...
- Let's try to enter a string that represents the number . And take the root from it.
Do you see? In fact, there are not so few tests! Separately, I would like to speak on the topic of “introducing very big number, as large as possible." You can try, why not? But this will have a more negative impact on the squaring scenario than on calculating the root.
In the root, you can not enter the largest possible number, but enter a number such that the root from it (fractional value) turns out to be very long and does not fit on the screen, so what will happen then, will it be cut off or broken?
2. Working with a shopping cart in an online store.
Here, again, you can find a number field and play with it, as we just did with the calculator. The “item quantity” field is very suitable here! But, on the other hand, it’s boring, such different applications and the same tests?
Remember just 2 words - different tabs !
Do you feel it? Let me explain. A negative test for deleting an item from the cart is to try to remove an already deleted item. And here are the options for how this can be done:
- Open the cart in 2 browser tabs. First we clicked “delete” in one, then in the second. That is, an attempt to delete something that you yourself have already deleted from your own trash.
- An attempt to delete a product deleted by the admin. In 1 tab under the admin we delete the product altogether, in principle, and in the other we try to remove it from the cart under the user.
And by the way, you can also try to add a product deleted by the admin or edit its quantity. And the admin may not delete the product, but move it to another category. And here nothing should break!!! If in case of deletion we should see the correct error message, then in case of transfer we can simply continue working.
What happens if the admin did not move the product in the store hierarchy (he moved it to another category, the product was originally placed incorrectly), but simply corrected it, edited the description? Nothing should break either!
And even if we don’t have a store, but something else, always think about how you can try to apply the technique of different tabs.
For example, we can create a card - a person, a building, the same book, something else... Try editing it in 2 windows. In one, change one field, save, and then in the second, change another field and also save. Or delete something, and add or change in the second window.
Always try to play with such things, it often ends disastrously for the program. And even if the team decides not to fix such defects for now as not critical, it doesn’t matter. The main thing for us is about them know ! Provide the information, and then decide what to do with it...
I would like to give one more example from real practice. Also a web interface where you can click “create” and add a new card. The user adds it, but every other time his mold falls off. Why?
They began to find out. And they understood. The user needed to create many cards at once (migration), so he clicked on “create” several times while holding down Ctrl (open in a new tab). And then I went through the tabs and created.
It would seem, where is the negative testing? After all, the user does not make contradictory changes by changing the same card. No, he creates new ones, that is, he enters information into different cards. But that’s how the system thought open window“new card” with one thing, loudly indignant at the user’s arrogant attempts to stuff one information or another into it.
So guys, go for it! Open different tabs and go ahead, look for information about how, well, how exactly does your program behave under contradictory influences on it?
PS - this is an excerpt from my book for beginning testers, written to help students at my school for testers
Come and visit us! ツ
Aaaaand... This is the last entry in the series! It is the shortest, the simplest and consists almost entirely of real stories. If possible - stupid and funny. There is even a video filmed specifically for recording right at the time of writing. Fresh, sir. Unfortunately, I didn’t think to take a screenshot of the message about the crash of the Youtube client; it would have been fine. Fell right when uploading the video that was inserted into the article. Okay, let it be my lock screen.
At the start of testing, regardless of whether it is a new project or one that should have been buried, in general it is always clear where to start. Unless, of course, by the time testing started, none of the links in the chain were working properly. Typically, testers read requirements and other documents with non-Russian names, such as “BEC”, “ESArchi” and “User Story” and figure out how to write a test case so that it checks the execution of all these documents. This is all clear on the surface and there is no point in dwelling on it. But there is also the behavior of Android itself, which sometimes not only analysts, but even architects and some developers are unaware of. And remembering that only with custom ones, quite a lot of such features pop up. And I’m not talking about stressful scenarios when there is no memory or the battery is suddenly removed (I once met a person’s indignation at the GNU/Linux terminal because it doesn’t show the password when entering, but he has a buggy keyboard and he doesn’t understand whether he’s entering the password or this is the keyboard not working again), but about the standard behavior of Android customization and even the behavior inherent in AOSP. That is, standard system behaviors that can negatively affect the product being tested. The so-called negative scenarios.
I will briefly describe some negative scenarios and try to give specific examples.
- Communication problems. The simplest example is Fly Mode. For example, the Google Keep notes application was either not tested in airplane mode, or the bugs found did not affect the release. It's very easy to reproduce the problem:
- Turn on airplane mode
- Tap on the line Take a note...
- On the screen that appears, perform the Delete action
- Enjoying frame-by-frame animation of the movement of previously saved notes
In addition to Fly Mode, there is also an unstable connection with packet loss, and a very slow connection, and closed ports, through which your application runs, and the availability Wi-Fi connections, but without Internet access.
- No access to app store. For example, to test in-app purchases, you need the assembly to be posted in a special section of the store. If it is not there, or if it is not the same version (we are talking about the version code - the internal version), then you will not test the purchase. If the user went on vacation to China, where, with a connection to Google Play everything is very sad, the license for which he paid money should not fall off.
- Application operation with limited permissions, if the Target API Level is lower than 23, that is, less than Android 6, and when the API version is 23 and higher. In the first case, the application is legacy, but permissions can still be taken away. In the second case, it will begin to receive new exceptions that it did not know before.
- Battery saving mode. The implementation of both Doze and App Standby, as well as alternative implementations from alternatively gifted manufacturers like Samsung (and STAMINA from Sony in the first version), when everything is implemented terribly wrong, but you have to live with it. It is permissible for an application not to perform checks on time, not to send statistics, not to update data. But it is not acceptable to crash, freeze, or never complete planned tasks.
- Changing date, time, time zone. People can fly on vacations and business trips to other countries where the time zone is different. If the plane crosses the 180th meridian, then the user may well end up “in yesterday” from the application’s point of view.
A true story of failure. Parental controls in KIS for Windows appeared in version 7.0 in 2006. At the same time, the product had a built-in news agent, not at all like what it is now. It was assumed that various news about threats, all sorts of “what’s new” and the like would be sent through it. There was a bug in the release version, which was already installed by users. If you set the time in Windows back to before the license began, the protection was disabled. Strictly speaking, non-administrators cannot change the time, but 10 years ago companies did not particularly monitor user rights and every accountant there was a local administrator. One of our clients set up in his small office parental control so that users cannot surf the Internet except to authorized sites. Drakonowski configured and password protected the settings. Everything worked fine until the built-in news agent received news that it was time to update to new version 7.0.1 where, among other things, a bug has been fixed due to which the protection is disabled when the time is set back in the opposite direction before the start of the license. The user read the news, was happy and turned off the protection using the proposed method. A few days later, this story from him appeared on the then popular bash.org.ru. Since then, news of this kind has no longer been received by users.
And don’t think that he doesn’t make such mistakes. Remember the story with iOS that happened this year, although only 3 months have passed since the beginning of the year ( Note: yes, this is quite an old lecture, I’ve been wanting to post it for a long time). Phones turned off if you set the time closer to the beginning of the unix time calculation. And how did Apple fix this error? They forbade changing the time further than the critical date, which was NOT a fix for the problem. The attackers began to raise their Wi-Fi points with names that are usually found in all sorts of McDonald's and through them they transmit fake time. The devices connected to such points automatically and detected NTP servers from which they requested the time. Apple just didn't make sure that iOS didn't use fake NTP servers. Thus, iOS was again rebuilt.
- Changing the system locale and interface language. The user has the right to change the system language a hundred times a day and no one can prohibit him from doing so. The tester’s task is to make sure that the product, firstly, responds correctly to this (changes the language to the desired one automatically), and secondly, does not crash at all. In addition to the locale, the user has the right to change typefaces and font sizes, choosing ones that he is comfortable reading. The application should not fall apart if the user makes reasonable changes.
- tapjaking. I mentioned this thing in the very first lecture. Let me remind you that this is an interception of taps received by application activity A, while the user was trying to get to application B. Simply, application activity A is transparent. It looks like it's not safe solution Google, but this is how applications for controlling brightness and color temperature on devices work. Users find such applications convenient, and since Android allows them to work without presence of root, this needs to be taken into account. For example, if you have an application that uses a code or, say, a picture for authorization, you must use protection against tapjacking, for example, set filterTouchesWhenObscured to true.
- Direct call to Activity. I've already said this, but let's repeat it. Activities are one of the entry points into the application. It is quite acceptable to have several different activities that can call external applications, you never know why. These will be exported activities. But it may be that to call some activity you need to pass parameters to it. A third party application will not pass them on. At best, the user will see some kind of crooked screen, at worst, your application will crash. So you shouldn’t, so to speak, show your bare butt outside unnecessarily. By default, the exported flag is set to true, and if you are sure that external applications should not call them, you should set it to false. Well, the tester must check how the application will behave if its activation is called from other applications.
- System killer. In general, it is called OOM Killer - Out Of Memory Killer. The system starts KILLING if the application with which the user is interacting at this particular moment does not have enough memory to work. Of course, the killer is not stupid, he obeys certain algorithms, choosing targets (for example, the system will easily kill the background service, but will save the foreground service until the last; the foreground service is usually the one that draws its icon in the notification area, for example - a player ), but that's the gist of it. As a rule, on modern devices OOM Killer is not very violent. Nowadays memory is set at one gigabyte and above. But this doesn't apply to games. Games are so heavy, they eat up so much memory that no matter how much you put in, it still won’t be enough. And in general, the more RAM they put into devices, the fatter the applications will be, and games will be the fattest. At the same time, they will remain just as dull and unnecessary.
The bottom line is that your product is guaranteed to fall under the OOM Killer. Your job is to make sure that nothing bad comes of this and the product rises as soon as the ssbbw application is slammed by the system (if the product requires this, of course). And the system will do this at the first opportunity; it will not allow such a ssbbw to live in the background.
Another takeaway is that your app shouldn’t be a ssbbw either. Any leaks must be detected by the developer before he writes real code. Your performance tests should definitely have test scenarios when monkey generates a ton of events. If the code is written well, the garbage collector will free up memory and the system will not kill the application process. If everything is bad and the application leaks from all the cracks, the system will shoot it. Of course, it will take off again after this and there will be no more memory, because after killing the process the garbage collector cleans up everything, but if the decoy showed that the application leaks in its test in 15 minutes, then the user will experience these leaks even later, but that’s all will show themselves equally. - Big Data. If your application works with user data, be prepared for the user to feed something very large without a second thought. For example, I, as a user, fully expect that the Youtube client will download my video, no matter how heavy this video is. I expect that the archiver will fit into any depth of the archive, which weighs 5 times more than the entire available RAM devices. This is fine. If someone tells you that “no one will ever feed such large files,” then most likely the speaker is simply not a very good developer.
- The most stupid and therefore funny situation that causes the application to malfunction, even crashing, is a simple screen rotation. How many similar falls were identified during the testing stage! Especially if some kind of pop-up appears. At pop-ups, an experienced tester immediately starts turning the phone over! It also happened that the entire team tested the product on phones alone, where screen rotation for the application was blocked. And then, when the tablets were delivered, it turned out that the applications on the tablets crashed on almost every screen. But because fragments. There were different interfaces on the screen and on the phone, and incorrect use of fragments led to a sad result.
- Double, triple taps. For some reason, some people think that no one does multiple taps on interface elements. But no! I do! And not because I’m testing, but because I may have it in my hands old phone on Android 4.0, which already barely moves, and its screen is not very responsive. It may not be clear whether there was a click or not and double taps result. Not because they are “double” (in the sense of not those that are done with an interval of less than a second), but because there were two or more of them while the application was “thinking”. For example, while it was forming a list of many elements.
- One of the convenient features of Android 6, if not tested enough, leads to terrible results. To the point that its use is explicitly prohibited in the application, which, for now, is allowed by Google. This feature - backup and restore from backup. By the way, it’s not new, backup appeared in Android 2.2, but I don’t know a single application that uses this bun.
A creation in themselves backup copy and its restoration are not scary. Problems begin if the product uses a binding to the device identifier and installation identifier. Even within the same device, this can lead to problems, but restoring from a backup is allowed by Android itself to any device with Android 6 on board: the system backs up applications from device A, and the user buys device B and restores them all on it. And these applications work simultaneously on two devices, although they have different identifiers. If this is a client-server application, where all communication is done using tokens, there are a lot of problems here.A fighting example is the cool Talon for Twitter application. I haven’t reset the device for a very long time and therefore I don’t know if the author has fixed this error. When I reported it to him, he told me why the error occurred (even though I already know why!), but did not say whether he would correct the behavior. In general, this application has a kind of installation wizard that talks about the capabilities of this Twitter client, requesting the necessary permissions along the way. Everything is clear according to Google guidelines, right from the notes. When the setup wizard was completed and the necessary permissions were received, a flag about this was raised so as not to have to go through the setup again every time. And the application was backed up along with this flag. Together with him, it was restored. Although by default for all applications of the new type (i.e. targetApi level >= 23) permissions are disabled. You launch the application, but it cannot work properly. Because there is no check for the availability of permissions, all checks remain in the master initial setup, which did not run because the flag was set to “master has already been passed.” In addition, after launch, the client did not load tweets, giving a blow to Twitter itself. Because the buried token was not valid on the new installation and it was necessary to request a new one, and this request was also made in the installation wizard at the first step!
- In Android, starting with version (if my memory serves me correctly) 2.2.1, it became possible to move some application data to the memory card. Slowly, this feature began to be killed off, until in Android 6 Google gave it a second life, significantly improving it. If the device manufacturer in its custom did not break the AOSP behavior in this situation, then as soon as Android detects the memory card, it offers a choice whether the user will sometimes pull it out or not. If the user says that he does not plan to disable it, then Android formats the card into its own file system and connects it as part of the main memory, allowing applications to be installed there. And here are a few pitfalls:
- If the application uses hardcoded paths, then everything is lost. But this is such bad taste that I hope no one does it.
- If the application asked the system for paths when it was first launched and stored them forever, then it will be exactly the same as with hard-coded ones
- As apps are updated, users will get new versions from the app store and install them on top of the existing one. Because checking application update to new version- mandatory script. IN normal situation everything should be fine, but when you have to support many specific devices with their specific behavior, the format of the settings may change. This almost never leads to crashes if the code is written more or less efficiently and handles various exceptions. But simply losing some of the settings is already bad. For example, we had a situation where users spent months creating an antispam list, blocking numbers of taxis, banks, and collection services, and then, after updating to a new version, all the lists were lost. Precisely because the format of the settings changed and it was here, precisely in this place, that the settings were not read by the new version of the product.
- In addition to updating the product to a new version, there is a rarer, but much more hardcore option - updating the firmware itself for a new version, but with a working product. I will give two examples, one of which has already been told.
- The usual Security Update for Android 5.1, which took and disabled the life-long features of the OS that the application was using
- After Android updates 4.4 on Android 5.0, paths changed installed applications. Previously, installed applications were stored in one familiar path (/data/app/com.package.name.apk). One of our products for internal security purposes checks to see which path the protected application is accessible from and whether it has changed. The update to 5.0 has arrived and the absolute paths have changed for already installed applications (data/app/com.package.name/base.apk). The product sounded the alarm that the application was compromised. Corrected, of course.
Bye bye!
When testing software product A huge number of different types of tests are used. The broadest and most detailed classification was proposed by the author of the book “Testing Dot Com” Roman Savin. He combined types of testing according to such characteristics as object, testing subject, level, testing positivity, and degree of testing automation. The classification was supplemented based on such sources as Sam Kaner's book, “Software Testing” and an online resource dedicated to testing, “About Testing - Software Testing”.
By testing object
- · Functional testing. Functional testing is one of the most commonly used types of testing today. The purpose of such testing is to determine how well the developed software meets the customer’s requirements in terms of functionality. In other words, conducting functional tests allows you to check the ability information system solve user problems.
- · Non-functional testing. Allows you to check the compliance of software properties with the specified non-functional requirements. Thus, non-functional testing is testing of all program properties that are not related to the functionality of the system. Such properties can be presented characteristics in terms of parameters such as:
- - Reliability (the ability of the system to respond to unexpected situations).
- - Performance (the ability of the system to work under heavy loads).
- - Convenience (study of user convenience with the application).
- - Scalability (the ability to scale the application both vertically and horizontally).
- - Security (research into the possibility of disruption of the application and theft of user data by attackers).
- - Portability (the ability to transfer the application to a specific set of platforms)
And many other qualities.
- · User interface testing. This is testing the correct display of user interface elements on various devices, the correctness of their response to the user performing various actions, as well as an assessment of how expected the program behaves as a whole. Such testing makes it possible to evaluate how effectively the user will be able to work with the application and how much appearance applications corresponds to approved documents created by designers. When conducting user interface testing, the tester's main task is to identify visual and structural flaws in the application's graphical interface, verify the ability and ease of navigation in the application, and ensure that the application processes input from the keyboard, mouse, and other input devices correctly. User interface testing is necessary to ensure that the interface complies with approved requirements and standards, and to ensure that the user can work with graphical interface applications.
- · Usability testing. This is a testing method that allows you to evaluate the degree of ease of use of the application, the speed of user learning when working with the program, and also how much users of the product being developed find it understandable and attractive in the context of given conditions. Such testing is necessary to ensure the most positive user experience when working with the application.
- · Security testing. Allows you to identify the main software vulnerabilities in relation to various attacks from intruders. Computer systems Quite often they are subject to cyber attacks with the aim of disrupting the operation of an information system or stealing confidential data. Security testing makes it possible to analyze the actual response and effectiveness of the defense mechanisms used in the system when attempting to penetrate. During security testing, the tester attempts to perform the same actions that a real attacker would perform. When a tester tries to hack a system, any means can be used: attacking the system using special utilities; attempts to find out logins and passwords using external means; DDOS attacks; targeted generation of errors to detect the possibility of penetration into the system during its recovery process; exploitation of known unpatched system vulnerabilities.
- · Installation testing. This term means testing the correctness of the installation (installation) of a certain software product. Such testing usually takes place in artificially created environments in order to determine the degree of readiness of the software for use. The main reasons for conducting such tests are related to the need to verify the correct behavior of a software product during automated deployment or update. Ensuring correct and stable installation of software is a very important factor when creating a software product, as it allows users to start using the product faster and with less effort, while ensuring that the product behaves equally correctly in all tested software environments.
- · Configuration testing. Configuration testing is designed to evaluate the performance of software under various system configurations. Depending on the type of software product being tested, configuration testing may serve different purposes. Usually this is either determining the optimal hardware configuration that provides sufficient performance parameters for the software to work, or checking a certain hardware configuration (or platform, which, in addition to the hardware, includes third-party software necessary for the program to run) for compatibility with the product being tested. If we are talking about client-server software, then configuration testing is carried out separately for the server and separately for the client. Usually, when testing the compatibility of a server with a certain configuration, the task is to find optimal configuration, since stability and server performance are important. While when testing a client, on the contrary, they try to identify software shortcomings in any configuration and, if possible, eliminate them.
- · Testing reliability and recovery from failures (stress testing). This type of testing is quite often carried out for software that works with valuable user data, the uninterrupted operation of which and the speed of recovery from failures are critical for the user. Failure testing and recovery tests a program's ability to quickly and successfully recover from hardware failure, network outages, or critical errors in the software itself. This makes it possible to assess the possible consequences of a failure and the time required for subsequent system recovery. Based on the data obtained during testing, the reliability of the system as a whole can be assessed, and, subject to unsatisfactory performance, appropriate measures aimed at improving recovery systems can be taken
- · Localization testing. Localization testing makes it possible to find out how well a product is adapted for the population of certain countries and how well it corresponds to its cultural characteristics. Typically, cultural and linguistic nuances are considered, namely the translation of the user interface, accompanying documentation and files into a specific language, and the correctness of the formats of currencies, numbers, times and telephone numbers is also tested.
- · Stress Testing. Load testing allows you to identify the maximum number of similar tasks that a program can perform in parallel. The most popular purpose of load testing in the context of client-server applications is to estimate the maximum number of users who can simultaneously use the application's services.
- · Stability testing. Stability testing checks the functionality of the application during long-term use under moderate loads. Depending on the type of application, certain requirements are formed for the duration of its uninterrupted operation. Stability testing seeks to identify application shortcomings such as memory leaks, the presence of pronounced load surges, and other factors that can prevent the application from working for a long period of time.
- · Volume testing. The task of volume testing is to identify the application's response and assess possible deterioration in software operation with a significant increase in the amount of data in the application database. Typically this testing includes:
- - Measuring the execution time of operations related to obtaining or changing database data at a certain request intensity.
- - Identification of the dependence of the increase in operation time on the volume of data in the database.
- - Definition maximum quantity users who are able to simultaneously work with the application without noticeable delays from the database.
- Scalability testing. This is a type of software testing designed to test a product's ability to scale up (or sometimes scale down) certain non-functional features. Some types of applications must be easy to scale and, at the same time, of course, remain operational and withstand a certain user load.
Change related testing
- · Sanity is one of the types of testing, the purpose of which is to prove the performance of a specific function or module in accordance with the technical requirements stated by the customer. Sanitary testing is quite often used to test some part of a program or application when certain changes are made to it due to environmental factors. This type testing is usually performed manually.
- · Smoke testing is a short series of tests, the purpose of which is to confirm that the installed application is running and performs the functions after the new or edited code has been built. Upon completion of testing of the most important application segments, objective information is provided about the presence or absence of defects in the operation of the tested segments. Based on the results of smoke testing, a decision is made whether to send the application for improvement or whether it needs to be fully tested.
- · Regression testing - testing aimed at detecting errors in already tested areas. Regression testing checks the product for errors that may have appeared as a result of adding a new section of the program or fixing other errors. The purpose of this type of testing is to make sure that updating the build or fixing errors does not result in new bugs.
By testing level
- · Unit testing (Unit tests). It consists of checking each individual module (an original element of the system) by running automated tests in an artificial environment. Implementations of such tests often use various stubs and drivers to simulate the operation of a real system. Automated unit testing is the very first opportunity to run and check the source code. Creating Unit tests for all system modules allows you to very quickly identify errors in the code that may appear during development.
- · Integration testing. This is testing individual system modules for correct interaction. The main goal of integration testing is to find defects and identify incorrect behavior associated with errors in the interpretation or implementation of interaction between modules.
- · System testing. This is testing the program as a whole; such testing verifies that the program meets the stated requirements.
- · Acceptance testing. This is a comprehensive test that determines the actual level of readiness of the system for use by end users. Testing is carried out based on a set of test scenarios covering the main business operations of the system.
By code execution
- · Static testing. This is the identification of artifacts that appear during the development of a software product through analysis source files such as documentation or program code. Such testing is carried out without directly running the code; the quality of the source files and their compliance with the requirements are assessed manually or using auxiliary tools. Static testing should be done before dynamic testing, thus errors found during the static testing phase will be less expensive. From point of view source code, static testing is expressed in code revision. Usually code revision separate files is carried out after each change of these files by the programmer, but the revision itself can be carried out either by another programmer, or by the lead developer, or by an individual employee engaged in code revision. The use of static testing makes it possible to maintain software quality at all stages of development and reduces product development time.
- · Dynamic testing. Unlike static testing, this type of testing involves running the application's source code. Thus, dynamic testing contains many other types of testing that have already been described above. Dynamic testing allows you to identify errors in the behavior of a program by analyzing the results of its execution. It turns out that almost all existing types of testing correspond to the class of dynamic testing.
By testing subject
- · Alpha testing. This testing is performed on the earliest versions of the computer software (or hardware device). Alpha testing is almost always carried out by the software developers themselves. During the alpha testing process, application developers find and correct errors and problems in the program. Usually, during Alpha testing, there is an imitation of working with the program by full-time developers, less often it occurs real work both potential users and customers with the product. As a rule, alpha testing is carried out at a very early stage of software development, but in some cases it can be used for a completed or near-complete product, for example, as acceptance testing.
- · Beta testing. Testing a product still under development. In beta testing, this product is made available to a small number of users in order to study and report emerging issues that users encounter. Such testing is necessary to find errors that developers may have missed. Typically, beta testing is conducted in two phases: closed beta testing and open beta testing. A closed beta test is testing on a strictly limited circle of selected users. Such users can be acquaintances of the developers, or their colleagues who are not directly related to the development of the product being tested. Open beta testing consists of creating and posting open access public beta. IN in this case any user can act as a beta tester. Feedback from such beta testers is carried out using reviews on the website and analytics and logging systems built into the program user actions These systems are needed to analyze user behavior and detect the difficulties and errors they encounter.
According to the positivity of the scenario
- · Positive testing. Tests with a positive scenario check the ability of a program to perform its intended functionality. As a rule, for such testing, test scenarios are developed, the execution of which, under normal operating conditions for the software, should not cause any difficulties.
- · Negative testing. Negative software testing occurs on scenarios that correspond to abnormal program behavior. Such tests check the correct operation of the program in emergency situations. This allows you to ensure that the program produces correct error messages, does not corrupt user data, and generally behaves correctly in situations in which the product's normal behavior is not intended. The main goal of negative testing is to check the system’s resistance to various influences, the ability to correctly validate input data and handle exceptional situations that arise both in the software algorithms themselves and in the business logic.
By degree of automation
- · Manual testing. Manual testing is carried out without the use of additional software, it allows you to test a program or website by simulating user actions. In this model, the tester acts as a user, following specific scripts while simultaneously analyzing the program's output and overall behavior.
- · Automated testing. Such testing allows, through the use of additional software for test automation, to significantly speed up the testing process. Such additional software allows you to monitor and manage the execution of tests and compare the expected and actual results of the program. Will be discussed in more detail later.