The Testing Approach of The Multi-tenancy Architecture
There is no doubt that knowing how the application was written by the developer when testing an application provides great benefits, but this is not mandatory for every type of test. On the other hand, a good understanding of how an application works and knowing the system architecture are indispensable for frontend testing and backend testing. The first step of successful test plans is always to analyze how the application works and to understand the functions expected from the application. A software testing process that starts without analyzing the system architecture is similar to the story of the blind men and elephant, that is, it is nothing but a waste of time and money.
In this article, I will examine the architecture of Multi-tenancy applications, which are becoming more and more popular to use, and mention the important points in the process of testing these applications.
To understand the architecture of multi-tenancy applications, I think the best starting point is SaaS.
What is SaaS?
SaaS stands for Software as a Service is defined as the capability provided to the user to use the provider’s applications running on a cloud infrastructure. It can be thought of as a way of delivering applications over the Internet. Basically, the applications are accessible from various client devices through either a thin client interface, such as a web browser or a program interface. The user does not manage the basic cloud infrastructure, including network, servers, operating systems, storage, and even individual application features, except for limited user-specific application configuration settings [5].
These are several popular examples of SaaS, including Microsoft Office 365, Dropbox, Google GSuite (Apps), Cisco WebEx, and GoToMeeting.
So what is the relationship between Saas applications and multi-tenant architecture?
One of the important features of SaaS is the ability to share computing resources in offering a software product to different customers. To benefit from this ability, the architecture of the SaaS application should provide to the sharing of software instances and databases. A popular way of achieving this is known as Multi-Tenancy architecture [2].
What is multi-tenancy architecture (MTA)?
Although a lot of work has been done about SaaS applications, as an organizational approach for SaaS applications MTA is a new concept that first came to light around 2005 [1] and according to Paul Bezemer and Andy Zaidman, MTA defines as following [1]:
Definition 1. A multi-tenant application lets customers (tenants) share the same hardware resources, by offering them one shared application and database instance while allowing them to configure the application to fit their needs as if it runs on a dedicated environment.
Definition 2. A tenant is an organizational entity that rents a multi-tenant SaaS solution. Typically, a tenant groups a number of users, which are the stakeholders in the organization.
Basically, Multitenancy is an architecture where multiple tenants share the same physical resources (such as VMs or storage) of the single platform but, each tenant gets its own logical isolated instance of the application.
The major advantages of multi-tenant architecture in SaaS applications
It is clear that the multi-tenant architecture plays a vital role in developing SaaS applications and many advantages can be said for multi-tanning architecture, but I think the three points I mentioned below are the three most important advantages of this architecture.
· Provides low investment costs
· The implementation is so easy for a new customer
· Maximizing Resource Usage
The Test approach for multi-tenancy architecture
In The Art of Software Testing book, Myers and Sandler say that the testing process consists of the following steps [3]:
1- Planning and designing of the test case,
2- execution of the tests,
3- analysis of the results.
I would especially like to emphasize the planning and design of the test cases.
During the planning phase, we have to ask ourselves the question: What tests should I perform on an MTA SaaS application? When the following blueprint (figure 1) of the multi-tenancy architecture is examined We can see that multi-tenant SaaS systems have certain similarities with other models. Therefore both, Functional and non-functional testing can be applied to an MTA SaaS application. Functional testing includes exploratory testing, end to end testing, automated regression testing, data integration, and data migration testing. On the other hand, non-functional testing includes security testing and of course performance testing.
We know what tests we need to do, but how do we design Test scenarios?
In this case, my general approach is to focus on the limits and challenges of the application I tested.
The testing for challenges of MTA
1. If a tenant clogs resources, the performance of all other tenants may be compromised. So we must make sure that all tenants are able to consume these resources properly. In this situation, this problem is solved by assigning an equal amount of resources to each tenant [4]. We have to test it.
2. In a multi-tenant environment, a security breach could compromise all other tenants’ data. For this reason, the safety of the system should be tested periodically.
3. In a multi-tenant application, customization must be made possible for each tenant. In order to enable multi-tenancy and for the different user-experience, we have to test wheater the application has at least the following types of configuration [1]:
Layout Style: The layout style configuration component allows the use of tenant-specific themes and styles.
General configuration: The general configuration component allows the specification of tenant-specific configuration, such as encryption key settings and personal profile details.
4. In a multi-tenant application, since all tenants use the same database, data isolation so important. We should carefully consider as much as possible in all parts of architecture design, from both non-functional and functional levels, such as security, performance, availability, administration, etc. In other words, it is necessary to make sure that all tenants can only access their own data. Therefore, we must test the data isolation by creating a new tenant in the database.
Despite the popularity of the MTA model, not much research has been done on the evaluation of software testing, especially in cloud computing. As far as I see the test approach of MTA applications is built on traditional software testing. Therefore, MTA SaaS applications will rise with the classical test approach until science says otherwise.
REFERENCES
[1] Bezemer, C.-P., Zaidman, A., 2010. Multi-tenant saas applications: maintenance dream or nightmare? In: Proceedings of the Joint ERCIM Workshop on Software Evolution (EVOL) and International Workshop on Principles of Software Evolution (IWPSE). ACM, pp. 88–92.
[2] Kabbedijk, J., Bezemer, C.-P., Jansen, S., and Zaidman, A. (2015). Defining multi-tenancy: A systematic mapping study on the academic and the industrial perspective. Journal of Systems and Software, 100(0):139 –148.
[3] Myers, G. J., and Sandler, C. (2004). The Art of Software Testing. John Wiley & Sons.
[4] Xin Hui Li, Tiancheng Liu, Ying Li, and Ying Chen. SPIN: Service performance isolation infrastructure in a multi-tenancy environment. In Proc. Int. Conf. on Service-Oriented Computing (ICSOC), volume 5364 of LNCS, pages 649–663, 2008.
[5] Vashistha, A., and Ahmed, P. (2012). Saas multi-tenancy isolation testing challenges and issues. International Journal of Soft Computing and Engineering. CLOSER 2018–8th International Conference on Cloud Computing and Services Science.
