The Effect of DevOps Implementation on Teamwork Quality in Software Development

Background: The Agile method, which is claimed to reduce time needed for software development cycle has been widely used. It addresses communication gaps between customers and developers. Today, the DevOps has been extended as part of the Agile process to address communication gaps between developer’s team members. Despite the rising popularity, the effect of DevOps implementation on the teamwork quality in software development is still unknown. Objective: The objective of this research is to conduct a study on the impact of DevOps on teamwork quality. Two software houses, PT X and PT Y, are chosen as the case studies. Methods: This research uses quantitative methods to analyse research data using simple linear regression. The questionnaire technique is used to retrieve respondent data using 62 questions, consisting of 20 DevOps questions from 4 indicators and 42 teamwork quality questions from 6 indicators. Results: The results from various quality tests indicate that all instruments are valid and reliable while hypothesis tests showed that the DevOps implementation variable has an influence on the teamwork quality variable by 75.6%. Conclusion: It can be concluded that the implementation of the DevOps in software development has a positive correlation with the teamwork quality.


I. INTRODUCTION
Nowadays, the number of software houses continues to grow. Although there is no definite data, it is predicted that the number has reached more than 500 companies in Indonesia only [1]. It shows that many companies or organisations need more and more information technology support from the software industry in enabling their business activities. Therefore, there is a need for more convenient methods to increase the effectiveness of the software development activities [2]. This is not only to maintain the quality of software products [3] but also to increase the quality of software processes themselves.
Software development methods play an important role in the development cycle. In a conventional development like the Waterfall model, the communication between customers or stakeholders and the software house tends to be poor and it leads to a longer cycle [4]. To solve the problems and to accelerate the development speed, a new method was created, namely Agile which means 'to move fast and easily'. The Agile method enhances the interaction between customers or stakeholders and developers. Some of the approaches that implement the Agile method include Scrum, Extreme Programming (XP), Lean, and Kanban [5]. The agile method encourages a transition from formal communications to frequent communications. In Scrum, for example, the communication gap is mitigated by conducting daily scrum (stand up meeting), sprint planning, sprint review, and sprint retrospective [6].
Although the Agile method can reduce the time of a software development cycle, there is still a gap between team members that must be addressed [7]. The development team focuses mainly on producing new features and ensures II. METHODS It was found in [23] that DevOps had a positive effect on the teamwork quality, agility to adopt new technologies, and responsiveness to business needs. A positive correlation was also found in [24] between DevOps and software quality. Therefore, we the hypothesis is: H1 There is a positive effect of the DevOps implementation on the teamwork quality in software development. The research model is shown in Fig. 1.
In the DevOps implementation, we determine four indicators as defined in [25] namely automation, collaboration culture, measurement, and sharing. DevOps plays an important role in controlling automation from the development stage to production deployment, as well as in saving time. It also prevents defects and creates consistency. Furthermore, it creates good collaboration culture since DevOps prepares the environment for interaction and allows the team to accept changes. As DevOps enables continuous delivery and deployment, a continuous improvement is needed. These things are measured with a set of key performance indicators which should be open, transparent, easily accessible, able to be processed and visualised. By implementing DevOps, each team member can also share knowledge, code, and documentation to collaborate more quickly and easily and to reduce repetitive work. In the teamwork quality, we determine six indicators as defined in [19] namely communication, coordination, even contribution among members, mutual support, effort, and cohesion. DevOps underscores the importance of openness and communication in sharing information among members, as well as coordination. Each team member should contribute relevant knowledge and experience to the work whenever relevant. Mutual support is needed and team members should be willing to help each other in carrying out their duties in the project. Moreover, each team member shares the workload and prioritises team tasks over other personal tasks. Thus, uniform effort or shared effort is also one of the main goals in a high-quality collaboration. Meanwhile, cohesion is to what extent an individual wish to stay in the team. If team members do not feel a sense of belonging, or if there is low motivation to keep the team running, then high-quality teamwork is difficult to achieve.
We designed data collection instruments that reflect the indicators as defined above by using questionnaire technique with Likert scale 1 to 5, ranging from (1) disagree, (2) disagree, (3) neutral, (4) agree, (5) strongly agree, to retrieve respondent data using 62 questions, comprising 20 DevOps implementation (PD) questions and 42 teamwork quality (TW) questions. The questionnaires were distributed online to the respondents, who were selected by using the purposive sampling technique, namely the determination of the sample using certain considerations [26]. Quality tests like validity tests, reliability tests, normality tests, and others were performed to check whether the questionnaires are valid or reliable enough. Meanwhile, the hypothesis tests are performed to verify whether the hypothesis is accepted or not.

III. RESULTS
The descriptive statistics of the respondents in terms of genders, ages, and experiences are shown in Fig. 2  The results of validity tests for the instruments of teamwork quality and DevOps implementation are given in Table  1 and Table 2 respectively. A Pearson correlation's value higher than 0.25 indicates that the data collection instrument is valid [27]. Thus, based on the results, all questionnaires are valid. We also performed a reliability test for more quality tests. The reliability test results are given in Table 3. We use Cronbach's alpha as the reliability measure, which is a statistic for internal-consistency reliability. The results show that all values are higher than 0.9, while a Cronbach's alpha higher than 0.5 is considered as an adequate value [27].
Furthermore, the test results can also be strengthened by looking at the normality test using the Kolmogorov-Smirnov [27]. The test result gives a significant value of 0.441. The residual regression model is normally distributed if the data has a significance value of more than 0.05 [27]. Moreover, the heteroscedasticity test is carried out to determine whether there is an inequality of a variant from one residual to another in the regression model. The test result gives a significant value of 0.083. The model is free of heteroscedasticity if the significant value is greater than 0.05 [27].
To determine whether the regression model correlates with the independent variables, the multicollinearity test is conducted. It is carried out by looking at the variance inflation factor (VIF) value and the tolerance value. The VIF is calculated by dividing the variance in a model with numerous terms by the variance of a model with only one term [28]. It evaluates the severity of multicollinearity which provides an index that measures the variance increment of a regression coefficient due to collinearity. Meanwhile, the tolerance is the inverse of the VIF. Multicollinearity happens when at least two indicators are correlated in the model and creates redundant information about the response [29]. The data does not contain multicollinearity if it has a tolerance value of more than 0.1 and a VIF value of less than 10 [27] where the test results yield a value of 1.000 for both indicators. The summary of the test results is given in Table  4.  We conducted two hypothesis tests, namely the determination coefficient test and the partial test for analysing the acquired data. The coefficient of determination test is carried out to determine the size of the independent variable to explain the dependent variable [28]. The test result gives the R Square value of 0.756. This means that the DevOps implementation variable has an influence on the teamwork quality variable by 75.6%. The remaining 24.4% is influenced by other variables outside this regression equation or variables that are not included in this study. Furthermore, a partial test is performed to find out whether the independent variable can influence the variation of the dependent variable. The results of this test can be obtained from the significant value and the unstandardized coefficients B value [29] where the hypothesis is supported if the significant value is less than 0.05 and the unstandardized coefficients B value is in line with the hypothesis that has been made. Since the results of significant value is 0.000 and the unstandardized coefficients B value is 2.123, which has a positive sign, then the null hypothesis is rejected.
The results show that the implementation of DevOps has a positive effect on teamwork quality in software development. In other words, the more precise the implementation of DevOps on a team, the better the impact it will have on the quality of teamwork in software development. Since the teamwork quality has a very positive effect on team performance [30] [31], then it more likely brings success to the software development project. It not only increases team productivity but also improves coordination and decision making so that it can be positively associated with team effectiveness [32].
The results of this research are supported in [33], which stated that the use of a collaborative system, like DevOps in the context of this study, influences teamwork quality and performance for tasks that are supported by the system. It is also interesting to mention that the project teams which are dominated by younger people could affect the teamwork quality since they are considered as digital native generations and more familiar with technologies [34], such as the collaborative DevOps system, in this context. This result also supports previous studies that claimed there is a positive correlation between DevOps and software quality [22] [24]. Since DevOps has a positive correlation with teamwork quality, then the teamwork quality could also have a positive correlation with software quality. Although, this hypothesis should be studied and tested in the future.
Regarding threats to validity, there could be other variables that are more suitable for this study. Moreover, the hypothesis in this research was also simplified since the study was limited in a such timeframe. As another limitation, the questionnaire was made by the authors only based on the literature in the references. It could be improved if other subject matter experts were involved in creating the questionnaire. Also, it should have been tested and validated before distributing it to the respondents. It is necessary to make sure that all questions in the questionnaire are wellunderstood by respondents. Furthermore, only two organisations and 57 respondents were selected as the samples, which prevent the generalisations of the results of this study.

V. CONCLUSIONS
A study to determine the effect of the DevOps implementation on teamwork quality has been conducted. This research involved 57 respondents spread across two software houses that have implemented DevOps in their software development activities. Based on the results, it can be concluded that the implementation of DevOps has a positive effect on the teamwork quality in software development. Thus, it is highly recommended for organisations to implement and adapt DevOps in their software development activities so that the teamwork quality can be maximised.
Further research is expected to explore more hypotheses and expand the research objects in order to provide more accurate results. Moreover, other data collection methods other than questionnaires such as direct interviews should be performed in the future to increase the objectivity level of the respondents. Furthermore, it is also expected that future studies are able to increase the size of the respondent samples and obtain more data in order to acquire better results that are closer to general conclusions. Funding: This research received no specific grant from any funding agency.
Conflicts of Interest: The authors declare no conflict of interest.