Functional Suitability Measurement using Goal-Oriented Approach based on ISO/IEC 25010 for Academics Information System

— Rapid of information technology development grow a new competitive environment. Including higher education, they need to improve their service quality in order to provide education service in more competitive. One of the ways of using information technology in higher education is the used of Academic Information System (AIS). AIS was developed to achieve the goals of the learning process which is one of vision and mission organization success factor. The measurement is needed to evaluate the quality of AIS. Functionality is one of the quality factors which is measured by observing the correlation between function and functional suitability. In this study, the quality of AIS functional suitability is measured using goal-oriented approach base on ISO/IEC 25010 in the perspective of a lecturer. The strategic plan of an institution is used as a reference to measure if the system used to have meet institution goals when using this approach. The result shows that the measurement using goal-oriented approach become more objective and suitable to the need of used AIS quality improvement for the institution than the measurement with ISO/IEC 25010 only


I. INTRODUCTION
The changes of the outside environment of higher education require the higher education to rethink how the changes affect them as a social institution and how to interact with the changes (Boyce, 2002).Moreover, a higher education needs to conform academics regulations goals and information technology to improve their academics quality (Tanuwijaya & Sarno, 2010).Academic information system (AIS) is an information system that developed for higher education academic activities which help the processing of students, lecturer, class/course, and administration data (Indrayani, 2013).AIS also is known as a system that provides academic services and an electronic form that support learning-teaching process and organize the institution business processes in higher education.AIS is associated with many stakeholders and vital data management.The delivery of academic data content through the AIS needs to be packaged in the form of centering the stakeholder involved in the learning process (Indrayani, 2013).Measuring related factors collectively and comprehensively could ensure the extent to which the factors affect and contribute the quality of information system for business effectiveness (Walkin & Castleman, 2002).That's why AIS quality measurement becomes important, and especially lecturer who is related to AIS in the learning process.(Yuhana, Raharjo, & Rochimah, 2014).
The information system evaluation is an evaluation made by using the measurement of the information system quality aspect that refers to the international measurement standard.This study uses ISO/IEC 25010 as a quality model for measurement because it has a complete measurement aspect among others quality measurement standard which are reliability, functional suitability, performance efficiency, operability, transferability, compatibility, security, and maintainability.(Miguel, Mauricio, & Rodriguez, 2014) (ISO/IEC, 2012).AIS itself consists of some different modules that need a relation for each module function.So that, functionality which is one of quality factor is measurable by observing the correlation between function and functional suitability (Yuhana, Raharjo, & Rochimah, 2014).Some of AIS measurement studies had been held such as for its performance (Maria & Haryani, 2011) requirement definition (Falahah, 2013) and usability (Rochimah, Rahmani, & Yuhana, 2015), but there is no measurement related to functional suitability.Based on ISO/IEC 25010 functional suitability has 3 characteristic (ISO/IEC, 2012) that is functional appropriateness, functional accuracy, and functional suitability compliance.
Most of the software engineer believe that software measurement should be goal oriented (Latum, Soligen, Oivo, Hoist, Rombach, & Ruhe, 1998).According to (Borque & Fairley, 2014), every measurement effort should be referred to the organizational goals and done by a set of measurement requirements established for the organization and the project.This can be done by using the goal-driven measurement process which is done by 3 precepts, that is (a) measurement goals are developed from business goals, (b) evolving mental models reserve context and focus, and (c) Goal-Question-(indicator)-Measure that translates informal goals into a measurement structures that can be measured.Goal oriented measurement process begins by identifying business goals and divides to some sub-goals.In the end, measurement with supporting indicators is implemented.During the process, measurement traceability is maintained to the goals, so that data collecting and processing still with the goals (Park, Goethert, & Florac, 1996).
One of approach that used for goal-oriented measurement is Goal-Question-Metrics (GQM), an approach that identifies process metrics to identify meaningful metrics for software development processes.The Goal/Question/Metric (GQM) paradigm define as a process that helps organizations to concentrate their measurement activity based on their goals.GQM states that an organization has to define their goals before collecting the data.GQM does not define fix goals, however, it defines goals and revises them into a set of quantifiable questions that present a specific set of measures and collected data that lead to realizing the goals (Khraiwesh, 2014).Some studies had been used goal-oriented measurement for functional suitability such as a measurement on e-payment system (Cechich & Piattini, 2004) but there is no AIS functional suitability measurement yet that use goaloriented measurement especially GQM.This research will measure the functional suitability using goal-oriented approach base on ISO/IEC 25010 on AIS in the perspective of a lecturer.By using goal-oriented measurement, the results of measurement are expected to be more objective for the institution goals.

II. METHOD
This research used a quantitative approach to identify a set of sub-characteristic used for functional suitability measurement of AIS.The data is collected from the questionnaire using Likert Scale filled by the lecturer of higher education.The research will be done as research flow which is presented in Figure 1.

A. Defining a Set of Questions of Functional Suitability Based on ISO/IEC 25010
The research begins by defining a set of questions of functional suitability based on ISO/IEC25010.The set of question is used for the measurement of AIS and being compared before and after using GQM to see whether there is any difference.The attributes of functional suitability are formulated that shown in Table 1.Functional appropriateness, define as how the software product can provide a set of function that appropriates for specific tasks and objective, has 4 attributes those are functional availability, functional completeness, functional suitability to the intended use, and user satisfaction.Functional availability is defined as the availability of several functions that needed by a user on AIS used by the institution.Functional completeness is defined as how complete the features of function can be provided by the system.Functional suitability to the intended use is defined as how functions can meet the needed or the purpose of the function.User satisfaction is defined as how the function ability can satisfy the user.
Functional accuracy, defined as how the software product can give a right or specified results and the needed precision, has 3 attributes those are data accuracy, data precision, and data consistency.Data accuracy is defined as how accurate the data generated by the system to the actual data.Data precision is defined as how the data generated by the system have good precision.Data consistency is defined as how a system can generate data consistently.
Functional suitability compliance, defined as how the software product adheres the standards, regulations in law or conventions, and similar prescriptions that related to functional suitability, has 2 attributes those are functional suitability to academics regulation and functional suitability to the procedure.Functional suitability to academics regulation is defined as how the system meets the academics regulation of institution.Functional suitability to a procedure is defined as the suitability of the function used to the procedure given to the user.
After defining the attributes, a set of questions are defined for functions that AIS should have for the lecturer, there are: Class Registration, consist of the subject matter by lecturer function and student guardianship function.Class management, consist of lecturer data function and courses function.Student's grade function.and Academics information function.
A set of question that has been defined with the total number of question is 72, should be tested for the validity and reliability.Validity and reliability test is done by 15 AIS expert including IT lecturer, institution IT academics staff, and AIS developer.Data processed by Analysis Statistics Program is declared valid and reliable.

B. Defining Goal Question Metrics (GQM)
GQM definition begins by identifying organization business goals.The intended business goals in this research are institution strategic plan.The steps of defining GQM are shown in Figure 2.
Goals are defined by the strategic plan that related to academic and informatics technology for each institution.The goals should be verified by the strategic planning team of each institution.Verification is needed to make sure that the goals that will be measured suit to the needed of AIS measurement.After the goals are verified, the questions to be measured are defined and verified.In this step, the questions are developed by detailing what institution would get as their goals.The questions defined will be chosen for the ones that suit for AIS functional suitability measurement based on ISO/IEC 25010 and be verified by AIS developing team.

C. Defining Measurement
Measurement is done to a college in Lampung a college in Surabaya.Respondents consist of 30 lecturers from ABC College and XYZ College.The weighting of functional suitability sub-characteristics value is done using AHP method with the result shown in Table 2. Measurement implementation is done using rating scale between 1 to 5 for the question designed.Measurement data is collected and its quality value is calculated using formula (1).

= ∑ ( ) 100
(1) Where: After the measurement is implemented, the evaluation should be done to verify whether the result of measurement already meets the real condition of each institution.The evaluation is done by the strategic planning team and AIS developing team of each institution as an agreement of the measurement result.

III. RESULT AND DISCUSSION
The measurement is done in two colleges to see whether there is any difference on how to use the method because every institution has different goals so the mapping of set question should be different for each institution depends on their goals.The selection of institution in Lampung and Surabaya are considered to make the comparison between AIS in a small city (Lampung-ABC College) and big city (Surabaya-XYZ College).Besides that, the number of academic community and time of AIS used also become a consideration.ABC College has a shorter time of AIS used that XYZ college.The number of an academic community in ABC College also less that XYZ College.The difference between these college lead to different goals of each college and also lead to the difference on how the method will be implemented.The set of measurement question will be mapped differently depends on what's become goals of each college.When using a set of measurement questions based on ISO/IEC 25010 with the total number of question is 72 questions, the measurement doesn't consider whether the college need the function that being measured or not.But, by implement GQM method, it will be considered and it makes the total number of a question be different.After implement GQM, the total number of measurement question based on ISO/IEC 25010 in ABC College becomes 38 questions exclude the function for academic schedule and student guardianship function.While in XYZ College, the total number of measurement question becomes 53 questions exclude lecturer data function, functional suitability to the intended use and functional suitability to the procedure.
The measurement result of ABC college using ISO/IEC 25010 only shown in Table 3 is 65,54 that means the AIS used on ABC college haven't met good quality system standard (Hidayati, Sarworsi, & Ririd, 2009) that is 70.The average quality is 3,28 that means the quality has not met the standard (Jumaizi, 2015) that is 3,4.Adjustment is done for the measurement item based on ABC college strategic plan by using GQM method.The after adjustment result shown in Table 4 is 70,25 and average quality is 3,51, and meet the standard.
Measurement result on XYZ college using ISO/IEC 25010 only shown in Table 5 is 80,03 that means the AIS used on ABC college have met good quality system standard (Hidayati, Sarworsi, & Ririd, 2009) that is 70.The average quality is 3,85 that means the quality has met the standard (Jumaizi, 2015) that is 3,4.The after adjustment result by using GQM method There are quality value changes after using GQM method by using the institutional strategic plan as reference.Table 7 shows the result difference before and after using GQM method in each college.Table 7 shows that after implementing the GQM method for AIS measurement based on ISO/IEC 25010 in functional suitability compliance, the quality value of each college is increasing.The adjustment of measurement items used to the college strategic plan makes the quality value gained changes.This table also shows that the difference between each college in some way makes the measurement result becomes different.
The result is agreed upon by strategic planning team and AIS developing team of each institution.It shows that the measurement quality model used could measure quality value more objective.

IV. CONCLUSIONS
The GQM method that implemented in functional suitability measurement of AIS based on ISO/IEC 25010 is expected to make the result more objective.The result shows that by implementing the method, the quality value is increased in each college and is suited to the real condition of college based on their needs or goals.It is proven by the resulting agreement for each college.
This quality model measurement can be implemented in vary college by adjusting the measurement items to the college strategic plan to makes the result more objective and more accurate to the actual condition of institution needs.Measurement result can be used as consideration to refine or improve AIS quality used in college.
For more accurate result, an interview or qualitative questionnaire can be done for the measurement items that haven't met the standard.

TABLE 3 .
ABC COLLEGE MEASUREMENT RESULT USING ISO/IEC 25010

TABLE 5 .
XYZ COLLEGE MEASUREMENT RESULT USING ISO/IEC 25010