A Systematic Design towards Usability for Novice Designers

Dependence on intuitive and experiences would add more complexity in the design process. Novice designers would follow the design of any existing systems without direction. This paper proposed a systematic design approach that provides guidance for novice designers in achieving the targeted usability goal using patterns, and evaluating the prototypes in achieving the determined usability goals. The proposed approach involved the cooperation of experts in providing direction to the novice designers in achieving usability goal in the system design. A collective decision both from users and experts would be gather to give a clear decision of a chosen prototype. Two experiments that followed the proposed approach were conducted with 7 groups or 33 undergraduate students. Ten industry experts, 10 field experts and 29 prospective users were involved in evaluating the appropriate use of the tool to assist them in making decision during comparative prototype evaluation. The result from both experiments concluded the systematic approach could help the novice designers to emphasize usability along the design process. It was observed that the achievement of important goals in their designed prototype could be easily done if pattern selection was provided to the novice designers. The designed tool to assist novice designers has high reliability and high acceptance level of appropriate of use in making decision for the preferred prototype.


Introduction
Experienced designers rely on the available design guidelines, past design experiences, templates and problem-solution sets from previous experiences.This concurs to the study claimed that practitioners were depended on their intuition and experiences gained [1]. Thus, novice designers would find difficulties in designing intuitively if experience and knowledge are not sufficiently gained.Subsequently, further research works have been explored on how knowledge and experience were used among experienced and novice designers such as in [2,3] and [4].Reference [4] suggested that design strategies, knowledge, and information should be included in developing support methods for novice designers as a guide to them.
Previous works in [5,6] explored the use of guidelines and patterns as an aid to teach design.The result from [5] has shown the use of design patterns displayed a greater impact on the novice designer's performance than guidelines.We considered all design related knowledge and information such as design decision and design pattern in the proposed method that leading to the development of the tool.
This research aims to understand how the proposed assistance tool and method drive the novice designers in designing a usable system, and thereby contribute towards developing a reliable tool and method for them in making design decision, particularly in determining usability goals and patterns, and selecting the best prototype.This paper was extended from [7] which included 2 experiments conducted following the proposed approach.The proposed tool was adopting the application of AHP technique in prioritizing the common usability goals and prototypes.AHP, a multi-criteria technique, has been applied in various disciplines.Recently, in engineering itself, the AHP technique was applied for the purpose of economic efficiency [8] and planning [9].Relevant to the study, it is also applied in product design such as in [10].A  ISSN: 1693-6930 TELKOMNIKA Vol.14, No. 2, June 2016 : 715 -724 716 proposed task-mapping template drives the novice designer to achieve the targeted usability goals.The design and development of the tool were to capture users' and experts' decision in selecting the best prototype, thereafter helping the novice designer to know the best choice of prototype based on the targeted usability goals.
This paper is structured as follows.First, the paper gives an overview of the proposed method and tool used in this study.This is followed by a description of the experiment and data collection methods.In the next section, findings are presented and discussed.Finally, we provide the conclusion and future work.

Research Method
In the beginning, we acquired the priority rank of the usability goal for the system design from experts.The result was served as a direction in the system design for the novice designers which assist them to determine the selection of patterns on the selected design tasks.
Prior to designing the interface, novice designers were conducting user and task analysis with the prospective users, in relating to the designed system.Based on the users analysis result, they chose the important task that the prospective users required.The designed tasks were used in their focus in applying usability patterns.This was followed by designing an interactive prototype using a prototype tool.In the evaluation stage, they were required to arrange the potential users to provide feedbacks on their design and selection of prototype from the comparative prototype evaluation.The decisions of usability goal prioritization and prototype selection were assisted by the proposed tool.
The design of the proposed tool was divided into four main modules.There were namely: usability goal prioritization, prototype selection to aggregate all the consistent decisions made by users and experts after walk-through all prototypes, the calculation to check for consistency decision and determine the most inconsistency decision in the matrix for reevaluation, and administration module to setup project description and evaluators' profile of users and experts.The system would help the designer to efficiently know the ranking of the usability goal and determine the prototype selection.The 3 main activities in the proposed approach include of prioritization usability goal, designing of user interface following predetermined usability goal, and evaluating and selecting the best prototype.
Industry experts from software development and field experts related to interaction design, usability, or interface design were invited to prioritize the four common usability goals based on a system design.The selection of the chosen usability goal for prioritization was adopted from the analysis made by [11] that showed learnability, efficiency in use, reliability in use and subjective satisfaction were the most commonly cited.The method of prioritization and selecting prototype were adapted from [12,13] called Analytical Hierarchical Process (AHP), a multi-criteria analysis approach.Further discussion on the methodology in the prioritization and prototype selection can be found in [14].The result of the prioritization was served as a goal for novice designers to target.
Following approaches in design for usability discussed in [15], we required the students or novice designers to follow user-centered design approach.The design assignment was started to understand users and tasks in the system design.The novice designers gather user information and performed the task analysis of the system.During the development of design, the novice designers started with sketches or wireframe prototype and improved their design as they went on designing the prototypes to a higher fidelity.The novice designer were provided a list of usability patterns supported with some usable design solutions.Based on the given template, the novice designers followed the selected patterns that matched the targeted usability goal set by the expert in the previous activity, for all identified important tasks.The usability patterns served as a reference and guided them to achieve the usability goal.The usability patterns were adopted from 19 user-perspective patterns of interaction patterns from [6], 21 architectural usability patterns by [16], and 10 functional usability pattern from [17], are categorized into the 4 common usability goal or usability attributes .Figure 1 shows a partial template screen of a task-pattern mapping for efficiency goal.The task-pattern mapping is to drive their design solution to meet the targeted usability goals.
A matrix of task scenario and usability goal was used to assist the novice designers to determine the possible usability goal to be designed to achieve the determined percentage of 717 usability goals.The percentage for each usability goal related pattern that adopted in the designed tasks is defined as: Where UGP is the number of patterns related to a usability goal, IUGP is the number of patterns that has inverse effect with the related usability goal, and P is total pattern applied in the designed tasks for the system.This was served as an approach for the novice designers to plan their design.Table 1 shows an example of a result of the usability goal fulfillment determined by the novice designers in a team.Novice designers would refer to the percentage weighting given by the experts via prioritizating 4 main usability goals, and the ranking of usability goal that determined the level of importance.Designers can start their work with these goals in mind and based on the decided usability pattern in all designed tasks.The novice designers were briefed on conducting usability testing.They have been practiced on conducting usability test during the pre-testing day in order to familiarize with the steps and data to collect during user observation.In order to conduct the prototype selection, invited users and experts were walk-through in all pre-determined prototype designs based on the testing scenario in the observatory usability testing.Users and experts had evaluated 2 and 4 prototypes respectively in the comparative evaluation.Quantitative measures such as time were taken to complete the tasks, errors made and comments from the evaluators were also recorded.However, based on the usability measurements, novice designers may find difficult to decide which best prototype to select for further improvement.We extended the short usability testing and walk-through design to let the evaluators judged their decision on the prototype selection using AHP.The majority of the users were not familiar putting their preferences scale in the AHP's matrix.We provided a guided dialogue for the evaluators to make their decision in a designed tool.Figure 2 shows a guided dialogue of pair-wise comparison scale for 4 prototypes in efficiency in use using the designed tool.Besides calculating for consistency ratio, the system was also checked for inconsistency ratio, which was more than 0.10.Inconsistency ratio found in a decision matrix will allow the evaluator to re-evaluate their decisions made.This is based on the discussion on the Saaty's theory on satisfying consistency matrix in [12].We adopted the theory and identified three locations on the matrix to evaluate based on the highest differences value between the user preferences scale and the satisfactory value from the consistent pairwise comparison matrix.After completing receiving all the evaluators' decision, the consistent evaluators' decision result will be aggregated and the chosen prototype will be determined based on the highest total weighted evaluation.This is achieved by multiplying the factor weight for each usability goal with factor evaluation for all prototype designs.Figure 2. A guided dialogue to assist an expert made decision on the evaluated prototype in a designed tool

Research Method
In understanding the proposed approach, we were conducting 2 experiments with private institution students who were taking the subject of designing user interface.They were students in Year 2 software engineering programme.Figure 3 gives a combination of an overall flow of the proposed approach to driving novice designers design towards usability, in both experiment designs.
In order to understand the usage of the proposed approach and tool, the students-cumnovice designers were required to follow a user-centered approach in designing user interfaces for student's organization management system based on the given general description.Data collection of the effect of the usage of the proposed approach and tool were conducted for all participants involved in the experiments.Throughout the implementation of the experiments, content analysis was conducted on design diaries and group design reflection written by the novice designers, and on prototype designs to assess the used of patterns.Design diaries and group design reflection were collected to understand the problems faced by the novice designers following the proposed approach.Furthermore, evaluation of the tool was also conducted among users and experts who involved in the comparative evaluation prototype.Table 2 shows the summary of data collection method conducted in the experiments.To evaluate the appropriateness of the proposed approach in designing the prototypes.
Prospective users and experts (field and industry experts)

Quantitative and qualitative survey
To evaluate the appropriateness of the proposed tool assisting their decisions.

Content analysis: Review the application of patterns in the prototypes
To evaluate the pattern usage in fulfilling the goal.
Two experiments were conducted in designing the user interface of different systems in different semester.In the first experiment, there were 4 groups, in which 6 members in each group, involved in the user interface design for a student community system for a university.Whereas in experiment 2, a total of 10 novice designers were grouped in 3, involved in the design of user interface for managing stationery.The difference in the number of participants involved in the experiments depends on the number of students in the second year software engineering taking the designing user interface unit in the semester.Both experiments follow the same approach and tool as described in section 2, except the decision for selecting appropriate patterns.Novice designers in experiment 1 were freely decided the appropriate patterns for the identified designed tasks and matched the targeted usability goal set from the result of usability goal prioritization.While novice designers in experiment 2 followed the predetermined patterns that deemed appropriate for the designed tasks.The patterns used in experiment 2 were decided by the researcher.

Results and Discussion
Data of seven groups of novice designers from two experiments were collected.Ten industry experts, 10 field experts, and 29 prospective users from experiment 1 or surrogate users from experiment 2 were involved in evaluating the tool.Evaluation of the appropriateness of the proposed method and tool are discussed in the following section.

Appropriateness of the Proposed Approach in Designing the Prototypes
Submissions of individual diary log and groups' design reflection were collected and each content was analyzed.Similar groups' theme that discussed and highlighted by the novice designers in the written logs and reports, was summarized in Table 3.Based on the result, the substantial problems occur were difficult in using prototyping tool and difficult in achieving targeted rank goal using the usability goal fulfillment.The difficulty in using prototyping tool occurred because novices took some time to familiar with the tool to complete their prototypes.However the problems were reducing after 2 hours hands-on and demonstration was conducted in the experiment 2 in order to familiarize the functions of the tool provided.Similarly, the problem of achieving targeted rank goal was reducing tremendously after the decision of selecting patterns was given to novice designers as a guide in the experiment 2 compare to the same problem was reported more than 30% in experiment 1.No aid was given to novice designers in experiment 1 to determine the patterns for each designed tasks.They have to decide themselves the usability pattern that would appropriate for the designed tasks.These had showed that guide to select patterns used for the designed task was needed on top of the usability goal fulfillment.However, design following usability patterns had given the novice designers an approach to trigger ideas in their design.This had been reported by 3 groups in both experiments.Guide: A: data obtained from novice designers' diaries B: data obtained from groups' design reflection reports

Appropriateness of the Proposed Tool Assists Evaluators' Making Decisions
A total of 49 returned and complete forms were used to analyze the designed tool for its appropriateness in usage and its usability.29 respondents or approximate 59% of total respondents were from university students who became the prospective users or surrogate users in evaluating the prototypes during usability testing and prototype selection process using the designed tool.The designed tool was used to assist evaluators and experts to make the decision in the comparative evaluation prototype and aggregate all consistent decisions to provide an absolute selection result, as Table 4. Approximately 20% or 10 respondents for each industry experts and field experts are involved in the evaluation.Four questions and 3 questions were designed to evaluate the appropriate use and usability of the tool respectively.Questions related to the usage of the tool comprised of 1 binary question and 3 questions of 5point Likert scale.A binary typed question was related to the matching of individual's decision with the decision suggested by the designed tool.Whereas questions in 5-point Likert scale were related to the use of the system in prototype selection decision-making, need of help in using the tool, and comfortableness to reconsider their inconsistent decisions.Result related to the usage appropriateness of the designed tool was shown in Figure 4. Three questions to assess the usability of the tool comprised of sufficient and understanding the questions asked as well as learnability of the tool.The result of the usability of the tool was shown in Figure 5.

Purpose of questions Hypothesis Questions
To assess the appropriate use -The result generated by the designed tool is matched with the result of an evaluator's decision.

Q1
-The designed tool assists evaluators to make decision in the preferred prototype.

Q2
-Decision could not make without the help of the tool.Q7 -It is not uncomfortable to reevaluate inconsistent decisions.Q6 To assess the usability of tool -The questions asked by the tool are understandable.Q3 -The questions are sufficient to determine the decisions made.Q4 -Evaluator would not need help in making decision to use the system.Q5 The designed tool has high reliability as 96% or 47 respondents stated their decision priorities were equal with the result generated by the designed tool.The designed tool has high usage appropriateness in terms of its assistance in helping evaluators to make decision based on the perception of evaluated usability goals.A total of 86% of respondents were agreed the designed tool could help them to make decision in choosing the preferred designed prototype.In contrary, 51% of the respondents agreed that they could easily know which preferred prototype to choose without the help of the tool compare to only 20% in need of help from the tool.This was because they would use their visual impression to select their preferred prototype in which this was not the purpose of the study.A lot of evaluators' commented for improvements related to the aesthetic values in the design that comprising colors, graphics, and pictures.Thus, we believe the method in the tool would assist them to judge their decision mainly based on the usability goals.More than 7% of the respondents did not satisfy with their inconsistent decision to be re-evaluated compare to only 30% respondents agreed to reevaluate their inconsistent decisions.We find that this is necessary in order to have consistent decisions to be aggregated to give a reliable absolute result in selecting the prototype during comparative evaluation.
The designed tool also has high usability in terms of understandable of the questions asked, with 92% of the total respondents, and sufficient of the questions (69% of the total respondents).Nevertheless, about 53% of the total respondents need help in using the tool.We observed that 2 experts were having difficulties to reconsider their inconsistent decisions.They informed that they did not know what needs to be changed to achieve a consistent result.This explained the high frequency of help was needed in using the tool.However, the tool provides the flexibility to change the evaluators' decisions according to the questions asked or to maintain their inconsistent decisions.Detailed assistance to help to achieve consistent decisions does not represent the decision from an evaluator.

Evaluation of the Usage of Usability Pattern
Figure 6 summarizes the usability patterns use in the design of a prototype in each novice designer's group in achieving the targeted priority of usability goal.All groups could not meet the targeted priority of the usability goal.All 4 groups have more patterns related to satisfaction than learnability.Among all groups, only one group (group 1) has achieved the highest priority in patterns relate to efficiency in use.Whilst other 3 groups focused on patterns related to reliability in use more than other goals.This result occurred as novice designers independently chose the patterns relate to their designed tasks.Improvements were made in the methodology of choosing usability patterns were conducted and evaluated in experiment 2. Selections of usability patterns related to the designed tasks were reviewed to determine its relevancy as well as in achieving the targeted usability goal.We have put emphasizing on the use of pattern for the designed tasks that would be evaluated in usability testing and comparative prototype evaluation.Table 5 shows the summarization of the planned usability pattern and the implementation of the targeted priority of usability goal in experiment 2.
Result in experiment 2 found that not all groups can completely meet each targeted percentage of pattern in their designed prototype as planned.Novice designers were new and require time to master their skills in the construction of designed prototype using the tool.Two groups, which were group 1 and 2, could place the highest priority target of reliability in use in their designed prototype with the achievement of 24 patterns and 20 patterns respectively.Whilst group 3 has 1 pattern difference between the goals of reliability in use and efficiency in use.The main finding from the improvement of the usage of usability pattern in experiment 2 was the achievement of important goals in their designed prototype could be easily done if pattern selection was provided to the novice designers rather than independently decided by them.Result from the analysis of the designed prototypes and comparison with the planned pattern found that some patterns were not designed in a prototype of a group but were designed in another prototype of another group.Inequality of the implementation of usability pattern in the designed prototype with the plan would derive some reasons.One of them was novice designers may not always refer to the patterns in the plan while implementing the prototype.Other reasons may include novice designers could not dominate the skill in using the tool to develop the prototype following the planned patterns.Novice designers in group 1 were able to realize the importance of trade-off usability goal following the targeted priority in usability goal.Some patterns that deemed relevant to the designed tasks were related to efficiency in use, reliability in use, and satisfaction.These patterns were added in their designed prototype without affecting the position of highest priority in usability goal.

Conclusion
Achieving balance of usability goal is important in the design process towards usability.Furthermore, novice designers would not follow blindly the design from any existing systems if a method of achieving usability along design process is imposed.Therefore, the systematic approach to design based on usability patterns discussed in this paper become the design rationale and guide towards achieving the targeted usability goals.AHP technique was customized in the design process to assist in collecting data prioritization of usability goals and prototypes.
The proposed method had shown a cooperation of experts in providing direction to the novice designers in achieving usability requirement in the system design.It provides a way to  ISSN: 1693-6930 TELKOMNIKA Vol.14, No. 2, June 2016 : 715 -724 724 novice designers how trade-off of usability pattern is made in fulfilling the targeted priority of usability goals.However, guidance to select appropriate pattern needs to come from industry or field expert to ensure its suitability and relevancy of chosen patterns with the designed tasks and achievement of usability goal.In deciding the best prototype in fulfilling the usability goals during the comparative evaluation process, it had shown a collective decision both from users and experts.This could give a clear decision of a particular prototype was chosen that supported with the weight of preference of each usability goals.
The result from both experiments concluded the proposed systematic approach using usability patterns to achieve usability goals could help the novice designers to emphasize usability in designing.Novice designers found that the usability patterns could easily trigger design ideas.The designed tool has high reliability due to high percentage of similarity of decision priority with the individual evaluators' decision.It also has high acceptance level of appropriate of use in decision-making for the preferred prototype and high acceptance level for usability in terms of understandable and sufficient questions.Supports from the tool to collect decision data and to guide evaluators to reassess their inconsistency decision to achieve the validity result of the selection of prototype were also included although evaluators may find the difficulty in achieving consistency in their decisions.The systematic design provides guidance for novice designers in prioritizing the usability goal, achieving the targeted usability goal using patterns, and evaluating the prototypes in achieving the usability goals.More studies will be required to determine if the approach could help in delivering good quality of prototype that has less number of iteration and less number of usability problems found.

A
Systematic Design towards Usability for Novice Designers (Hoo MeeiHao)

Figure 1 .
Figure 1.An example of a partial screen of the task-pattern mapping for efficiency

A 719 Figure 3 .
Figure 3. Overall flow of the proposed approach for user interface design

Figure 4 .
Figure 4. Survey result on appropriate use of the tool

Figure 5 .
Figure 5. Survey result on the usability of the tool

Figure 6 .
Figure 6.Percentage of usability pattern applied in prototype design for experiment 1

Table 1 .
Summary of usability goal fulfillment in design planning

Table 2 .
Summary of data collection methods ParticipantsMethod/ analysis PurposeNovice designers Content analysis: Group design reflection Individual's design diaries

Table 3 .
Number of themes discussed in the reports and logs for experiment 1 and experiment 2

Table 5 .
Total and percentage of usability pattern usage in prototype design for experiment 2