An Analysis of Students’ Errors in Completing Essay HOTS Questions Based On Watson's Criteria Viewed From The Cognitive Style Perspective

Received : 30-12-2020 Revised : 27-03-2021 Accepted : 30-03-2021 Online : 16-04-2021 This study aimed to determine the students’ errors classified as Field Independent (FI) and Field Dependent (FD) in solving essay HOTS questions based on Watson criteria and to find out the causes of errors experienced by FI and FD types of students in finishing those questions. This was qualitative research which implemented the case study method. This research was carried out at Junior High School 3 Kepanjen Malang of 2019/2020 academic year. Research subjects were determined based on The Group Embedded Figures Test (GEFT) test results. From the test, two students were selected from each type of cognitive style. The applied data collection included tests and interviews. The qualitative data analysis applied four stages: data reduction, data presentation, conclusion, and verification. The results showed that FI students experienced errors in manipulating calculation, false conclusions, incorrect use of mathematical formulas, and false calculation. On the other hand, FD students had errors in mathematical operation signs, erroneous formula writing, irresponsive to problems, errors of manipulating calculation, and not writing mathematical formulas. Keyword:

feel happier when essay questions relate to their lives or their classmates because it makes them have a natural curiosity as a trigger for learning during maths class (Noh et al., 2013).
Problem-solving is one of the teaching strategies that develop Higher Order Thinking Skills (HOTS) (Abdullah et al., 2015). HOTS will activate students' abilities when dealing with problems, uncertainties, questions, or unknown problems (Tanujaya et al., 2017). HOTS enables students to become more independent, creative, and proficient learners in solving problems and applying scientific knowledge in the everyday life context (Hugerat & Kortam, 2014). HOTS skills include creativity, critical thinking, analysis, problem-solving, and visualization (Ramos et al., 2013;Heong et al., 2011). To develop students' understanding regarding HOTS questions, students must be involved in learning tasks that encourage application, analysis, synthesis, and evaluation activities (Yen & Halili, 2015). Human thinking skills can be classified into two broad groups; LOTS and HOTS skills (Abdullah et al., 2015;Tanujaya et al., 2017). LOTS is the first three aspects of Bloom's Taxonomy, namely: remembering (C1), understanding (C2), and applying (C3), while the next three levels, namely: analyzing (C4), evaluating (C5), and creating (C6) belong to HOTS (Mardiana & Kuswanto, 2017;Tanujaya et al., 2017). Implementing HOTS in teaching and learning mathematics is essential to change the social stigma in mathematics (Abdullah et al., 2015).
Based on the observation and interview results conducted at State Junior High School 3 Kepanjen Malang, it was found that students still have difficulty completing sets of questions in the form of a short essay. From an interview with one of the mathematics teachers of State Junior High School 3 Kepanjen Malang, it was found that students still made errors when asked to work on essay exercises related to problem-solving. This is proven by the result of student tests displaying 70% of students doing their tests incorrectly when answering sets of questions in the form of essays. Therefore, it is necessary to analyze the types of errors made by students of grade VII of State Junior High School 3 Kepanjen Malang in solving essay questions and to know the cause of errors on the sets questions.
To determine the error types and their causes, conducting a more in-depth analysis of each error made by students is required. Analyzing students' errors can display the inappropriate problem-solving process and provide information about students' understanding and attitudes towards mathematical problems (Oktaviani, 2017). The implemented error analysis is based on the Watson Criteria. Watson (Patahuddin et al., 2017) highlighted the analysis to describe mathematics lessons related to mathematical problems and intellectual demands required from each exercise. Criteria implemented to analyse student errors based on Watson's criteria include: inappropriate data, inappropriate procedure, omitted data, omitted conclusion, response level conflict, undirected manipulation, skills hierarchy problem, etc. (Maharani et al., 2020;Sofiyah, 2018).
The cognitive style has been recognized as a fundamental characteristic of individual cognitive processes (Yang et al., 2013). The relationship between cognitive-style and learning achievement shows a positive and statistically significant correlation (Zamzam & Patricia, 2018). Cognitive style is the process of how students think about what they are learning (Motvaseli & Lotfizadeh, 2015;Udiyono & Yuwono, 2018). Onyekuru (2015) defines cognitive style as a relatively stable strategy, preference, and attitude determining the individual's unique way of comprehending, remembering, and solving problems. According to Perdikaris (2011), cognitive style is a potential variable that affects students' thinking in their learning and academic development. Teachers have to know their students' cognitive development as each student has different cognitive levels (Udiyono & Yuwono, 2018;Motvaseli & Lotfizadeh, 2015). When the teaching style is adjusted to the students' cognitive style, the learning and teaching process will be more productive and useful (Onyekuru, 2015).
Cognitive style is classified into two; Field Independent (FI) and Field Dependent (FD). When processing information, students with FI type are not affected by external factors, whereas students of FD are the opposite, so they sometimes have difficulty focusing (Udiyono & Yuwono, 2018). Students with FI style can understand things analytically, analyze and isolate relevant details, detect patterns, and evaluate data critically (Fatemi, 2014). On the other hand, FD students have a problem to organize new information and link it to the previous one (Udiyono & Yuwono, 2018). When studying mathematics in groups, FD students often interact with their teachers for extrinsic reinforcement (Rahman & Ahmar, 2017). Yang et al., (2013) displayed the cognitive style of FI / FD fits perfectly for learning design because it reveals how well students can restructure information based on obvious clues and external stimulation.
Some research on analyzing students' errors in finishing mathematical essay questions has been conducted before. A study on analyzing errors in solving those questions related to Watson's criteria was previously carried out (Sofiyah, 2018;Maharani et al., 2020). However, the study of error analysis in finishing mathematic essay questions in the form of HOTS connected to Watson's criteria viewed from students' cognitive style has not yet been conducted before. The difference between this research and the previous one is that this research dealt with mathematical essay questions in the form of HOTS and viewed from cognitive style.
Based on the elaboration above, the issues regarding the error types of both FI and FD students in solving HOTS essay exercises in mathematics is very interesting for researchers, so the researchers examined further about the types of errors undergone by FI and FD students in solving HOTS math story tasks based on Watson's criteria, and the factors causing those two groups of students make errors in doing the tasks. It is expected that the study outcomes can be used as a basis by the teacher in providing appropriate assistance to students of FI and FD type who still create errors in solving HOTS essay questions in a mathematics course. Also, they can be used as a reference to determine what point needs to be emphasized in teaching mathematics related to those questions. Based on the aforementioned background, then the objectives in this study are to: 1) knowing the errors made by the FI and FD type students in solving HOTS mathematics essay questions based on Watson's criteria, and 2) knowing the causes of the errors of FI and FD type students in solving HOTS mathematics essay questions based on Watson's criteria.

B. METHODS
This research applied a qualitative approach with a case study method. This research was carried out at State Junior High School 3 Kepanjen Malang in the odd semester, 2019/2020 academic year. The implemented subject selection technique was random sampling. The research subject was determined based on the results of the GEFT (Group Embedded Figure  Test) test. Based on the GEFT test result, it was obtained the types of cognitive students belonging to cognitive FI and FD. Then, the researchers selected two students from each type of cognitive style to analyze the errors and causes of those errors. So, the overall subjects were 4 students, two FD, and two FI students, respectively. Then, the researchers gave the GEFT test, who had previously been through the validation process, to 28 participants who were there. After completing the test, it was found that students who had FI cognitive style were 15, while the rest belonged to the other group. The next one was distributing HOTS questions, which had previously been through the validation process.
The data collection method applied in the research was tests and interviews. There were two types of tests implemented in this study; the GEFT test, to find out the group of cognitive style students, and a test description in the form of HOTS essay questions to collect data about students' errors in completing questions. The interview results that were used as data are statements about the things that students did in completing test questions, students' errors when finishing the exercise, and the causes of their errors. Prior to the test, the validity of the test items was tested. The aspects of items validity are material, sentence construction, language, and validity of HOTS questions. The validity of the instrument is content validity. The validators in the test items were three mathematics lecturers and two mathematics teachers in junior high school. The validity of the data in this study was carried out with a credibility test. The credibility test implemented triangulation techniques.
The applied data analysis technique is a descriptive narrative with the Miles and Huberman models. Furthermore, it employed an interactive technique which includes: (1) data collection which was in the form of test results and interviews with the research subjects, (2) data reduction, which includes activities to summarize, choose the main things, focus on the things that are crucial and discard unnecessary parts, (3) data presentation, which was intended to find the meaning of the words obtained then arranged systematically and logical, so it becomes understandable. The data presented in this study were GEFT test results, students' error description tests, interviews, data analysis results, and (4) conclusion or verification, based on data obtained from various sources. This step was still temporary, but as the collected data through the verification process increased, reliable conclusions will be achieved.

C. RESULT AND DISCUSSION 1. FI Type Students' Errors Based On Watson Criteria
There were two subjects for the FI type. They are grouped based on code, S1, and S2. The results of FI student work with sets of questions based on Watsons' criteria are as follows. a. Student 1 (S1) The following is the S1's answer when finishing the first question. Based on the students' answers above, it is clear S1 made an error in the undirected manipulation category because S1 was false in manipulating on the calculation. When simplifying between right and left side, students divide with 3, but S1 did not divide all the 3 numbers in the left, so S1 is incorrect to write what should be written. In the Venn diagram, S1 has no difficulty. The subject is appropriate to describe and include members of the set. It appears that S1 can carry out the procedure correctly because it has already obtained similar questions. S1 received the final answer that follows the steps of completing the question, but the student is careful less in creating mathematical simplification resulting in a false final answer. It is supported by the interview result with S1 as follows: P: "Why do you simplify the two segments by dividing 3? S1: "I simplified it so that it became easier to gain the x value; therefore, I divided 30 by 3".

Undirected Manipulation
The following is the S1's answer when finishing the second question. It could be seen that S1 can comprehend the issue thoroughly by writing down what is known and asked correctly. S1 also performed the set procedures well, proven by the correct result for the y value. However, S1 experienced an error categorized as an omitted conclusion, in which the drawn conclusion was not quite right. S1's conclusion in determining only 19 students are joining the Boy Scout was true. Also, it is appropriate to conclude that students who joined the Student Council (OSIS) were 5 students. Also, it is appropriate to conclude that students who did not join extracurricular were 14. However, S1 was false to conclude the number of students following the scout. In this case, S1 was not careful enough to not calculate correctly regarding the number of students joining the scouts. Instead, S1 drew a final conclusion directly. This was reinforced by the interview result saying S1 had never experienced that type of a question before, but S1 knew the steps needed to be done. S1 was wrong to draw a conclusion for the third statement because "I already got a y score. It means the scout member is 7 students, so I think the statement is wrong".
The following is the S1's answer when finishing the third question. It could be seen S1 can comprehend the issue properly. This is identifiable by S1 elaboration regarding what is known and asked correctly. However, S1 made an error in miscategorizing the procedure. Here, S1 applied the formula/model incorrectly. S1 also had difficulty describing the Venn diagram. Furthermore, S1 experienced confusion in designing a mathematical model, which results in incorrect answers. This was supported by the student interview result as follows:

Omitted Conclusion
Inappropriate Procedure P: "Do you understand the question well?" S1: "I understand what is asked in the question, but I am confused about making the model, so I make it based on my understanding." Q: "Why do you apply x and y symbols?" S1: "Because I was confused to design the models, so I symbolized them with x and y." b. Student 2 (S2) The following is the S2's answer when finishing the first question. The picture above shows S2 can comprehend the issue well. This is identifiable through S2's elaboration regarding what is known and asked correctly. However, S2 creates an error categorized as undirected manipulation. Here, S2 improperly manipulate the calculation. S2 described the question in the Venn diagram and wrote to his members correctly. S2 had also put the correct formula but made an error when simplifying between the right hand and the left-hand side. The left side is meant to write the answer correctly, S2 just wrote with. S2 also created an error on the right side, in which he was supposed to write become. Figure 4 shows the student was confused to simplify calculations, even though S2 understood the question well. This is in accordance with the interview as follows: P: "Why do you simplify the right and the left segment?" S2: "I moved the segment to get the x value, but I was confused in the right segment when 30 is divided by 5." P: "Why do you put the x value back to the mathematical model after obtaining its value?" S2: "I was actually confused about what the next step is supposed to do, so I tried to reenter the x value into the model, and it was wrong. Then, I tried to enter the x value into science and mathematics".
The following is the S2's answer when finishing the second question.
Undirected Manipulation Based on S2's answer above, it can be seen that S2 understands the problem well regarding what is known and asked correctly. However, S2 has a problem categorized as an inappropriate procedure where the student miswrites mathematical formulas. It also appears that S2 has difficulty in drawing the Venn diagram with the members inside. Although S2 can understand the problem well, S2 was confused in making mathematical models. This was in line with an interview conducted with S2: "I am confused with working on this question, so I just put the known value from the question to solve it." The following is the S2's answer when finishing the third question. S2 can understand the problem well; students can write what is known and asked correctly. However, S2 has an issue grouped as an undirected manipulation. Here, the student was wrong to calculate mathematically. Picture 6 shows S2 has difficulty in computing the number that is supposed to be subtracted is added. So, the result was wrong. This was reinforced by the interview result that S2 understood the question, but he was in a hurry and confused in working on it, so he made an error calculation.

FD Type Students' Errors Based On Watson Criteria
There are 2 students as subjects for the FD type. They are identified as S3 and S4. The results of their work with sets of questions based on Watson's categories are as follows. a. Student 3 (S3) The following is the S3's answer when finishing the first question.
Inappropriate Procedure

Undirected Manipulation
128| JTAM (Jurnal Teori dan Aplikasi Matematika)| Vol. 5, No. 1, April 2021, pp. 121-133  Based on S3's answer above, it can be seen S3 can understand the problem well, S3 can also design a mathematical model well. S3 can make a Venn diagram by deciding its members correctly. However, S3 has an incorrect procedure classified as an inappropriate procedure. Here, the student was false in using mathematical operation marks. S3 is already good at making mathematical models, but S3 is less thorough when operating it. For instance, the number should be positive, but it was written as become. S3 also seems to have an error calculation. It can be seen S3 has a confusion when working on the first question. It is supposed to be addition, but it became multiplication resulting in a wrong answer. This is supported by the results of the interview as follows: P: "Why do you multiply -2x by x?" S3: "I am confused to simplify the equations, so I multiply the number with the x variable to simplify it." P: "Why do you move 2x-5 forward and become (-2x-5)?" S3: "I moved it to simplify the x variable to make it easier to do." The following is the S3's answer when finishing the second question. The figure above shows that S3 writes what is known, but S3 does not write what is asked from the problem. S3 made a mistake categorized as an inappropriate procedure. He miswrote the formula. S3 is correct to determine the members in the Venn diagram, but S3 is confused in making mathematical models. S3 does not add 2x to the mathematical model, so the mathematical model becomes improper. This is supported by the interview result with S3 as follows: Inappropriate Procedure whether 2x is included in the model or not." For the third question, S3 creates an error by not answering/responding to the problem. S3 experienced confusion in completing the third question. He is confused to determine the model that must be applied to work on the question. Based on the interview results, it was also the first time for him to do the story question of sets. This means that he was not appropriately trained to work on HOTS type of story questions like the third question. The following is the interview result with the S3: P: "Why do you not finish the third question?" S3: "I can't do it because I'm confused about that type of question and don't know the proper steps to finish it." b. Student 4 (S4) The following is the S4's answer when finishing the first question. Based on the answer above, it is identifiable S4 can understand the question well, by writing down what is known and asked correctly. S4 can also design the Venn diagram and its members precisely. Nonetheless, S4 has an error categorized as undirected manipulation. The student did it incorrectly when manipulating calculation. S4 created an error while simplifying the right and left segments. Both the right and left segments should be written with the only word. S4 can understand the first question and can complete it well, but he is less careful because he does not re-check the answer to his answer, resulting in the wrong result. This is consistent with the interview results as follows: P: "Why do you simplify 3 times of the left segment?" S4: "I do that because I want to find out x." P: "How do you know that your answer is correct?" S4: "I do not re-check the validity of the question, but when I got the x value, I entered it into the Venn diagram." The following is the S4's answer when finishing the second question.

Undirected Manipulation
It can be identified S4 does not write what is instructed of the question, and S4 writes what has been misunderstood. It means students cannot understand the question well. S4 does what is categorized as an inappropriate procedure. Here, students did not write down the mathematical formulas/ models. Instead, he gave a final answer directly. From Figure  10, it is clear S4 is confused in determining the model that must be applied to finish the question, so S4 only writes the answer from what he has known by directly writing down 27 as the number of students who join the scout. The interview result supports that that students understand the purpose of the question, to determine whether the statement is true and false, but he is confused to get the question done.
The following is the S4's answer when finishing the second question. The figure above shows S4 can understand the question well by writing down what is known and asked correctly. Nevertheless, S4 does a false step categorized as an incorrect procedure. He does not write the mathematical formula/model. Figure 11 shows S4 experienced confusion in making the mathematical model, so he stopped working on the exercise. This is recognizable based on the results of the interview found that S4 does not comprehend the point of the question. Also, he is confused to complete the question, although he has tried to do it.
The analysis results regarding the students' errors in completing HOTS essay questions based on Watson's criteria based on the cognitive style has several different categories. Firstly, FI students in completing the question have an error categorized as an undirected manipulation. Here, the students are wrong to manipulate calculations and are also less precise in conducting calculations. Secondly, FI students also have an error classified as an omitted conclusion. It means the drawn conclusions are wrong. Lastly, FI students also experience an incorrect procedure categorized as an inappropriate procedure. They applied inaccurate mathematical formulas/models. Those findings are in line with the results of Nasution (2018) study, saying FI students have not been able to apply the model correctly to complete the given questions.

Inappropriate Procedure
FD students in doing the mathematical essay questions of HOTS type experienced what is called an inappropriate procedure. They do not apply correct mathematical operation signs, write mathematical formulas/models correctly. Also, FD students do not write mathematical formulas/models. FD students' errors are also categorized as undirected manipulation. They are less precisely to manipulate calculations. FD students also create errors other than the seven categories. Those are students who do not answer or respond to questions. That is typical of the investigation (Anwar et al., 2020), saying FD students not only experience misunderstandings of concepts but also misconceptions about prerequisite concepts, such as: making graphic sketches. This is similar to the study results of Nasution (2018), saying FD students tend not to be able to prescribe the known and requested information in the form of variables or labels (mathematical models). Students with the FD type also have difficulty to manage new information and connect it to the previous ones (Udiyono & Yuwono, 2018).
Some factors causing FI students to face errors in manipulating calculations are their lack of accuracy and confusion in simplifying calculations. Their other issue is an inaccurate inference that is drawn because they are less careful in determining a final conclusion. They think that they have obtained all the answers. FI students also made errors by implementing formulas/models incorrectly. It is caused by students' failure to comprehend the questions and confusion in mathematical models. The next error is their error as they calculate improperly. This occurs because they are in a hurry and confused when calculating. These findings are parallel to the research of Hidayat et al., (2013) saying the cause of FI students' errors is students are less careful in operating algebraic arithmetic. The other errors of students with the FI cognitive style are a lack of understanding of the basic concepts of algebra and a lack of interest in learning (Nasution, 2018).
Other factors causing FD students' errors in applying mathematical operations signs are students' confusion in using mathematical operations signs. They also create errors by miswriting mathematical formulas or models. It is due to students' confusion in creating mathematical models. Furthermore, FD students' errors are caused by not answering/responding to questions because they do not understand the point of the questions and got less training over that type of question. Besides, FD students create an error to manipulate calculations affected by students' carelessness when calculating. Lastly, FD students' error is not writing mathematical formulas/models. It is caused by students who have less understanding of the question itself and confused in finishing the questions. These findings are in line with Sulistyorini et al., (2018) research that found the causes of FD students' error are they do not understand the question thoroughly and have low skills in the previous stage.
FI and FD students have similarities when doing the exercises. It is their errors when manipulating calculations, and the other one is writing mathematical formulas/models incorrectly. Those findings are comparable to the results of Mulbar (Rahman & Ahmar, 2017). Firstly, both FD students and FI students apply formulas/models, but the process is not proper to do the exercises. Next, they have similarities in completing the questions but have different types of errors. Lastly, FI students take inaccurate conclusions and make incorrect calculations that are not experienced by FD type students. On the other hand, FD students have some errors in the mathematics operation signs, do not answer/respond to questions, and do not write mathematical formulas/models not experienced by FI students. All those findings are in line with Nasution (2018) the study showing misunderstandings between FI students and FD students are different.

D. CONCLUSION AND SUGGESTIONS
Based on previous studies and discussions, some conclusions can be drawn: firstly, FI students' errors in solving HOTS mathematical essay exercises based on Watson's criteria include manipulating calculation, inaccurate conclusions, and improper mathematical formulas/models, and incorrect calculation. Also, FD students' errors when finishing those questions based on Watson's criteria are mathematical operations signs, inaccurate mathematical formulas/models writing, not answering/responding to questions, manipulating calculations, and not writing formulas / mathematical models.
FI students' errors are caused by some factors. Those factors encompass their confusion and carelessness in manipulating calculations and making final conclusions. Additionally, the students do not understand the point of the questions, are confused to decide correct mathematical models, and carelessness when calculating. Factors that cause FD students' mistakes are: less careful to apply mathematical operations signs, their confusion to make mathematical models, understanding the points of the questions, are not getting sufficient training to complete that type of questions, and less careful to manipulate calculation.
Based on the results of this study, the suggestion for further research is to analyze the student errors in completing HOTS essay questions based on Watson or Newman Criteria based on other factors, such as factor of perspective of adversity quotient or student abilities.