Using Problem-Based Learning to Enhance Mathematical Abilities of Primary School Students: A Systematic Review and Meta-Analysis

Suparman1, Maximus Tamur2, Yunita3, Tommy Tanu Wijaya4, Syaharuddin5 1,3Departement of Mathematics Education, Universitas Pendidikan Indonesia, Indonesia 2Depart. of Mathematics Education, Universitas Katolik Indonesia Santu Paulus Ruteng, Indonesia 4Department of Mathematics and Statistics, Guangxi Normal University, China 5Departement of Mathematics Education, Universitas Muhammadiyah Mataram, Indonesia 1arman95@upi.edu, 2maximustamur@unikastpaulus.ac.id, 3yunita1996nita@upi.edu, 4tanuwijayat@gmail.com, 5syaharuddin.ntb@gmail.com

characteristics such as sample and publication characteristics is indispensable in analyzing heterogeneity of effect size data. However, several meta-analysis studies do not investigate sample characteristics (Puyada & Putra, 2018). Likewise, several meta-analysis studies do not investigate publication characteristics (Batdi, 2014b;Dochy et al., 2003;Gijbels et al., 2005;Kadir et al., 2013). Thus, the study characteristics can cause the effect size data to be heterogeneous but not investigated and analyzed. Therefore, this meta-analysis study investigates some of the characteristics of sample and publication. Also, previous metaanalysis research on the effect of PBL at primary school focuses on mathematical critical thinking abilities (Anugraheni, 2018). However, this study focuses on mathematical problemsolving, creative thinking, communication, literacy, conceptual understanding abilities. Thus, this study summarizes, estimates, and evaluates PBL implementation effect in enhancing the MA of primary school students by integrating various relevant primary studies and investigates the study characteristics that are likely to cause heterogenous effect size data.

B. METHODS
The method employed in this study was a systematic review and meta-analysis. The choice of a systematic review as a method in this study was due to the systematic review aimed to find and synthesize various relevant study results (Purssell & Mccrae, 2020). The systematic review and meta-analysis collaboration in this study was due to this study synthesized various relevant primary studies using a quantitative approach. In this literature, (Bernard et al., 2014;Borenstein et al., 2009) revealed that as a method, systematic review and metaanalysis had several stages, namely: (1) defining the problem, (2) inclusion criteria, (3) literature search strategy, (4) study selection, (5) data extraction, (6) statistical analysis, and (7) interpretation and report. Therefore, these stages were used in systematic review and meta-analysis studies.

Inclusion Criteria
The problem of the inconsistency of the effect of PBL implementation in enhancing MA was still very general so it needed to be limited by inclusion criteria so that this systematic review and meta-analysis was more focused and specific. In their literature, (Liberati et al., 2009) stated that the PICOS approach (Population, Interventions, Comparator, Outcomes, and Study Design) could be used to define inclusion criteria more specifically. Based on this approach, the inclusion criteria in this study were determined based on the PICOS approach, namely: a. The population in the primary study was primary school students in Indonesia or several other countries. b. The intervention or treatment in the primary study was the implementation of PBL. c. The comparator of the intervention in the primary study was the implementation of conventional learning. d. The outputs in the primary study were MPSA, MCTA, MCA, MLA, and MCUA. e. The type of research in the primary study was a quasi-experimental research with a causal-comparative type. f. The primary study reported statistical data such as mean, standard deviation, sample size, t-value, and p-value in both the intervention and comparison groups. g. The primary study was published in the period 2011 -2020 in national and international journals indexed by Scopus or not indexed by Scopus. Primary studies that did not correspond to the study selection process's inclusion criteria were excluded from this systematic review and meta-analysis study.

Literature Search Strategy
Primary studies were searched using several databases such as semantic scholar, education resources information center (ERIC), directory open-access journal (DOAJ), and IOP science. Through this database, primary studies were traced using the keywords "problembased learning" AND "mathematical problem-solving ability" OR "mathematical creative thinking skills" OR "mathematical communication ability" OR "mathematical literacy ability" OR "mathematical conceptual understanding ability". Thus, these databases and keywords could assist in finding and obtaining various primary studies that matched the inclusion criteria.

Study Selection
The selection of primary studies was guided by the inclusion criteria that had been established. To achieve a high-quality systematic review and meta-analysis, Pigott & Polanin (2020) suggested that the primary study selection process went through four stages guided by PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analysis), namely: (1) identification, (2) screening, (3) eligibility, and (4) included. Thus, this systematic review and meta-analysis study used these stages in selecting studies.

Data Extraction
Primary studies that had met the inclusion criteria and gone through the study selection stage were extracted or coded into key data or information that would be used in the metaanalysis process. The authors' data or information included statistical data (mean, standard deviation, sample size, t-value, and p-value), MA type, sampling technique, publication year, and Scopus indexed status. Two coders who were experts in systematic review and metaanalysis study were involved in the data extraction process to ensure that the data or information generated from the extraction process was valid and credible (Vevea et al., 2019). Coding reliability tests were carried out to produce valid and credible data or information from the data extraction process (Üstün & Eryilmaz, 2014), specifically related to consistency between coders (inter-rater reliability). The Cohen Kappa test was selected to measure consistency between coders in this study because the data extraction process only involved two coders (Vevea et al., 2019). The agreement level value between coders obtained was interpreted using the Kappa coefficient classification (McHugh, 2012). Kappa coefficient calculations were carried out with the help of SPSS software version 16. The calculation of Cohen's kappa is done according to the following formula: Pr (a) represented an observed agreement, and Pr (e) represented a coincidence agreement. A deal level of 0.85 or greater was predefined to be considered high. Thus there was a substantial to the almost perfect match between coders.
Statistical data in primary studies such as mean, standard deviation, sample size, t-value, and p-value were prone to bias. Every publication of the study results was never free from publication bias. To ensure that each primary study's statistical data was valid, publication bias analysis and sensitivity analysis were critical (Bernard et al., 2014;Furuya-Kanamori & Doi, 2020). This meta-analysis study, examining the impact of publication bias, used a funnel plot, and was assisted by the fill and trim test or the Rosenthal fail-safe N test (Harwell, 2020). Besides, the effect size data's stability and normality were investigated through a sensitivity analysis using the "One study removed" tool in the CMA software (Bernard et al., 2014).
There were two types of effect models used in meta-analysis studies, namely the fixed effect model and the random effect model (Borenstein et al., 2009;Mike & Cheung, 2015). The p-value of the Q Cochran statistic and the heterogeneity analysis's inconsistency value was used to justify the selected effect model in the meta-analysis process and the heterogeneity of the effect size data (Higgins et al., 2003). If the results of the heterogeneity analysis showed that the effect size data were heterogeneous, the analysis of study characteristics or moderating variables was carried out to investigate more deeply about the variables that had the opportunity to cause the heterogeneous effect size data (Borenstein et al., 2009;Siddiq & Scherer, 2019). Besides, in this study, the p-value of Z statistics in the null hypothesis analysis was used to justify the significant effect of PBL implementation in enhancing the MA of primary school students (Borenstein et al., 2009). All calculations and statistical analysis in this systematic review and meta-analysis used Microsoft Excel, SPSS, and CMA software.

C. RESULT AND DISCUSSION 1. The Search Result
The study selection process in this systematic review and meta-analysis study is presented in Figure 1. Through the semantic scholar (147 primary studies), ERIC (78 primary studies), DOAJ (83 primary studies), and IOP science (7 primary studies) databases, 315 abstracts were identified from primary studies' search results. However, at the screening stage, it was found that 135 primary studies were the same, so they were not included in the later stages. Then, 93 studies out of the remaining 180 primary studies could not be included in the later stages because the 93 primary studies were not relevant to the title or abstract based on the screening results. Then, 62 primary studies had to be excluded at the eligibility stage because 48 primary studies were incomplete in reporting statistical data, and fourteen primary studies had no comparison group. Thus, only 25 primary studies met the inclusion criteria. However, during the statistical data validity process, it was found that the nine primary studies tended to have a high risk of publication bias, so that nine primary studies were excluded from the meta-analysis process. Thus, only sixteen primary studies were included in the meta-analysis process.

Data Extraction Result
The result of data extraction from the sixteen primary studies is presented in Table 1. The consistency between coding using the Cohen Kappa test showed that the Kappa coefficient value was 0,88. These findings indicate that significantly the level of agreement between the two coders in the data extraction process is classified on a substantial agreement. Therefore, these findings interpret that the data or information generated from the extraction process is valid and credible.

Analysis of Publication Bias and Sensitivity
The distribution of effect size data from the sixteen primary studies involved in this metaanalysis study is presented in Figure 2. The funnel plot analysis in Figure 2 shows that the distribution of effect size data from the sixteen primary studies in this meta-analysis study was symmetric. The fill and trim test results showed that no effect size data needed to be added or excluded in this meta-analysis study. It indicates that the distribution of effect size data on the funnel plots is symmetric. The result of the fill and trim test calculation is presented in Table 2. Rosenthal's fail-safe N test results showed that the Rosenthal's fail-safe N value was 1.732, and the p-value was 0,000. This finding indicates that this meta-analysis requires 1.732 null effect studies so that the p-value exceeds α = 0,05. These findings interpret that the effect size data included in this meta-analysis study are resistant to publication bias. The various publication bias analyses conducted provide strong evidence that the sixteen primary studies' effect size data have a small risk of publication bias. The result of Rosenthal's fail-safe N test is presented in Table 3. Sensitivity analysis was used to identify potential sources of abnormal effect size data sets. Table 6 shows that based on the random effect model, this meta-analysis study's overall effect size was g = 1,050. Using the "One study removed" tool in the CMA software, based on the random effect model, the highest mean was g = 1,143, and the lowest mean was g = 0,964. It indicates that the effect size data set in this meta-analysis study is still stable and reasonable despite excluding one or more effect size data. Thus, these findings indicate that the effect size data included in this meta-analysis study are not sensitive to an abnormality of effect sizes and change in sample sizes. Table 4 shows that the smallest and the highest effect sizes of PBL implementation on enhancing primary school students' MA were -1,046 and 3,223, so that the range was 4,269. Based on the classification of effect size developed (Thalheimer & Cook, 2002), the collection of effect size consisted of seven excellent effect sizes, six high effect sizes, three medium effect sizes, two small effect sizes, three ignored effect sizes, and one undetailed effect size. Therefore, the distribution of effect size data from each primary study conducted tended to be even. The overall effect size of PBL implementation in enhancing the MA of primary school students from each of the primary studies is presented in Table 4. A heterogeneity test was used to determine the effect size model selected in this metaanalysis study (Higgins et al., 2003). The results of the heterogeneity test of effect size data from the sixteen primary studies are presented in Table 5. The heterogeneity analysis results in Table 5 showed that the overall effect size data analyzed had significant differences. Also, the p-value of the Q statistic, which was less than 0,05, indicates that the random effect model is significantly better than the fixed effect model (Mike & Cheung, 2015). Thus, the effect model used in this meta-analysis study is the random effect model. In this meta-analysis study, the null hypothesis test was conducted to determine whether PBL implementation significantly enhanced primary school students' MA. The result of the null hypothesis test is presented in Table 6. The null hypothesis test result in Table 6 shows that the p-value of the Z statistic was less than 0,05. These findings interpret that the implementation of PBL significantly enhances the MA of primary school students. Besides, the effect size of the sixteen primary studies analyzed was 1,050, which could be classified as a high effect size. In their study, Kadir et al. (2013) showed a similar result that the effect size of PBL implementation in enhancing students' problem-solving abilities in mathematics and science learning was 1,079, which is classified as a high effect size of the sixteen primary studies analyzed. Other empirical evidence (Puyada & Putra, 2018) in their meta-analysis study on twenty primary studies showed that PBL implementation positively affected student learning outcomes. Likewise, (Gijbels et al., 2005), in their meta-analysis study, showed that PBL implementation was significantly effective in increasing knowledge and skills of the 25 primary studies analyzed. In other related metaanalysis studies, (Dochy et al., 2003)(Dochy et al., 2003)(Dochy et al., 2003)(Dochy et al., 2003it was found that the application of PBL has a positive effect on developing conceptual understanding, connections, and concept application (Dochy et al., 2003). Susanti et al. (2020), in their study which synthesized twelve primary studies, showed that the MCA of junior high school students after the implementation of PBL was significantly higher than the MCA of junior high school students after the implementation of conventional learning with high effect sizes. These findings provide strong evidence that PBL is an effective learning tool that mathematics teachers can use in enhancing the MA of primary school students in learning mathematics. PBL effectively increases the MA primary school students because the PBL design builds knowledge flexibly and broadly, enables them as individuals who can apply their abilities and skills in various conditions, and develop effective problem-solving support effective creative and critical thinking (Yew & Goh, 2016).

b. Analysis of the Study Characteristics
The heterogeneity analysis in Table 5 shows that the effect size data in this meta-analysis study differed significantly from one another. Therefore, the study characteristics are essential for further analysis because the study characteristics have the opportunity to cause the heterogeneous effect size data (Siddiq & Scherer, 2019). The study characteristics analyzed in this meta-analysis study were the MA characteristics, substantive characteristics, and extrinsic characteristics.
The MA characteristics analyzed in this meta-analysis study were MPSA, MCTA, MCA, MLA, and MCUA. The heterogeneity analysis in Table 7 shows that the effect size of PBL implementation in enhancing MPSA, MCTA, MCA, MLA, and MCUA was significantly different. These findings indicate that the MA characteristics lead to the heterogeneity of the overall studies' effect sizes in this meta-analysis. It was found that the effect size of PBL implementation in enhancing primary school students' MLA was higher than the effect size of PBL implementation in enhancing primary school students' MPSA, MCTA, MCA, and MCUA of the five characteristics of MA analyzed in this meta-analysis study. However, these findings can not be generalized in general because there are relatively few studies on the MLA of primary school students analyzed in this meta-analysis study. Likewise, in the previous metaanalysis studies, there has been no statement stating that the implementation of PBL has the greatest effect in enhancing students' MLA compared to other students' MA. The result of the analysis of the characteristics of MA is presented in Table 7. The null hypothesis test result in Table 7 shows that the primary school students' MPSA, MCTA, MLA, and MCUA after the implementation of PBL were significantly higher than the primary school students' MPSA, MCTA, MLA, and MCUA. These findings provide strong evidence that the implementation of PBL in enhancing the MCTA and MLA of primary school students has a very high effect size. (Katminingsih & Widodo, 2015) in their study showed similar findings that the implementation of PBL has a positive effect on the MCTA of primary school students. Another study (Firdaus et al., 2017) found the same findings that primary school students' MLA experienced a positive increase by implementing PBL. Other findings provide strong evidence that the implementation of PBL in enhancing the MPSA and MCUA of primary school students has a high effect size. In their study, (Pohan et al., 2020;Riswari et al., 2018) showed the same results that the implementation of PBL positively affects the MPSA of primary school students. Likewise, (Kaharuddin, 2018) in his study found the same results that the MCUA of primary school students had a positive increase by implementing PBL. Findings in the form of the effect of PBL on MA in this meta-analysis study provide strong evidence and information for mathematics teachers in the primary school level to select PBL as one of the best solutions in implementing mathematics learning in the classroom to enhance primary school students' MPSA, MCTA, MLA, and MCUA.
Another interpretation of the null hypothesis test result in Table 7 shows that the MCA of primary school students by implementing PBL were not significantly different from the MCA of primary school students by implementing conventional learning. These findings provide strong evidence that the implementation of PBL in enhancing MCA of primary school students has a very small effect size and can be ignored. (Rosmala et al., 2017) their study also showed similar findings that the implementation of PBL did not have a significant effect in enhancing the MCA of primary school students. These findings interpret that mathematics teachers at the primary school level should make more efforts by considering the important things in enhancing the primary school students' MCA through the implementation of PBL. The heterogeneity analysis in Table 8 shows that the effect size of the PBL implementation in enhancing the MA of primary school students whose students selection was made by random sampling and purposive sampling was significantly different. It interprets that the characteristics of the sampling technique used in selecting students led to a significantly heterogeneous effect size of the overall meta-analysis study. (Siddiq & Scherer, 2019) in their meta-analysis study found a similar thing that the heterogeneity of the effect size conducted was due in part to the characteristics of the sampling technique used in selecting students as study subjects. However, the effect size of PBL implementation in enhancing the MA of primary school students viewed by sample size did not differ significantly. It indicates that the characteristics of the sample size do not significantly lead to heterogeneity in the effect sizes of the whole studies in this meta-analysis study. This result was similar to the previous MA study that the sample size characteristics did not cause the heterogeneity of the effect size of the studies conducted in the meta-analysis process (Demirel & Dağyar, 2016;Suparman et al., 2021;Yunita et al., 2020;Paloloang et al., 2020).
In terms of the sample size, the MA of primary school students by implementing PBL was significantly higher than the MA of primary school students by implementing conventional learning. PBL implementation at a sample size greater than 30 students was better than a sample size that was less or equal to 30 students. However, , in their study, showed different findings that a sample size of less than or equal to 30 students was better than a sample size of more than 30 students. Likewise, (Demirel & Dağyar, 2016) in their study showed that a sample size of less than or equal to 32 students was better than a sample size of more than 32 students. The difference in findings in this study and the meta-analysis studies conducted (Demirel & Dağyar, 2016;Maximus Tamur, Juandi, & Adem, 2020) can be due to different research focus. In their meta-analysis study, Demirel & Dagyar investigated and evaluated the effect of PBL on student attitudes at the education level of primary school, secondary school, and high school. Likewise, , in their meta-analysis study, focused on investigating and evaluating the effect of the RME approach on students' mathematics learning outcomes at the education level of primary school, secondary school, and high school. However, this meta-analysis study investigated and evaluated the effect of PBL on students MA at primary school.
On the other hand, in terms of the sampling technique used in student selection, the MA of primary school students after the implementation student was significantly higher than the MA of primary school students after implementing conventional learning. The implementation of PBL in which research in selecting students using random sampling techniques was better than using a purposive sampling technique. In their study, Siddiq & Scherer (2019) showed similar results that the effect size using random sampling techniques was higher than purposive sampling techniques. These findings provide recommendations to researchers that in conducting research, especially regarding the implementation of PBL in enhancing the MA of primary school students, researchers should use random sampling techniques in selecting samples to be used as research subjects.
The extrinsic characteristics analyzed in this meta-analysis study were publication year and indexed Scopus status. The heterogeneity analysis in Table 9 shows that studies indexed by Scopus differed significantly from studies that were not indexed by Scopus. The effect size of the studies indexed by Scopus was significantly higher than the effect size of the studies not indexed by Scopus. These findings interpret that through the implementation of PBL, the MA of primary school students whose reports are published in Scopus indexed journal are better than those of the primary school students whose reports are published in a journal not indexed Scopus. However, Scopus and non-Scopus indexed studies have an equal chance of being included in this meta-analysis study. Likewise, studies that were indexed by Scopus and non-Scopus reported that the MA of primary school students' effect size after the implementation of PBL was significantly higher than the MA of primary school students after the implementation of conventional learning. Therefore, these findings provide information that researchers or mathematics teachers, especially primary school mathematics teachers, can use literature published in the form of Scopus and non-Scopus indexed journals as information to enhance students' MA in mathematics learning after the implementation of PBL. However, researchers should prioritize literature from journals indexed by Scopus. The result of the analysis of the extrinsic characteristics is presented in Table 9. Another finding from the heterogeneity analysis in Table 9 shows that the studies published in 2010 -2015 and 2016 -2020 were not significantly different. However, the effect size of studies publ6 -2020 was higher than the effect size of studies published in 2010 -2015. Studies published in 2016 -2020 reported that the MA of primary school students after implementing PBL was significantly higher than the MA of primary school students after the implementation of conventional learning. However, studies published in 2010 -2015 reported that the MA of primary school students after implementing PBL was not significantly different from the MA of primary school students after the implementation of conventional learning. The findings of studies published in 2016 -2020 interpret that the implementation of PBL to enhance the MA of primary school students is getting better and more massive. This finding is indicated by the effect size that has increased from year to year and the increasing number of PBL implementation in mathematics learning to enhance primary school students' MA.

D. CONCLUSION AND SUGGESTIONS
The synthesis of sixteen primary studies relevant to the effect of PBL on MA provides a summary. It estimates that PBL implementation on enhancing the MA of primary school students is significantly high. The heterogeneity of the effect size of the primary studies analyzed in this meta-analysis was significantly due to the characteristics of MA, sampling techniques, and Scopus indexed status. Thus, this meta-analysis study provides suggestions to policymakers in education, especially mathematics teachers at the primary school level, to select PBL as a useful learning model in enhancing MPSA, MCTA, MLA, and MCUA in learning mathematics. Likewise, researchers should use the random sampling technique in selecting students who will be used as research samples, especially in examining the effect of PBL implementation in enhancing the MA of primary school students. It is also better if primary school mathematics teachers and researchers prioritize literature from Scopus indexed journal to find information specifically about the implementation of PBL to solve students' MA problems.
In this meta-analysis study, several weaknesses, such as the number of primary studies indexed by Scopus and the number of literature search engines were a relatively small number. Several substantive characteristics, such as PBL treatment duration, study area, and study year, were not observed. Therefore, for further meta-analysis studies, especially on the effect of PBL implementation in increasing students' MA, the number of primary studies indexed by Scopus and the number of literature search engines should be increased. Likewise, substantive characteristics such as the PBL treatment duration, the study area, and the study year should be included in the meta-analysis study because they could potentially influence the effect size's heterogeneity.

ACKNOWLEDGEMENT
The writers would like to deliver the greatest gratitude to the Indonesian Endowment Fund of Education (LPDP) for financial support.