Under-Utilization of Analysis of Covariance in Behavioral Research

INTERNATIONAL JOURNAL OF MULTIDISCIPLINARY RESEARCH AND ANALYSIS ISSN(print): 2643-9840, ISSN(online): 2643-9875 Volume 08 Issue 10 October 2025 DOI: 10.47191/ijmra/v8-i10-07, Impact Factor: 8.266 Page No. 5547-5551 IJMRA, Volume 08 Issue 10 October 2025 www.ijmra.in Page 5547 Under-Utilization of Analysis of Covariance in Behavioral Research Denis Achung Uyanah, Ph.D. Department Of Educational Foundations, Faculty of Education University of Cross River State Calabar-Nigeria ABSTRACT: One of the greatest source of value of research results is variance control attainable by applying a principle codenamed “MAXMINCOM”. This principle has three components: Maximization of systematic or desirable variance, minimization of error variance and control of variance arising from the effect of unwanted variables, generally called extraneous variables. When the extraneous or nuisance variables can be removed, the research design takes care of that. When such complete removal of the influence of such extraneous variable is not possible or difficult, such variable are deliberately included so that their influences are considered part of the study. Their influences are removed by partialing out the variance attributed to such variables, from the total variance. Analysis of covariance is one of the statistical analysis techniques that is utilized to accomplish this purpose. Many of the reported application of analysis of covariance have one factor with two levels and one covariate. This has caused so many researchers to think that this is a general rule. When there are more than two levels of a factor for instance; A1, A2, A3, you find the pairs A1 and A2, A1 and A3, A2, and A3, for three different hypotheses. If the aim from the beginning is to compare the treatment effect of A1, A2, and A3, then this can be handled on one single analysis with an appropriate post-hoc-test that can only be applied after the observed mean values for A1, A2, and A3, have been adjusted for the effect of the covariate(s). There has been sharp protest from students’ project supervisors and even external examiners when students take the three levels or more of the factor e.g teaching methods, together, carry out an adjustment of treatment means and applying post-hoc test on adjusted means. Students who carryout such proper procedures have been denied graduation, down-graded and sometimes made to reanalyze and interpret the results for such pairs as different hypotheses. The vehemence with which this rejection of the legitimate procedure is done, necessitated this paper. The purpose was therefore to provide theoretical justification of such detailed analysis, to save mainly the students’ who are the victims of the ignorance of the power of ANCOVA and a clarification for project supervisors, examiners and data analysists, that are confronted or consulted with data from such students and researchers. INTRODUCTION There have been very ugly situation that students have been subjected to in the course of executing their research project as a requirement for obtaining a degree (First, Second or Third) where analysis of covariance was applied. This paper is a product of such ugly situations. The students had carried out their studies with a single factor (e.g. teaching method) with more than two methods, meaning the factor had more than two levels. The students most times (there have been several encounters observed by this author) logically followed the procedures in analysis of covariance, beginning with finding the regression coefficient for the regression of the dependent variable on the covariate for each group and testing for the significance of their differences. Where the differences where not significant, a common regression coefficient was computed. Next the students carried out the ANCOVA, removing sum of squares attributable to the covariate from the total sum of squares and testing for significance of the effect of the factor. The observed group means were subsequently adjusted for the effect of the covariate before applying a post-hoc test. The works were turned down or rejected, sometimes by their supervisor, or internal examiner or even an external examiner. There have been cases where the students were made to re-analyze the data, taking the teaching methods in pairs, removing the regression component of the analysis as well as the adjustment made before comparison and report the results purely on the sum of squares mean square and F-ratio test and drawing conclusion from there. There were cases where external examiners vehemently rejected explanations that could disabuse their mindset. Such instances have been traumatic to both students and their internal supervisors and examiners. The students encores huge financial loses, dampened morale and fear as well as wrong resolve, either not to carry out any study that will result in the application of ANCOVA or use the wrong approach of pairing teaching methods, for example and reporting the results, without preliminary tests and any adjustment of treatment means. The reputation of project supervisors were drag down with a feeling Under-Utilization of Analysis of Covariance in Behavioral Research IJMRA, Volume 08 Issue 10 October 2025 www.ijmra.in Page 5548 that students can no longer trust their expert advise in the course of their research work. Some examiners further claimed that they had done a survey of application of ANCOVA in academic journals and have not seen anywhere regression analysis is associated with ANCOVA neither have they seen any adjustment carried out. Explanations that, researchers whose works are reported in journals may have reduce their studies to two levels for economic and simplicity purposes and cannot be taken as a rule, fell on deaf ears. This expository work on the capacity of ANCOVA became very imperative. The purpose of research design generally and experimental design in particular is to ensure that the outcome and conclusion drown have both internal and external validity. This is achieved by maximizing the systematic variance, the variance in the dependent variable(s) accounted for by the independent variables of interest to the researcher; the minimization of error variance, if possible the reduction in the error to random error and control of variances whose source are extraneous to the study or unwanted. Sometimes this control is easy through a careful setting of the experiment. However, there are variables whose influence cannot be eliminated so easily. In fact, in some cases, it is close to impossibility. For example, in a study of the effect of teaching method on students’ achievement in mathematics, the influence of the differences in their previous knowledge, their intelligence quotient, their aptitudes etc. are very difficult to remove. In such cases, it is almost a convention that such variables should be built into the design. Appropriate statistical techniques are then applied to remove the variance attributable to such built-in factors from the total variance, called partialing. One such statistical technique widely applied, especially in experimental studies, is analysis of covariance. The independent variable is traditionally called a factor and there is always a dependent variable or response variable. The variable built-in whose effect or influence is unwanted is called a Covariate. When there is only one factor, it is one-way analysis of covariance (ANCOVA). So, there could be two-way, three-way etc. depending on the number of factors. When the dependent variable is just one, the analysis is called Univariate analysis of covariance. When there are more than one dependent variables, it is called Multivariate analysis of Covariance (MANOCOVA). The number of covariates could be more than one. The logic of analysis of Covariance When a variable has been residualized (such as finding the difference between the original variable y and the predicted y1) the correlation between the predictor variable x and the predicted variable y1 is always one while the correlation between the predictor variable x and the residual ye is zero. So one may write this as: Rxy1 = 1 rxye = 0 where ye = y-y1 The residualized variable, according to Kerlinger (1973), is one from which whatever variance it shared with the predictor variable has been removed. If one were studying the effect of teaching methods on students achievement in mathematics, and wants to adjust the mathematics achievement score for differences in quantitative aptitude, the independent variable is teaching method, the dependent variable is mathematics achievement and the covariate is quantitative aptitude. One would first use the quantitative aptitude to predict their mathematics achievement, by regressing mathematics score on aptitude score. Suppose the students had been grouped such that yίȷ is the actual achievement score of students ί in group j, then yίȷ is the predicted mathematics score and yίȷ - ŷίȷ is the residual. One can calculate this residual for all students in the study, the resulting set of scores have zero correlation with the students’ quantitative aptitude. A test for significance of differences of the groups will indicate whether the group means are different when their scores have been adjusted (purged) for possible differences in mathematics aptitude. This is the logic behind the celebrated analysis of covariance in design and analysis of experiment. One can therefore write that yίȷ = 𝑦 + Tȷ + b (xίȷ - 𝑥 ) + eίȷ where yίȷ the score obtained by a ίth student under jth treatment, 𝑦 is the grand mean mathematics achievement, Tȷ is the effect of treatment j (teaching method), b is the common regression coefficient when all treatment groups are pooled with y as dependent variable and x (Covariate) as independent variable j xίȷ is the score obtained by the ίth student in the jth group on the covariate j 𝑥  is the grand mean of the covariate and eίȷ is the error in the score of the ίth student in the jth group. This Equation can be rewritten as: yίȷ - b (xίȷ - 𝑥 ) =𝑦 + Tȷ + eίȷ This Equation shows clearly that after the adjustment (LHS), what ever is left is made up of the grand mean (𝑦), the treatment effect (Tȷ) and an error term (eίȷ). This also shows that the end point of ANCOVA is the adjusted mean score (Kothari & Gorg, 2014). The implication is that where the level of a factor or treatment groups are only two, the adjusted means are merely reported. Some researchers, either out of ignorance or deliberately, do not even report the adjustment means. However, if the levels of the factor are more than two, post hoc comparison is done using the adjusted means and not the unadjusted means. Under-Utilization of Analysis of Covariance in Behavioral Research IJMRA, Volume 08 Issue 10 October 2025 www.ijmra.in Page 5549 This is where most researchers don’t get to, either because they do not know that they should get to this point of the analysis and report same or they deliberately do not want to. The former reason looks more plausible. Hence, the need for this paper. The usefulness of getting to this end cannot be over emphasized. It enables the researchers to identify as well as take into account sources of variance attributable to concomitant factors thereby providing more of the much needed control and increasing the validity of the statistical conclusion. An adjustment for the effect of the covariate(s) will hence lead naturally to the reduction in the error term and consequently to an increase in the sensitivity of all associated tests. What is accomplished is very comparable to what designs like repeated measures or treatment-by-levels or randomized block designs do in the maximization of desirable systematic variances (Feldt, 1958; Cochran, 1957; Elashoff, 1969). In the main table of ANCOVA results, the sum of squares due to the covariate is removed by subtraction, from the total variance, just as the sum of squares due to other factors in the ANCOVA model are subtracted to get the error term. The mean squares are computed and the Fvalues at the end, together with their associated P-values. When the treatment effect is significant, it is necessary to determine specifically which of them differ significantly from each other. This is done by applying a suitable multiple comparisons test, which may be pairwise or a combination of treatment means, usually predetermine at the design stage of the study. The multiple comparison, according to Edwards (1972) and Kerlinger and Pedhazur(1973) is done between adjusted means not the unadjusted means. When the level of the factor(s) is (are) just two (2) the unadjusted and the adjusted are included in the table of the ANCOVA results. If the factor(s) have more than two levels, the multiple comparison results may be presented in a separate (detached] table. Such a table should show only the adjusted means, the mean differences, the confidence limits (if desired) and the associated P-values. The P-values are used to determine if a paired comparison is significant or not. It should equally be remarked that, when all treatment groups have equal mean on the covariate, no adjustment of means takes place, since the sum of squares due to covariate will be zero. This explains why the preliminary regression analysis for the influence of the covariate on the dependent variable, is very important. In other words, if the β -value is zero or very close to zero, covariance analysis may not be necessary. The normal ANOVA will be enough. It should equally be noted that an ANOVA of the covariate with the treatment as factor will equally serve the purpose of the regression analysis in part, since further analysis may be required to determine the value of β when the means of the covariate by treatment are significantly different. That means, if the ANOVA results are significant, β will still be calculated. That is why regression analysis is preferred at that preliminary stage of the analysis. Basic assumptions of ANCOVA For the application of ANCOVA and all its’ extensions to situations where there are many factors or many response variables or many covariates, with the conclusion drawn therefrom, to have both internal and external validity, the following assumptions should be satisfied: 1. There is some kind of relationship between the dependent variable(s) and the covariates(s). When the regression coefficient if zero (0) or close to Zero (0), the ANCOVA ceases to be useful. Therefore regression analysis should be carried out as a preliminary test. Where the regression of the dependent variable (y) on the covariate (x) is significant, the process should continue. Where the regression analysis yield no significant results, the ANCOVA process may not be very useful or a wasted efforts and resources (Kothari & Garg 2014). 2. The relationship between the dependent variable(s) and the covariates) is the same in all the treatment groups. This assumption should always be tested or checked. This assumption is referred to in literature of ANCOVA as the Homogeneity of regression coefficients. According to Kerlinger and Pedhazer (1973), testing this assumption proceeds in exactly the same manner as the testing of the difference between regression coefficient and constants. The matters to be resolved are: Does using separate regression coefficient for each experimental group add significantly to the regression sum of squares, when compared to the regression sum of squares obtained using a common regression coefficient? It should be pointed out that when the regression lines are parallel, the bs are identical, implying that the sum of squares obtained from using each b for its own group is the same as the regression sum of squares obtained from using a common b for all groups. Therefore, the difference in the two sums of squares is a valid measure of the differences in the b’s, that is due to departure from parallelism of the regression lines of the separate groups. It also means that when this difference is not significant, it can be concluded that, there are no differences between the b’s and a common b will serve the purpose. If there are three experimental group) and let S1 S2 and S3 be their respective sums of squares, then, their sum S will be S1 + S2 + S3. The data is then pooled and sum of squares computed. Let this be Sp. The Sp-S represents the discrepancy, which can be tested for significance using the f-ratio text. The requirement is that this test should not be significant. When it is not significant, the use of common’ b’ is justified. Under-Utilization of Analysis of Covariance in Behavioral Research IJMRA, Volume 08 Issue 10 October 2025 www.ijmra.in Page 5550 That the ‘b’s are not significantly different, only shows that the regression lines are parallel. The lines need to be coincident ie the regression constants should at most not be significantly different. If they, are different then an interaction exist among the independent variables and further analysis may be required, making the picture clear. If they are not significantly different, the difference among the intercepts is investigated. Testing the difference among the intercepts is the same as testing the difference between treatment effects of the factor. This is done by testing the difference between the Rs or between two proportions of variance. If this test is significant then the difference between the treatment means is significant. When there are two groups, the analysis ends here. However, if there are more than two groups, an appropriate post hoc text is carried out on the adjusted means. Analysis of Covance with multiple covariates The procedures for ANCOVA with single covariate can be easily extended to two or more covariates. The logic remains the same. Preliminary analysis will require the determination of the collective influence of the covariates on the dependent variable. If this yield result that are not significant, or the collective influence is approximately zero ANCOVA may be discontinued. If the collective influence is significant, the squared multiple correlation coefficient (R12) is noted. Another squared multiple correlation of the dependent variable with the covariates and the vectors representing the treatments (R22) is then computed. The difference between these two R-squared values (R12 – R22) indicates the proportion of the variance in the dependent variable accounted for by the treatments when the influence of the covariates have been removed or adjusted for. This difference is tested for significance using the F-ratio test. With the advent of computer programmes all you need to do is to enter the dependent variable, the factor (fixed or random) and the covariates in their dialogue boxes. Specify what you need in the output: the model type of sums of squares, post hoc tests (fixed or random) and the covariates in their dialogues boxes. Specify what you need in the output: the model, type of sums of squares, post hoc tests (if any) etc as is done for one covariate. The results come with unadjusted and adjusted means, the Rsquared, adjusted R-squared, the βs and the mean value of the covariates used in adjusting the observed mean of the dependent variable (Using SPSS) ANCOVA with multiple factors (categorical variable) The logic of analysis of covariance can be extended to situations where there are more than one independent variable. For example, a study may be conducted to determine the effect of teaching method and area of specialization on students’ achievement in mathematics. Teachers who studied social sciences, physical sciences, engineering and mathematics are often drafted to teach mathematics. So the area of specialization is the second factor with four (4) levels of specialization. In this case, there are tow categorical variables-teaching methods and area of specialization. There may be covariates-premeasured. This is a factorial analysis of covariance case. The analysis begins with the determination of the nature of the relationship (influence) between the dependent variable and covariate(s). This leads to either stop the ANCOVA process or “proceed” to determine or compute the multiple R-squared. Next, the two factors are represented by two coded vectors-one for each factor. Multiple R-squared is the computed using the covariates and the two coded vectors (product vectors between the categorical variables are generated to produce or represent the interaction). The difference between the multiple R-squared that includes all the covariates and all the coded vectors on one hand and the multiple R-squared with covariate only, is computed. This difference represents the effects of the two factors, when the influences of the covariates have been adjusted for. The results can be tested for significance using the normal F-ratio test. This procedure is for manual computations. With the advent of statistical packages like the SPSS, it is much easier. Enter the data in the data spread sheet and define the variables in the variable spread sheet. Enter the dependent variable in its dialogue box. Enter the factors in the factors dialogue box in this example, all the factors are fixed. So, both teaching method and teachers’ qualification will enter the same box. Take the covariate or covariates to covariates dialogue box. Specify what you need in the output, descriptive statistics, post-hoc tests, level of significance, plots, the interactions that are of interest to you, else the system will produce all interaction, indicate effect size, if you want and click OK. You are done. You will learn how to interpret and the necessity of each result. CONCLUSION This paper, within the limit of presentation due to deliberate exclusion of computational details has shown that it is grave error to limit the application of ANCOVA to one factor with just two levels; it is an error to compare unadjusted means in post hoc tests; ANOCOVA can be carried out with multiple factors, multiple factors levels and multiple covariates; the endpoint is the adjustment of treatment means; the ANCOVA process may be terminated if all treatment groups have equal mean on the covariate or when the influence of the covariate on the dependent variable is zero or very close to zero. Under-Utilization of Analysis of Covariance in Behavioral Research IJMRA, Volume 08 Issue 10 October 2025 www.ijmra.in Page 5551 REFERENCES 1) Kerlinger, F.N. & Pedhazur, E.J. (1073). Multiple regression in behavioral research. New York: Holt, Rinehart and Winston. 2) Elasholt, J.D. (1969). Analysis of covariance: a delicate instrument. American Educational Research Journal, 6,383-401 3) Cochran, W.G. (1957). Analysis of covariance: its nature and uses. Biometrics, 13, 261-281 4) Feldt, L.S. (1958). A comparison of the precision of three experimental designs employing a concomitant variable. Psychometrika, 23, 335-353. 5) Edwards, A.L. (1972). Experimental design in psychological research (4th ed). New York: Holt, Rinehart and Winston, Inc. 6) Kothari, C.R. & Garg, G. (2014). Research Methodology: methods and techniques (3rd ed). New Delhi: New Age International Ltd. There is an Open Access article, distributed under the term of the Creative Commons Attribution – Non Commercial 4.0 International (CC BY-NC 4.0) (https://creativecommons.org/licenses/by-nc/4.0/), which permits remixing, adapting and building upon the work for non-commercial use, provided the original work is properly cited.