An Alternative Solution to Hotelling T square under the Heteroscedasticity of the dispersion Matrix

Abstract

This work focus on developing an alternative procedure to the multivariate Behrens-Fisher problem by using an approximate degree of freedom test which was adopted from Satterthwaite univariate procedure. The proposed procedure was compared via R package simulation and real-life data used by Tim (1975) with six (6) existing procedures namely: Johanson, Yao, Krishnamoorthy, Hotelling T square, Nel and Van der Merwe and Yanagihara. And it was discovered that proposed procedure performed better in term of power of the test than all existing procedures considered in all the scenarios that are at different: (i) random variables (p), (ii) variancecovariance matrix, (iii) sample size and (iv) significant level (α). And compete favorably well in term of type I error rate with Johanson, Yao, Krishnamoorthy, Nel, and Van der Merwe.

Keywords

Variance co-variance matrix, linear combination, Type I error rate, power the test, Heteroscedasticity, R statistical package

Introduction

Suppose we have a random sample of size n₁, x₁₁, x₁₂, x₁₃, ……x_1n1 for N(μ₁,σ₁²) and a second random sample of size n₂, x₂₁, x₂₂, x₂₃, . . . x_2n2 for N(μ₂,σ₂² ). It is desired to test H0: u1= u2 against H₁: u₁≠u₂. If σ₁ and σ₂ are both known a normal test is used. If σ₁=σ₂ but both are unknown a t-test is commonly used with the test statistics.

Where

A t-test is a type of statistical test that is used to compare the means of two groups, such as men vs. women, athletes vs. non-athletes, young vs. elderly, or you may want to compare means measured on a single group under two different experimental conditions or at two different times. T-tests are a type of parametric method; they can be used when the samples satisfy the conditions of normality, equal variance, and independence. T-tests can be divided into two types. There is the independent t-test, which can be used when the two groups under comparison are independent of each other, and the paired t-test, which can be used when the two groups under comparison are dependent on each other. If σ₁≠σ₂ and both are unknown then we are confronted with the Behrens-Fisher problem. There is no universally accepted testing procedure for this problem although arrays of tests have been developed and will be discussed in the Review of Literature. Behrens [1] proposed the statistics.

In the literature, there is a modified test statistic (t-test) when the assumption of equal variances is violated has been known as the Behrens-Fisher problem [1,2]. Early investigations showed that the problem can be overcome by substituting separatevariances tests, such as the ones introduced by Welch [3,4], and Satterthwaite [5], for the Student t-test. These modified significance tests, unlike the usual two-sample Student t-test, do not pool variances in the computation of an error term. Moreover, they alter the degrees of freedom by a function that depends on sample data. It has been found that these procedures in many cases restore Type I error probabilities to the nominal significance level and also counteract increase or decrease of Type II error [6-9].

Student’s t-test is univariate and analog to Hotelling T square which is the multivariate version of T-test and this Hotelling’s T2 has three basic assumptions that are fundamental to the statistical theory: independent, multivariate normality and equality of variance-covariance matrices. A statistical test procedure is said to be robust or insensitive if departures from these assumptions do not greatly affect the significance level or power of the test.

To use Hotelling’s T² one must assume that the two samples are independent and that their variance-covariance matrices are equal (Σ₁=Σ₂=Σ). When variance-covariance matrices are not homogeneous and unknown, the test statistic will not be distributed as a T². This predicament is known as the multivariate Behrens-Fisher problem.

The Behrens-Fisher Problem addresses interval estimation and hypothesis testing concerning the differences between the means of two normally distributed populations when the variances of the two populations are not equal. While Multivariate Behrens- Fisher problem deal with testing the equality of two normal mean vectors under heteroscedasticity of dispersion matrices. These are the some of the existing Multivariate Behrens-Fisher problem: Yao [10], Johansen [11], Nel et al [12], Kim [13], Krishnamoorthy and Yu [14], Gamage et al [15] Yanagihara and Yuan [16],and Kawasaki and Seo [17] and so on. But with all these procedures there is no one with a hundred percent (100%) satisfactory in term of power of the test and type I error rate. And each of these scholars works on the degree of freedom using a different method which is classified into four (4): Approximate degree of freedom tests, Series expansion-based tests, Simulation-based tests, and Transformation-based tests.

Scheffe [18],Lauer and Han [19], Lee and Gurland [20], Murphy [21], Yao [10], Algina and Tang [22], Kim [13], De la Rey and Nel’s [23], Christensen and Rencher [24] , Oyeyemi and Adebayo [25]. All these authors mentioned and many more have work on the comparison of some of the Multivariate Behrens-Fisher problem procedures.

The purpose of this work is to develop an alternative procedure for multivariate data that will be robust compared to other procedures and the work will begin with an introduction to the statistical notation that will be helpful in understanding the concepts. This is followed by a discussion of procedures that can be used to test the hypothesis of multivariate mean equality when statistical assumptions are and are not satisfied. We will then show how to obtain a test that is robust to the covariance heterogeneity.

Multivariat Behrens-Fisher Problem (Existing Procedure)

Consider two p-variate normal populations and where and are unknown p × 1 vector and Σ₁ and Σ₂ are unknown p × p positive definite matrices.

Let and

denote random samples from these two populations, respectively. We are interested in the testing problem

For i=1,2, let

Then and which are sufficient for the mean vectors and dispersion matrices, are independent random variables having the distributions:

Where W_p(r,Σ)denotes the p -dimensional wishart distribution with df=r and scale matrix Σ.

The following are the existing procedures or solutions to Multivariate Behrens-Fisher problem considered in this study

Yao Procedure

Yao [10], invariant test. This is a multivariate extension of the Welch ‘approximate degree of freedom’ solution provided by Turkey and the test statistic is based on a transformation of Hotelling T². And is based on with the

Statistical significance is then assessed by comparing the T_Yao statistic to its critical value F_α(p,v-p+1), that is, a critical value from the F distribution with p and v–p+1 degrees of freedom (df)

Johansen Procedure

Johansen [11], invariant test, Yanagihara and Yuan [16], Kawasaki and Takashi [17]. They used T² ~ qF_pv where

And his proposed test statistic

Statistical significance is then assessed by comparing the T_Johan statistic to its critical value Fα(p,v_Joh), that is, a critical value from the F distribution with p and v_Joh degrees of freedom (df).

Nel and Van der Merwe (1986) noninvariant solution

Here we use

except that v is defined by

Krishnamoorthy and Yu (2004)’s, Lin and Wang (2009), Modified Nel/ Van der Merwe Invariant Solution

We use the idea as before, namely,

with the d.f.v defined by

Statistical significance is then assessed by comparing the T_Krish statistic to its critical value Fα (p,v_kY-p+1), that is, a critical value from the F distribution with p and v_kY-p+1 degrees of freedom (df)

Yanagihara and Yuan Procedure used the Series Expansion Based Test to Developed an Alternative Procedure to Multivariate Behrens-Fisher problem

Hotelling’s T²

Where

The test statistic can also be converted to an F statistic,

Where N=n₁+n₂. Statistical significance is then assessed by comparing the v_hotel statistic to its critical value Fα (p, N-p-1), that is, a critical value from the F distribution with p and N-p-1 degrees of freedom (df).

The Mean and the Variance of the Χ² Distribution with N degrees of Freedom

The chi-square distribution is defined with n degrees of freedom by

where are independent random variables, each with distribution N(0,1).

Find the expected value and variance of both sides, then we have

and

And all the instances of Z_i have identical distributions, then

and

Where Z is the random variable with distribution N(0, 1). Then

Therefore

(1)

For Var(Z²)

Now

(2)

To find E(Z⁴), we will use the fact that for any continuous random variable X with probability density function f, and any exponent k,

And that the probability density function f of the N(0,1) random variable is given by

Then,

By integration by parts, we have,

(3)

Therefore substitute equation (3) into equation (2) then we have

(4)

For two sample t-test, we will limit this work to the version of the test where we do not assume that the two populations have equal variances. Let random sample x1,…,x_n1 from a random variable X with distribution N(μ₁,σ₁) and a random sample y1,….,y_n2 from a random variable Y with distribution (μ₂,σ₂). We have

Strictly speaking, this statistic does not follow t-distribution, therefore;

The variance of is and as an estimator for we have

For t to bet–distribution, there would have to be some multiple of that is chi-square distribution and this is not the case. However, remember that in the one-sample case, had a chi-square distribution with n-1 degrees of freedom. By analogy, we have has a chi-square distribution with r degrees of freedom. Satterthwaite found the true distribution of and showed that if r is chosen so that the variance of the chi-square distribution with r degrees of freedom is equal to the true variance of , then, under certain conditions, this chi-square distribution with r degrees of freedom is a good approximation to be the true distribution of so from this point, we are assuming that has distribution. So from equation                                    (4)

(5)

(6)

Equating the equation (5) and (6)

(7)

Now and s₁ and s₂ are independent so

(8)

We know that has a chi-square distribution with n₁-1 degrees of freedom, from equation (3)

(9)

Substitute equation (9) into equation (8)

(10)

Substitute equation (10) into equation (7)

(11)

In practice, the values of the population variances, and are unknown, and so we replace and by their estimators and also from equation (11)

Multivariate Approach

We shall consider the test statistic y'S^-1 y and use Univariate Satterthwaite approximation of degrees of freedom method to suggest multivariate generalization based on the T²–distribution. We have

Where b is an arbitrary constant vector

Multivariate of a version of equation (5)

(12)

Multivariate of a version of equation (5) is equation (12)

Note (14)

Put equation (14) into equation (13)

(15)

Multivariate of a version of equation (9) is

(16)

Put equation (16) into equation (15)

(17)

Equation (17) becomes

The values of the population variances Σ_i are unknown, and so we replace Σ_i and b'Σb by their estimators S_i and b'Sb

(18)

Put equation (14) into equation (18) to have

(19)

Set b=S^-1 y then equation (19) becomes

(20)

Equation (20) can be in this form

(21)

Let then equation (22) become

(22)

(23)

Then equation (23) is the test statistic of the proposed procedure. Statistical significance is then assessed by comparing the T_Prop statistic to its critical value that is, a critical value from the F distribution with p and degrees of freedom (df)

Simulation Study

A simulation study using R package was conducted in order to estimate and compare the Type I error rate and power for each of the previously discussed approximate solution [10,11,14], Proposed procedure, Hotelling’s T square, [14,16]. The simulations are carried out when the null hypothesis is true and not true, for Multivariate normal distribution, when there are unequal variancecovariance matrix. Five (5) factors were varied in the simulation: the sample size, the number of variables p, variance covariance matrices, mean vectors, and significant levels. For each of the above combinations, an ni ×p data matrix X_i(i=1 and 2) were replicated 1,000. The comparison criteria; type I error rate and power of the test were therefore obtained and the results were presented in both tabular.

The following are the levels used for each of the three factors.

These levels provide 36-factor combinations the values for sample size are shown in Table 1.

Table 1: Levels used for each of the three factors.

Result

From the Table 2 Nel and Van der Merwe have the highest power of the test when the sample sizes are small (20, 10) but at (50, 30) and (100, 60) the proposed procedure has the highest power than all other procedures.

Table 2: Power of the test.

From Table 3, when the sample size is (20,10) the proposed procedure is on nominal level exactly while Hoteling T square and Yanagihara [16] are below the nominal level, but at (50,30) and (100,60) all the procedures are below the nominal level, at significant level 0.01. At α=0.025, all the procedures are inflated at (50,30) and deflated at (20,10) and (100,60).

Table 3: Type I error rate.

From Table 4, It is obvious that proposed procedure performed better than all other procedures at (50,30) and (100, 60) but Nel and Van der Merwe is better when the sample size is (20,10) in all the scenarios considered.

Table 4: Power of the test.

Table 5, the type I error rate of all procedures considered are fluctuating, either inflated or deflated. At significant level 0.01, 0.02, 0.05 their inflation in type I error rate, when sample sizes are (50,30) and (100,60), but at (20, 10) all most all the procedures are deflated.

Table 5: Type I error rate.

Table 6 shows when that sample size is (20,10) Nel and Van der Merwe performed better than other procedures, but when sample size increases to (50,30) proposed procedure is better. And there was a great competition among the procedures at (100,60).

Table 6: Type I error rate

Hotelling T [26] square and yanagihara [16] are below the nominal level in all the scenarios considered, while other procedures fluctuated (Inflated or deflated) round the nominal level as in Table 7.

Table 7: Power of the test.

Data Set (For Illustrated Example)

The data used here is an illustrated example used by Timm. The two sample sizes considered are ten and twenty respectively (n₁=10 and n₂=20) and two random variables (p=2) form each population.

The sample means and their covariances are

n₁=10, n₂=20

The difference between the means is

And the test statistic is

T²=11.58542

From Table 8, the proposed procedure has the highest power followed by Yanagihara [16], Krishnamoorthy [14], Yao [10], Hotelling T square [26], Johansen [11] and Nel and Van der Merwe [14] at all the significant level α considered (α=0.05, 0.025 and 0.01).

Table 8: The result from the illustrated example.

Findings

From the simulation, it is obvious from Tables 1, 3 and 5 that when the sample size is very small (20,10) proposed procedure is not at his best, but when sample size increases to (50,30) and (100,60), the proposed procedure performed better than the all procedures considered. Nel and Van der Merwe [14] performed better when the sample size is very small (20,10) followed by Yao [10], Krishnamoorthy [14] and Proposed procedures in term of power of the test in all the scenarios considered.

In term of Type I error rate, proposed procedure compete favorably well with the other procedures selected for this study. Yao [10], Krishnamoorthy [14], Johansen [11], Nel and Van der Merwe [14] and the proposed procedures are fluctuating (Inflated and Deflated) around the nominal level while Hotelling T [26] square and Yanagihara [16] are below the nominal level.

References

1. Behrens WU. Ein Beitrag zur Fehlerberechnung bei wenigen Beobachtungen. Institut für Pflanzenbau. 1929;68:807-837.
2. Fisher RA. The Fiducial argument in statistical inference. Annals of Eugenics. 1935;6:391–398.
3. Welch BL. The significance of the difference between two means when the population variances are unequal. Biometrika. 1938;29:350-362.
4. Welch BL. The generalization of Student’s problem when several different population variances are involved. Biometrika. 1947;34:29-35.
5. Satterthwaite FE. An approximate distribution of estimates of variance components. Biometrics Bulletin. 1946;2:110-114.
6. Overall JE, et al. Tests that are robust against variance heterogeneity in k × 2 designs with unequal cell frequencies. Psychological Reports. 1995a;76:1011-1017.
7. Overall JE, et al. Power of a test that is robust against variance heterogeneity. Psychological Reports. 1995b;77:155-159.
8. Zimmerman DW. A note on preliminary tests of equality of variances. Br J Math Stat Psychol. 2004;57:173-181.
9. Zimmerman DW, et al. Rank transformations and the power of the Student t-test and Welch t' test for non-normal populations with unequal variances. Can J Exp Psychol. 1993;47:523-539.
10. Yao Y. An approximation degrees of freedom solution to the multivariate Behrens-Fisher problem, Biometrika. 1965;52:139-147.
11. Johansen S. The Welch-James approximation to the distribution of the residual sum of squares in a weighted linear regression. Biometrika. 1980;67:85-95.
12. Nel DG, et al. A solution to the multivariate Behrens-Fisher problem. Commun Stat-Simul C. 1986;15:3719-3735.
13. Kim S. A Practical Solution to the Multivariate Behrens-Fisher Problem. Biometrika. 1992;79:171-176.
14. Krishnamoorthy K, et al. Modified Nel and Van der Merwe test for the multivariate Behrens-Fisher problem. Stat Probab Lett. 2004;66:161-169.
15. Gamage J, et al. Generalized p-values and generalized confidence regions for the multivariate Behrens-Fisher problem and MANOVA. J Multivar Anal. 2004;88:177-189.
16. Yanagihara H, et al. Three approximate solutions to the multivariate behrens fisher problem. Comm Statist Simulation Comput. 2005;34:975-988.
17. Kawasaki T, et al. A two-sample test for mean vectors with unequal covariance matrices. American Journal of Mathematical and Management Sciences. 2012;31:117-135.
18. Scheffe' H. Practical solutions of the Behrens-Fisher problem. J Am Stat Assoc. 1970;65:1501-1508.
19. Lauer GN, et al. Power of Cochran’s test in the Behrens-Fisher Problem. Technometrics. 1974;16:545–549.
20. Lee A, et al. Size and power of tests for equality of means of two normal populations with unequal variances. J Am Stat Assoc. 1975;70:933–947.
21. Murphy BP. Comparison of some two sample means tests by simulation. Commun Stat-Simul C. 1976;5:23-32.
22. Algina J, et al. Type I error rates for Yao’s and James’ tests of equality of mean vectors under variance-covariance heteroscedasticity. Journal of Educational Statistics. 1988;13:281–290.
23. De la Rey DG. A comparison of the significance levels and power functions of several solutions to the multivariate Behrens-Fisher problem. S Afr Stat J. 1993;27:129-148.
24. Christensen WF, et al. A comparison of Type I error rates and power levels for seven solutions to the multivariate Behrens-Fisher problem. Commun Stat Simul Comput. 1997;26:1251–11273.
25. Oyeyemi GM, et al. Comparison of the invariant solution to the multivariate Behrens-Fisher problem. J Nigerian Assoc Math Phy. 2016;34:199-208.
26. Hotelling H. The generalization of student’s ratio. Annals of Mathematical Statistics. 1931;2:360-378.