The income distribution in economic theory describes how a region’s total wealth is distributed amongst its population (Sullivan, 2003). Back to classical economists time, income distribution was a key concern of economic theory, as it plays an important role in measuring the health of a region’s economy. For modern economics, more attention is paid to inequalities in certain measurements of income distribution, an example of which is the Gini coefficient (Yitzhaki, 1979), now universally used by many international organizations such as the United Nations and the World Bank. The Gini coefficient, however, summarizes information about a distribution in one number, and loses some information such as the intrinsic structure of the distribution. Other measurements such as the Lorenz curve (Lorenz, 1905), also have such kind of disadvantage.

The probability density of the income distribution can often be estimated when the sample size is large enough. From the 1890s, many parametric distributions were introduced to model income distribution. In 1895, Pareto (1964) (originally published in 1895) first proposed the Pareto density function to model the income distribution. Gibrat (1931) suggested the usage of a two-parameter log-normal distribution. Salem and Mount (1974) proposed a two-parameter Gamma density to approximate the distribution of personal income. Bartels and Van Metelen (1975) suggested another two-parameter density, the Weibull distribution, to model personal income. McDonald (1984) considered two generalized Beta distributions as models for the distribution of income. Furthermore, McDonald and Xu (1995) introduced a five-parameter beta distribution which nests the generalized Beta and Gamma distributions to model income distribution. In summary, these methods used heavy tail distributions to model personal income. When testing the equality of income distributions, people often compared parameters in the parametric models. This method leans on the assumption that the parametric models sufficiently describe both income distributions. Still, some information loss would occur in such comparisons. In addition, it is often not reasonable to assume the same parametric model form for different regions, and such comparison would be impossible when the regional models are parameterized differently.

The energy distance (Székely and Rizzo, 2004) is a measure that is often used to test for equality of distributions. Rizzo and Székely (2010) provided a nonparametric version of the analysis of variance (ANOVA) called the distance components (DISCO), which partitions the total dispersion in the dataset into components that are analogies of ANOVA’s variance components. In addition, the energy distance can also be used in feature selection and generalizations of clustering algorithms (Rizzo and Székely, 2015). Li and Rizzo (2017) extended the usage of the energy distance and proposed k-groups, a generalization of the K-means clustering algorithm. K-groups aims to put similar samples in the same cluster so that the dispersion between the k groups are maximized. The maximization is achieved upon computation of energy distances, instead of Euclidean distances.

In order to address the disadvantages of the Gini coefficient and parametric models, we use the energy distance between different regions to test the equality of income distributions, which is a parametric-free method. Based on the energy distance, we propose a clustering method which clusters not individual observations, but income distributions, in different regions. The main contribution of this work is that all proposed methods are nonparametric - we do not require any parametric assumptions for income distribution.

The remainder of this paper is organized as follows. In Section 2, we give a brief introduction to the energy distance, and propose our testing and clustering procedures. In Section 3, we present the performance of the proposed illustrate our proposed procedures using extensive simulation results. In Section 4, we apply our methods on China Health and Nutrition Survey data. We conclude this paper with a brief discussion in Section 5.

2.1. Energy distance

The energy distance is a statistical distance defined for two probability distributions. Following Székely and Rizzo (2004), we consider two distributions denoted by F_X and F_Y. The energy distribution between these two distributions is defined as:

where X, X′, Y, Y′ are independent random vectors in ?^d with finite expectations, and || · || denotes the Euclidean norm. The distance D_e(F_X, F_Y) equals 0 if and only if F_X ≡ F_Y, while a large energy distance corresponds to very different distributions. Hence it provides a parametric free similarity measure of two distributions.

Now consider two independent samples from F_X and F_Y: X = {x₁, …, x_n} and Y = {y₁, …, y_m}. From Székely and Rizzo (2004), the energy distance associated with the independent random samples can be written as:

where || · || is the Euclidean distance. Based on (2.2), we can calculate the distance between two income distributions.

2.2. Hypothesis testing

For the two-sample problem, suppose we have two samples X₁, …, X_n1 and Y₁, …, Y_n2. The null hypothesis of two-sample is:

where F₁ and F₂ are the respective distributions of X₁, …, X_n1 and Y₁, …, Y_n2. The two-sample test statistic is defined using (2.2):

Let n = n₁ + n₂ be the total sample size of the sample. Under the null hypothesis, X₁, …, X_n1 and Y₁, …, Y_n2 follow the same distribution. Therefore, the random permutation without replacement from X₁, …, X_n1, Y₁, …, Y_n2 is $W_1^{\pi },\dots ,W_{n_1}^{\pi },\dots ,W_n^{\pi }$. Then we can calculate the test statistics T_π based on $W_1^{\pi },\dots ,W_{n_1}^{\pi },\dots ,W_n^{\pi }$. For a fixed number of permutations, under the null hypothesis, the empirical distribution of the test statistic can be obtained. Based on the empirical distribution of the test statistic, the critical value for the statistic, c_α, can be chosen to satisfy:

where α ∈ (0, 1), and T_n is the full permutation test statistic. Proofs of the limiting distribution of test statistics T, the existence of c_α and consistency results are given in Székely and Rizzo (2004). From this limiting distribution, we can reject null hypothesis if T > c_α.

It is easy to extend the two-sample testing problem to k-sample problem. Suppose X₁, …, X_k are k independent samples of random vectors with respective distributions F₁, …, F_k. The null hypothesis is:

and the alternative is F ≠ _i, F_j for some i ≠ j.

For X_i and X_j, where i ≠ j, we can calculate the energy distance T_{i j} between X_i and X_j using (2.4). The k-sample test statistic is:

Similar to in the two-sample case, we can get the critical value from the random permutations from X₁, …, X_k.

2.3. K-means clustering

According to the energy distance calculated by (2.2), we can perform clustering for different samples. Before clustering, we need to describe the relationship between the different samples based on the energy distance. The technique we use is the multidimensional scaling (MDS) (Kruskal, 1964).

The purpose of the MDS is to provide a visual representation for the pattern of proximities among a set of objects, when only a table of the distances between them is given. When the objects are income distributions, the MDS is used to reduce the matrix of distance to two dimensions, which can be plotted on a two-dimensional coordinate system. Similar functions are placed near each other on the map, while very different functions are placed far from each other. See Algorithm 1 for a brief description of the MDS. Based on the MDS, following the steps of K-means clustering in Algorithm 2, we are able to cluster income distributions from different regions using their energy distance matrix.

In this section, we set up different scenarios to assess the accuracy of hypothesis testing and K-means clustering using the methods described in Section 2. For hypothesis testing, we test cases of two-sample and multi-sample under the null and alternative hypotheses respectively. Different sample sizes are considered to compare the error rates under different circumstances. For K-means clustering, we consider 2-means clustering procedure within different sample sizes.

3.1. Simulation under the null hypothesis

3.2. Simulation under alternative hypothesis

3.3. Simulation: K-means clustering

In this case, we test the accuracy of K-means clustering using energy distance through simulation. We considered the case where in each replicate, 60% of the data are generated from Gamma(2, 0.2), and the other 40% are generated from another Gamma distribution with different rates as displayed in Table 5. For each sample size k ∈ {500, 600, 700, 800, 900, 1000, 1500}, 1,000 replicates of simulations are performed. The error rate in clustering procedure is the proportion of observations that are incorrectly clustered. They are reported in Table 5. It can be seen that the accuracy is related to the sample size as well as the extent how different the two distributions are. With the same alternative distribution, the accuracy of clustering increases with larger sample size. With the same sample size, as the difference between the distributions gets larger, the accuracy of clustering increases.

We apply the proposed methods to the analysis of a China Health and Nutrition Survey (CHNS) dataset for year 2011. The dataset contains information of 4,346 households in 12 provinces. Based on our simulation results, this sample size is big enough to have powerful testing results and clustering results. A brief description of the dataset by province is given in Table 6.

From the Gini coefficient calculated in Table 6, Beijing has the lowest Gini coefficient, which means that Beijing has the most balanced income distribution. On the contrary, Henan has the highest Gini coefficient, which means the income distribution in Henan is the least balanced. Most provinces have similar Gini coefficients, and it is difficult to set a threshold to say whether there are statistically significant differences among these provinces.

4.1. Hypothesis testing results

We calculated the pairwise test statistic in (2.4) for the 12 provinces, giving rise to 66 statistic values. The resulted p-values are visualized in Figure 1. While most p-values are small, indicating a significant difference of the two distributions they correspond to, some p-values suggest that the two income distributions are not significantly different, such as Liaoning and Hubei, Liaoning with Hunan, and Hunan with Hubei.

Also, we do the multi-sample test for all the provinces using (2.6). The energy statistic of this data is calculated to be 87,944,000. Based on the permutation test procedure, the p-value is 0.000999. We therefore reject the null hypothesis, and conclude that not all income distributions are the same in these 12 provinces.

4.2. Clustering results

From the test results of previous analysis, we learned that the income distributions among 12 provinces are different, but smaller subsets of provinces have similar poverty pattern. Therefore, it is of interest to further cluster the 12 provinces. Some exploratory data analysis is performed before clustering. We used the hierarchical clustering methods (Bibby et al., 1979; Johnson and Wichern, 2007) based on the energy distances calculated using (2.1) to see the clustering pattern of different provinces. The result is plotted in Figure 2.

From Figure 2, we can easily find that there are four major clusters based on the energy distance. Therefore, we chose k = 4 for the clustering method described in Section 2.3, and the result is visualized in Figure 3.

From Figure 3, it is found that Beijing and Shanghai are in cluster 1. Jiangsu province is the only one province in cluster 2. Hunan, Hubei, Chongqing, Guizhou, Heilongjiang, and Liaoning are in cluster 3, and Henan, Guangxi, and Shandong are in cluster 4. These clustering results are consistent with explanatory analysis based on the Gini coefficient calculation and testing results. Also, the clustering results reflects actual development of economy in these provinces. Beijing, Shanghai, and Jiangsu have relatively better economic development, while Henan, Guangxi, and Shandong have worse economic development.

We developed nonparametric test and clustering procedure for income distribution in different geographical regions. The test was inspired by the energy distance between distributions (Székely and Rizzo, 2004), and the MDS (Kruskal, 1964) for reducing the dimension of data based on a distance matrix so that an appropriate clustering algorithm can be implemented. In simulation studies, the nonparametric test has size close to the nominal 0.05 level. When the samples are generated from different distributions, or the same distribution with different parameters, the proposed test has strong power in detecting the violation of the null hypothesis, and the power is even stronger than other frequently used testing procedures. In the application to CHNS data, the proposed methods rejected the null hypothesis, and identified four clusters of income distribution, which is robust against choice of initial centroids and consistent with the benchmark hierarchical clustering results. In this work, there are no significant differences between the results of hierarchical clustering and K-means clustering because of the relative small number of provinces we analyzed. If we compare the county level data, however, it is very difficult for us to interpret results form the dendrograms. K-means clustering method is more robust than the HC.

In this work, we are only concerned with using household incomes to calculate the energy distance between different regions. There could be other information sources, such as the value of property, amount of mortgage, and consumptions, that can be used to provide more accurate description of the distribution of income, as the energy distribution provides good measurement of high-dimensional cumulative distribution functions. Geographical information, in addition, can be taken into account in comparing income distributions.

Dr. Hu’s research was supported by Dean’s office of College of Liberal Arts and Sciences in University of Connecticut.