Let Ω denote an open interval and let F_θ, θ ∈ Ω, denote a one-parameter exponential family of probability distributions; that is, for each θ ∈ Ω,

where ψ is a twice continuously differentiable function on Ω and λ is a non-degenerate sigma-finite measure on the Borel sets of (−∞,∞). It is well known that if X is a random variable with a distribution F_θ, then the mean and variance of X are ψ′(θ) and ψ″(θ), respectively (Lehmann, 1959).

Let ℘₁ and ℘₂ denote populations with independent distributions F_θ1 and F_θ2 , where θ₁, θ₂ ∈ Ω are unknown. A total of t observations are to be taken from the two populations, and the objective of the study is to estimate the mean difference ψ′(θ₁) − ψ′(θ₂) with respect to the squared error loss, using a Bayesian approach.

Let X₁, X₂, … denote observations sampled from the first population, ℘₁, and let Y₁, Y₂, … denote observations from the second population, ℘₂. In the Bayesian framework, it is assumed that X₁, X₂, … are conditionally independent sharing a common distribution F_θ1, given Θ₁ = θ₁. Similarly, Y₁, Y₂, … are presumed to be conditionally independent with a common distribution F_θ2, given Θ₂ = θ₂. Additionally, X₁, X₂, … are conditionally independent of Y₁, Y₂, …, given Θ₁ = θ₁ and Θ₂ = θ₂; and that Θ₁ and Θ₂ are independent random variables with respective prior density functions ξ₁ and ξ₂.

For m ≥ 1 and n ≥ 1, let ℱ_m,n indicate the sigma-algebra generated by X₁, … , X_m, and Y₁, … , Y_n. Then, denotes a sequential design defined as a sequence of indicators D₁, … , D_t, where D_k = 0 if the k^th value is sampled from ℘₂ and D_k = 1 if the k^th value is from ℘₁. The constants, D₁ and D₂, satisfy D₁ + D₂ = 1 where D_k is ℱ_mk,nk-measurable for k = 3, … , t with m_k = D₁ + · · · + D_k and n_k = k−m_k for k = 1, … , t. In the remainder of this paper, we denote m_t and n_t by M and N and ℱ_mt,nt by ℱ_t.

A sequential procedure for estimating the difference between the two population means, μ(θ₁, θ₂) = ψ′(θ₁) − ψ′(θ₂), is the pair , where is the sequential design defined above and μ̂_t = E{ψ′(Θ₁) − ψ′(Θ₂)|ℱ_t}. The Bayes risk incurred by the sequential procedure is defined as

The problem considered is to find a sequential design for which the risk is minimal and to derive its optimality. To this end, Woodroofe and Hardwick (1990) devise a quasi-Bayesian approach to derive an asymptotic lower bound for the integrated risk and propose a three-stage procedure for two normal distributions with a unit variance. For non-linear estimation, Shapiro (1985) adopts an allocation strategy that has shown that the myopic rule is asymptotically optimal. In this context, Rekab (1989, 1992) derives a first-order sequential procedure and asymptotic lower bound for the Bayes risk, and Benkamra et al. (2015) further derive a nearly second-order asymptotically optimal three-stage design. Song and Rekab (2017) extend the former approaches with a three-stage design to obtain first-order efficiency.

In order to yield a minimal error in estimating the function of the parameters from two populations, obtaining a lower bound contributes to a more refined approximation. However, obtaining a closed-form expression of an exact lower bound for the Bayes risk, particularly in the absence of explicitly specified prior density functions, is notably challenging. Rekab (1990) derives the first-order Bayes risk lower bound for the difference between the means with the conjugate priors. Rekab and Tahir (2004) further extend to a second-order lower bound for the Bayes risk. However, their work specifies the priors as a conjugate, in which forms are provided by Diaconis and Ylvistaker (1979).

Now consider the difference between the two populations from the one-parameter exponential family with the Bayes risk. When the conjugate prior is known, the lower bound of the Bayes risk is as follows:

as t → ∞.

The objective of this study is, without an explicit assumption on the conjugate priors as such provided by Diaconis and Ylvisaker (1979), to establish an asymptotic second-order lower bound for the Bayes risk. This result extends the lower bound result of Woodroofe and Hardwick (1990) and further generalizes Rekab (1990), obtained for the difference between the means of two normal populations with unit variance, using the difference of the sample means instead of the Bayes estimator.

The paper is organized as follows. In Section 2, we further describe the preliminary notations and present the main result with an example. In Section 3, the proof of the main result is provided with lemmas and remarks. In Section 4, we illustrate the implementation of the main result with a numerical simulation showcasing the performance of the Bayes risk lower bound. Section 5 concludes with some remarks on the main result and the future direction.

Let R_t(℘) be as in (1.1). Then, it follows that

Furthermore, Lemma A.2 (see Appendix) shows that

and

where for i = 1, 2,

Moreover, if ξ₁ and ξ₂ have compact supports in Ω, it follows from Woodroofe (1985) that

and

where for i = 1, 2,

Hence, the Bayes risk becomes the following.

where

Next,

Thus, (2.3) becomes

In the remainder of this paper,

For the following main result, the simple regularity conditions, as in Woodroofe (1985), are assumed.

If Θ₁and Θ₂have compact supports in Ω, then for any sequential proceduresuch that

then, the Bayes risk of ℘ satisfies the following asymptotic lower bound:

where γ(η)=ψ″(g(η)) with g being the inverse of ψ′ and α₁(θ), α₂(θ), β₁(θ) and β₂(θ) are defined by (2.1) and (2.2).

The proof of Theorem 1 hinges on lemmas provided in Section 3.

The following lemmas are needed for the proof of the theorem.

Let γ(η) be as in the statement of Theorem 1. Then,

and

w.p.1.

Let γ(η) be as in the statement of Theorem 1.

w.p.1.

For any sequential procedurethat satisfies Condition (2.5)

w.p.l. as t → ∞.

In this section, we specialize the outcomes from Section 2 to Bernoulli trials, that is X_i takes values in 0, 1 for i = 1, 2, … with probabilities 1 − θ₁ and θ₁, respectively. Similarly, Y_i takes values in 0, 1 for i = 1, 2, … with probabilities 1 − θ₂ and θ₂, respectively. Here, θ₁ and θ₂ are independent random variables constrained within 0 < θ₁ < 1 and 0 < θ₂ < 1. The prior density functions of θ₁ and θ₂ are denoted as ξ₁ and ξ₂, respectively. Both ξ₁ and ξ₂ are standard uniform distributions, defined as ξ₁(θ₁) = 1 for 0 ≤ θ₁ ≤ 1 and 0 otherwise, and ξ₂(θ₂) = 1 for 0 ≤ θ₂ ≤ 1 and 0 otherwise.

Table 1 displays two columns of Bayes risks where ℛ_t(P) and ℛ_t(O) each correspond to the outcomes of the fully sequential procedure and the optimal sampling scheme as described in Rekab (1990). In Table 1, we denote EF as the rate of convergence or the excess of second-order Bayes risk as follows:

The results indicate that the Bayes risks of the fully sequential procedure and optimal sampling scheme decrease with increasing t. The absolute excess of the Bayes risk comparing the fully sequential procedure to the optimal sampling scheme diminishes. Overall, the Bayes risk of the fully sequential procedure performs closely to the optimal sampling scheme in terms of the second-order excess or rate of convergence. Up to t = 100, the variability gain drives up the rate of convergence; however, it diminishes again as t increases. The numerical simulation confirms that the second-order excess behaves with increasing t under a non-conjugate prior for a density of one-parameter exponential families.

In this study, we address the problem of estimating the mean difference between two populations, ℘₁ and ℘₂, modeled by one-parameter exponential families of probability distributions. The objective was to estimate the difference ψ′(θ₁) − ψ′(θ₂) using a Bayesian approach, with a focus on minimizing the Bayes risk through sequential designs. The central contribution of this work lies in establishing an asymptotic second-order lower bound for the Bayes risk without explicit assumptions on conjugate priors. By extending the results of previous works, such as Woodroofe and Hardwick (1990) and Rekab (1990), we generalized the framework beyond normal distributions with unit variance, offering a more comprehensive approach to Bayesian estimation in the context of one-parameter exponential families. The main result underscores the complexities inherent in Bayesian estimation, particularly in the absence of explicit prior specifications.

Application of the main result to the exponential distribution with nonstandard gamma prior, as well as a numerical illustration with Bernoulli distribution with a uniform prior, are given. The numerical simulation illustrates the second-order lower bound given by the full sequential design with the best optimal design for several values of the sequence size. While the study presents a fully sequential design with Bayes risk, and it is not specified for a stage-wise procedure, there are other designs that are of interest, including the two-stage design and the myopic design (see Terbeche, 2000). Furthermore, in order to refine the approximation of the Bayes risk further, it may be desirable to attain higher-order optimality (see Martinsek, 1983). Last but not least, this study can further benefit from examining the tightness of the lower bound without specifying the conjugate prior.

For i = 1, 2, let μ̂_in = E{ψ′(Θ_i)|X_i1, … , X_in}, where X_{1 j} = X_j and X_{2 j} = Y_j . If Θ_i has a a compact support on Ω, then

where α_in = E{α_i(Θ_i)|X_i1, … , X_in}.