Questions & explanations
1. Give an example of a continuous function with φ(0)=1 that is not positive definite, and explain why it cannot be a characteristic function.
Consider φ(t)=1/(1+t^2). This function is continuous and φ(0)=1. But it is not positive definite. For instance, take n=2, t1=0, t2=1, z1=1, z2=1. The sum is φ(0)+φ(1)+φ(-1)+φ(0)=1+0.5+0.5+1=3, nonnegative. However, a known necessary condition for a characteristic function is that |φ(t)| ≤ 1, which holds. But positive definiteness fails? Actually, 1/(1+t^2) is the characteristic function of the Laplace distribution? No, the Laplace distribution has characteristic function 1/(1+t^2). Wait, that is correct: the Laplace distribution with density (1/2)e^{-|x|} has characteristic function 1/(1+t^2). So it is positive definite. I need a different example. Consider φ(t)=e^{-|t|^3}. This is continuous, φ(0)=1, but it is not positive definite because the function is not a characteristic function (the corresponding distribution would have negative variance? Actually, e^{-|t|^α} is a characteristic function only for 0<α≤2. For α=3, it is not positive definite. For n=2, t1=0, t2=1, z1=1, z2=1: sum=1+e^{-1}+e^{-1}+1=2+2/e≈2.736, nonnegative. But positive definiteness requires all such sums to be n
2. Derive the LMP test for H0: θ=0 vs H1: θ>0 based on a single observation from a distribution with density f(x|θ)=θx^{θ-1} for 0<x<1 (Beta(θ,1)).
The likelihood is L(θ)=θ x^{θ-1}. The score function is S(θ)=∂logL/∂θ = 1/θ + log x. At θ=0, the derivative of the power function is proportional to E[S(0)|θ=0] under H0? Actually, the LMP test rejects for large values of the derivative of the log-likelihood at θ0. For θ0=0, S(0) is not defined because 1/θ diverges. So we consider the limit as θ→0. Alternatively, use the fact that the LMP test is based on the score statistic evaluated at θ0. But here θ0=0 is a boundary. Instead, we can use the approach of maximizing the slope of the power function. The test statistic is T = -log x. Under H0, θ=0 gives f(x|0) is improper? Actually, Beta(θ,1) as θ→0 gives density ~ θ x^{θ-1}, which tends to 0 for x<1? This is tricky. Perhaps a better example: For normal mean with known variance, the LMP test for one-sided alternative is the same as UMP test.
3. For a Poisson(θ) sample, find the UMP unbiased test for H0: θ=2 vs H1: θ≠2 at α=0.05 with n=1.
The Poisson(θ) family is exponential with T = X. The UMP unbiased test rejects H0 when X ≤ c1 or X ≥ c2, where c1 and c2 are chosen so that P(X ≤ c1 | θ=2) + P(X ≥ c2 | θ=2) = 0.05 and the test is unbiased. For n=1, X ~ Poisson(2). From tables: P(X=0)=0.135, P(X=1)=0.271, P(X≥5)=0.053, P(X≥6)=0.017. Choose c1=0 and c2=6 gives size 0.135+0.017=0.152 >0.05. Try c1=0, c2=7: size 0.135+0.005=0.140. To get size 0.05, we need randomized test. For exact size, use c1=0 and c2=6 with probability γ such that 0.135 + γ*0.053 = 0.05? No, that's not correct. Actually, we need to find nonrandomized test: c1=0 gives left tail 0.135, too big. So maybe c1= -1 (no left rejection) and c2=7 gives size 0.005, too small. So no nonrandomized test of exact size 0.05 exists; a randomized test is needed. The UMP unbiased test may be randomized.
4. Check whether φ(t)=cos(t) is positive definite. Can it be a characteristic function?
The function cos(t) is not positive definite. For example, take n=2, t1=0, t2=π, z1=1, z2=1. Then the sum is φ(0)*1*1 + φ(π)*1*1 + φ(-π)*1*1 + φ(0)*1*1 = 1 + (-1) + (-1) + 1 = 0, which is nonnegative. However, for n=2, t1=0, t2=π/2, z1=1, z2=1, the sum is 1+0+0+1=2, nonnegative. But consider n=3 with t1=0, t2=π/3, t3=2π/3 and z1=1, z2=1, z3=1: the sum is 3 + 2(cos(π/3)+cos(2π/3)+cos(π/3)) = 3+2(0.5-0.5+0.5)=3+1=4, nonnegative. Actually cos(t) is positive definite? Wait, cos(t) is the characteristic function of a random variable taking values ±1 with equal probability? Actually, the characteristic function of a Bernoulli(1/2) is cos(t). So it is positive definite. The test with n=2, t1=0, t2=π gives 0, which is okay. So cos(t) is positive definite and is a characteristic function.
5. Give an example where Wald's equation is applied to compute the expected total number of customers in a queue.
Suppose customers arrive at a service counter with interarrival times that are independent exponential with mean 1 minute. The server takes a random service time S with mean 2 minutes, independent of arrivals. Let N be the number of customers served during a busy period of length T. If T is a stopping time with E[T] finite, then by Wald's equation, the expected total number of customers served is E[N] = E[T] / E[interarrival time] = E[T] / 1 = E[T]. But also, the expected total service time is E[N] E[S] = 2E[T]. This must equal E[T] because the server is busy the whole period, so E[T] = 2E[T] implies E[T]=0, which is a contradiction unless the busy period is infinite. This shows that Wald's equation must be applied carefully with stopping times.
6. Give an example of an aperiodic, irreducible, positive recurrent Markov chain and compute its limiting distribution.
Consider a three-state chain with transition matrix: from state 1, go to 2 with probability 1; from state 2, go to 1 with probability 0.5 and to 3 with probability 0.5; from state 3, go to 2 with probability 1. This chain is irreducible (all states communicate) and aperiodic (state 2 has a self-loop? Actually no, but period of state 1 is 2? Let's check: state 1→2→1 has length 2, but also 1→2→3→2→1 length 4? Actually need aperiodic: state 2 can return in 2 steps (2→1→2) and also in 1 step? No self-loop, so period is 2? Better example: add a self-loop at state 2 with probability 0.1. Then aperiodic. Solve πP=π: π1 = 0.5π2, π2 = π1 + 0.5π2 + π3, π3 = 0.5π2. Summing gives π1=0.2, π2=0.4, π3=0.4. Limiting distribution is (0.2,0.4,0.4).
7. Use the Borel-Cantelli lemmas to prove that if X_n are independent and X_n converges to 0 in probability, then X_n converges almost surely to 0.
Convergence in probability means for each epsilon, P(|X_n|>epsilon) -> 0. But this does not guarantee almost sure convergence. However, if we have a subsequence n_k such that sum P(|X_{n_k}|>epsilon) < infinity, then by first Borel-Cantelli, |X_{n_k}|>epsilon occurs only finitely often almost surely. For independent X_n, we can choose a subsequence where probabilities decay fast enough, and then the whole sequence converges almost surely because the tail of the subsequence controls the whole sequence. Actually, a standard result: if X_n -> 0 in probability, there exists a subsequence converging almost surely. But for the whole sequence, we need additional conditions like monotonicity or use of the second lemma.
8. What is a Bayesian instrumental variable analysis and when is it used?
Bayesian instrumental variable (IV) analysis is used when there is an unmeasured confounder between treatment and outcome. An instrument is a variable that affects treatment but not the outcome directly, except through treatment. In a Bayesian IV model, you specify a joint model for the treatment and outcome, with the instrument as an exogenous variable. For example, in studying the effect of education on earnings, you might use college proximity as an instrument. Bayesian methods allow you to incorporate prior information about the strength of the instrument and provide a posterior distribution for the causal effect, which is more robust than frequentist IV in small samples.
9. How does Bayesian multiple imputation differ from standard multiple imputation?
Bayesian multiple imputation is a fully Bayesian procedure where the imputation model and analysis model are combined in a joint model. Standard multiple imputation often uses a two-step approach: first impute missing values using a model, then analyze each imputed dataset separately and combine results using Rubin's rules. In the Bayesian approach, the imputation and analysis happen simultaneously in one MCMC, so uncertainty is fully integrated. For example, in a Bayesian linear regression with missing covariates, the missing values are imputed within the same MCMC that estimates regression coefficients. This avoids the need for separate imputation and pooling steps.
10. Compare the robustness of the Huber M-estimation based test with a nonparametric test like the Wilcoxon signed-rank test. Which is more efficient under normality?
Both tests are robust to outliers, but they differ in efficiency. The Huber M-estimation based test is designed to be nearly as efficient as the t-test when data are normal, often achieving 95% efficiency. The Wilcoxon signed-rank test is also robust but has lower efficiency (about 95% relative to the t-test for normal data, but actually it's around 95.5% for large samples). However, the Wilcoxon test does not assume a specific distribution and is simpler to compute. The Huber test can be tuned to balance robustness and efficiency, while the Wilcoxon is fixed. In practice, the Huber test may be preferred when normality is approximately true but outliers are a concern.
11. What does the monotone class theorem for functions state and how is it used?
The monotone class theorem for functions says that if a vector space H of bounded functions contains the indicator functions of a π-system and is closed under monotone limits, then H contains all bounded measurable functions with respect to the sigma-algebra generated by that π-system. It is used to prove that a property holds for all measurable functions by first checking it on simple functions from the π-system, then using monotone limits to extend to all bounded functions. For example, to prove Fubini's theorem, one first shows it for indicator functions of rectangles, then uses the monotone class theorem to extend to all non-negative measurable functions.
12. Explain how Bayesian methods can estimate the average treatment effect (ATE) in observational studies with confounding.
Bayesian methods estimate the ATE by modeling the outcome conditional on treatment and confounders, then averaging over the confounder distribution. For example, you might fit a Bayesian linear regression with treatment and confounders as predictors, then compute the posterior of the treatment coefficient. Alternatively, you can use Bayesian propensity score matching, where the propensity score (probability of treatment given confounders) is estimated with a Bayesian logistic regression, and then matched pairs are analyzed. The Bayesian approach provides full posterior uncertainty for the ATE, including the uncertainty from the propensity score estimation.