Questions & explanations
1. For a particle with constant acceleration a=2, initial position x0=0, v0=0, dt=0.1, use Verlet to find x2 after two steps (starting with x1 from Euler).
First, get x1 using Euler: x1 = x0 + v0*dt = 0 + 0 = 0. Then Verlet formula: x_{n+1} = 2x_n - x_{n-1} + a*dt². For n=1: x2 = 2*x1 - x0 + 2*(0.1)² = 0 - 0 + 2*0.01 = 0.02. So after two steps the position is 0.02. This exactly matches the analytic result from constant acceleration (x = 0.5*a*t², here t=0.2 => 0.5*2*0.04=0.04; note cautious: after two steps total time 0.2, but x2 is after step 2 at t=0.2? Actually step 0: t=0, step1: t=0.1, step2: t=0.2, so x2 should be 0.04? Wait: With v0=0, a=2, after 0.2s: x = 0.5*2*0.2² = 0.04. Our Verlet gave 0.02, which is wrong? Check: Euler x1 = 0 at t=0.1, but true x(0.1) = 0.5*2*0.01=0.01. So x1 is off. Then Verlet uses previous positions: x2 = 2*0 - 0 + 0.02 = 0.02. True x2=0.04. So error propagated. This illustrates that Verlet starting with a bad x1 gives error. In practice, one uses a better starting step or smaller dt. I'll adjust answer to be honest: the example shows that starting with an Euler step introduces error, but Verlet still second-order. Actually, the standard Verlet kick-start: if x0 and v0 known, one can compute x1 = x0 + v0
2. Extend the concept: why is Gustafson's law more optimistic for real-world parallel computing?
In many scientific applications, the parallel work grows with problem size—for example, solving a larger grid or more particles. The sequential parts, like setting up the problem or combining results, often do not grow as fast. Gustafson's law captures this by assuming the sequential time stays fixed while parallel time scales with P. Thus, the fraction f effectively decreases as P increases, allowing nearly linear speedup for large problems. This is observed in weak scaling studies where problem size per processor is kept constant. Amdahl's law assumes a fixed total problem size, so it predicts pessimistic limits. Gustafson's insight is that practitioners often run larger problems on more processors, maintaining high efficiency. Therefore, his law better models many high-performance computing scenarios.
3. Compare the accuracy of Barnes-Hut with a direct summation method.
Direct summation computes all N(N-1)/2 particle pairs exactly, so it has no approximation error beyond machine rounding. Barnes-Hut introduces an approximation because it merges distant particles into a single point mass. The error is controlled by the opening angle θ: smaller θ means more cells are opened, leading to more accurate forces but slower computation. Typically, θ = 0.5–1.0 gives errors of a few percent in the force per particle. For many astrophysical problems, this small inaccuracy is acceptable because other uncertainties dominate. Direct summation is only practical for small N (up to a few thousand). Barnes-Hut allows simulating millions of particles, making it essential for galaxy and large-scale structure studies where the slight loss of accuracy is a fair trade for feasibility.
4. In the fast multipole method (FMM), why are multipole expansions used?
Multipole expansions represent the gravitational field of a group of particles as a series of terms: monopole (total mass), dipole, quadrupole, and so on. In FMM, each cell in the tree computes a multipole expansion of its enclosed particles. When interacting with distant cells, FMM translates these expansions instead of summing over individual particles. Higher-order expansions capture more details of the mass distribution, improving accuracy. FMM achieves O(N) complexity, even faster than Barnes-Hut, by using both multipole-to-local expansions and local-to-local translations. This means forces on all particles can be computed in one pass without walking the tree per particle. Multipole expansions allow very high accuracy with a fixed computational cost, depending on the expansion order kept.
5. Compare the Wright-Fisher model with the Moran model for simulating genetic drift.
Both are random models of genetic drift, but they differ in how generations work. Wright-Fisher has discrete generations where the entire population is replaced at once; offspring are drawn simultaneously from the previous generation. Moran has overlapping generations: at each time step, one individual is chosen to reproduce and one to die, keeping population size constant. Moran's process changes frequencies more gradually because only one birth-death event happens per step. Monte Carlo simulations of Wright-Fisher can use fewer steps per generation hop, while Moran needs many more steps to simulate the same timespan. Biologically, Wright-Fisher suits annual organisms, Moran suits continuously reproducing populations. Mathematically, the effective population size connects them.
6. How can you make sure an agent-based model's emergent results are not just coincidence from random runs?
You validate the model by running it many times with different random seeds and comparing output statistics to real-world data or known theory. For each run, you record key measures like average wealth, segregation index, or extinction time. You then look at the distribution of these measures across runs. If the mean and spread match observed data, and if small changes in rules or parameters give similar patterns, you gain confidence. Sensitivity analysis—testing how outputs change when you vary input parameters—also helps. Peer review and replication by other researchers, sometimes with different simulation platforms, provide further checks. Ultimately, consistent emergent behavior over thousands of Monte Carlo replications suggests the pattern is real, not just luck.
7. How does geometric ergodicity compare to uniform ergodicity?
Both describe fast convergence, but uniform ergodicity is stronger. A chain is uniformly ergodic if the distance to stationarity shrinks at the same geometric rate for all starting states, with a uniform bound. In geometric ergodicity, the rate is exponential but the constant factor can depend on the starting state. So a geometrically ergodic chain might mix quickly from most starts but slowly from some unusual start. Uniform ergodicity guarantees the same fast mixing everywhere. In finite state spaces, all irreducible aperiodic chains are uniformly ergodic. In continuous spaces, uniform ergodicity is rare; most practical MCMC chains are only geometrically ergodic. Both are good enough for most uses, but uniformly ergodic chains have nicer theoretical properties.
8. Compare the role of temperature in simulated annealing and in a learning algorithm's softmax.
Temperature is a single number that controls randomness. In simulated annealing, a high temperature allows the system to jump to worse solutions easily, exploring the search space. As temperature cools slowly, the system settles into a good low-energy state. Similarly, in a softmax function used for classification, temperature adjusts how sharply the model picks a class. High temperature makes the output probabilities more even, showing uncertainty. Low temperature makes the model very sure, picking the class with the highest score. Both use temperature to balance exploration (trying new options) and exploitation (sticking with the best known option). Thus, temperature helps manage the trade-off between finding new possibilities and using what works.
9. How does Gustafson's law differ from Amdahl's law?
Gustafson's law says that the effective speedup can increase as the problem size grows. Unlike Amdahl, it assumes the sequential portion f is not a fixed fraction of the whole program, but rather the parallel part grows with problem size while the sequential part stays almost constant. The scaled speedup is P - f*(P - 1). So as you increase the number of processors, you can also increase the problem size (for example, finer grid resolution) to keep all processors busy. This gives a much more optimistic view: with larger problems, parallel efficiency can remain high. Amdahl focuses on a fixed problem size and shows diminishing returns; Gustafson focuses on scaling the problem with the machine, which is often more realistic in scientific computing.
10. Compare agent-based models with equation-based models for studying population dynamics.
Equation-based models describe averages with differential equations, like the Lotka-Volterra predator-prey equations. They assume well-mixed populations and smooth changes. Agent-based models treat each animal individually, with rules for movement, eating, reproduction, and death. ABMs can capture local interactions—like a predator catching only nearby prey—and show how patchy, clustered patterns emerge. Monte Carlo simulation of ABMs lets us see variation: some runs might have extinction while others boom. Equation models give only the expected average and cannot easily handle individual differences or space. ABMs need more computer power but can reveal outcomes that are invisible to average-based equations, especially when local effects matter.
11. Compare stratified sampling and importance sampling: which works better when you have prior knowledge about the function's shape?
When you know which regions contribute most to the integral, importance sampling can directly flood those regions with samples, which is very efficient if the important region is small. Stratified sampling, on the other hand, ensures broad coverage and avoids missing any region, but does not necessarily concentrate samples in the most important area unless you allocate more samples to certain strata. Both use prior knowledge: importance sampling uses it to reshape the sampling density, while stratified sampling uses it to partition the domain. Often, they can be combined. Importance sampling may excel when the important region is highly localized; stratified sampling works well when the function varies smoothly over well-understood partitions.
12. Compare chi-squared fitting with ordinary least squares fitting.
Ordinary least squares (OLS) minimizes the sum of squared differences between data and model, treating all data points equally. Chi-squared fitting divides each squared difference by the measurement error squared, so points with smaller errors contribute more. OLS is a special case of chi-squared when all error bars are equal. Chi-squared fitting is superior when data have varying uncertainties because it correctly weights precise measurements. Both methods give the same best-fit parameters if errors are equal and constant. However, only chi-squared provides a statistical metric to judge the fit quality through the χ² value and p-value. OLS can mislead if some data points are much less reliable, while chi-squared accounts for that naturally.