Nov 10
I finished all of the simulations that I was hoping to do, though there may be other qualities that I want to look at. I did simulations comparing the estimates to a simple random sample (SRS), looking at the design effect (ratio of variance to variance of a SRS), looking at the percent error of the estimates, and looking at the percent of simulations the actual population falls in a 95 percent confidence interval.
For the simulations I used the same estimates that I've been using, which take a scale factor and a precision. I used two ranges of scale factors, one from .5 to 2 by .15 and the other with .1,.25,.5,.75,1,1.333,2,4, and 10. For the precisions I ranged the value from 0 to 1 by .1.
Most things that I found through the simulations make sense mathematically to me, but a couple of things on the design effect don't. First, the design effect is highest when the scale factor is around 1. Second, when the scale factor is high, design effect is lowest for precise estimates (which makes sense, since I'd expect precise estimates to lead to precise actual populations). But when the scale factor is low, the design effect is lowest for imprecise estimates (which I can't explain).
I spent a lot of this week working on how to display and talk about the data that I found. I made some plots that I've put in my Plan paper, but I'm not sure how effective all of them are, and have posted them.
The first is a plot of the percent of estimates in 95% confidence intervals. The plots on the left are scale factors (b). The plots on the right are precision (p_c). The bottom plots are the average of each setting.
The second is a plot with the design effects for changing b and p_c on both ranges of scale factors (the plot on the left is the range from .1 to 10, the plot on the right is the one from .5 to 2).
The third file is the design effects as they are affected by b for the larger range. You can see that they peak at about b=1.
The fourth file is how the SRS comparison changes with p_c and b for the range of b from .5 to 2 (SRS score is a number from 0 to 1. When the method gives better estimates on all simulations for a given setting the SRS score is 1, when it always gives worse settings the SRS score is 0).
The last file shows how the SRS comparison changes with just the scale factor.
![[paper clip]](/courses/source/wiki_images/paper_clip_tilt.png)
name last modified size
