schedule
Jan 24 - Feb 7 : data analysis with Jim and Matt
your groups
- Group 1: Roy, Melanie, Julie
- Group 2: Leo, Nate, Daniel, Rebecca
- Group 3: Patrick, Billy, Moses, Liz
- Group 4: Gabriel, Kristen, Jesse
what to expect
- Jan 24'th: Jim shows the math; some group data collection (time allowing).
- Jan 31'st: Excel demo; more group data collection; some open time.
- Feb 7'th: Matt shows sample results; more open time.
stuff you should know by the end
- mean and standard deviation (ó)
- normal distribution, especially 68, 95, 99.7% at ±1ó , 2ó , 3ó ,
- average of N similar results : óN = ó1/sqrt(N)
- error bars and how to report a measurement
e.g. "With 95% confidence we find ... " or "At 2ó we find ...
- hypothesis testing, the null hypothesis (H0), "fail to reject" language
- population vs sample; correlation vs causation; observe vs experiment
- where you can go wrong
- data before hypothesis
- misleading graphics - units, labels, proportions, ...
- how to process the numbers and draw the plots
- .csv (comma separated value) text files and other compatability issues
- Excel - tables, plots, analysis
- Mathematica - powerful but complicated
- online tools (see the resources page)
- what a lab write-up looks like: description, method, data, results
your assignment
... has two parts. For each, your group should make lab write-up
either as a web page or a powerpoint-like presentation, complete
with the appropriate table of data and graphics with error bars.
We'll discuss more of what's expected in class,
- Measure the distance from the science building to the campus center.
- Test the following hypothesis: "There's a better than even chance that at least one five will come up when I roll these four dice."
Feb 14 - Feb 21 : Measure the acceleration of gravity, g
your groups
- Group 1: Roy, Gabriel, Nate, Rebecca
- Group 2: Leo, Liz, Jesse
- Group 3: Daniel, Patrick, Kristen, Melanie
- Group 4: Billy, Moses, Julie
what to expect
- Feb 14:
- Introduction, discussion of systematic (vs. random) errors
- some time in groups to plan strategy, etc.
- Feb 21:
- Probably more open group time
- but come to class in case there are any announcements
stuff you should know by the end
- What is the difference between a random error and a systematic error?
- What is the best way to reduce each kind of error?
- What sorts of error-reduction-techniques start to be a bit non-kosher?
- How accurately can one actually measure
g
using your technique?
your assignment
...is simple: measure the acceleration of gravity,
g
, using some
procedure to be decided on by your group. Since the point here is to get you
to think about and correct systematic errors, make sure you do some of both
and make that clear in your write-up. You'll want to read
the background
essay I wrote up to get you thinking in the right direction. Then you'll want
to spend some time as a group thinking seriously about the best way to do this.
Figure out what materials you need; we should have most of them already, and
we'll arrange to get other things you think you need (within reason).
Some general-interest comments on the lab write-ups...
First off, the write-ups this time were significantly better than the ones from the first lab. They all showed a level of effort that was at least reasonable. That effort is noted and appreciated. I'm going to list a bunch of pervasive or interesting errors/confusions that showed up in the
write ups. Hopefully these will be clarifying and help you understand what
we, the faculty, are looking for in these reports. (...something, by the
way, we're just figuring out for the first time too!) But let me just state
that you shouldn't be too bummed out by the long list of negative comments,
nor the similar ones that are scribbled on your individual reports. It's
only when you put some effort into both the lab and the write-up that it
becomes possible for us to say "aha, you misunderstood this point", etc. So
in a way the long list of criticisms here is a testament to the much improved
level of effort.
The write-ups were typically (perhaps universally) too short. Most groups included at least some prose to lay out clearly the goal of the lab, and then the specific methods/procedures used. But there wasn't enough detail on these. Remember, the goal of any scientific paper is to convince rational readers of your conclusions. If someone reads your paper but (a) can't tell what you did or (b) can't tell how you did it or (c) can't tell how you analyzed your data (etc.), then (if they are rational) they will not be convinced. So in the future, try to lay everything out in a clear enough way that they will be convinced!
All the groups included their "raw data" in some form. Typically this meant that their report was about one page of actual report, followed by 3 or 5 or 8 pages of just plain numbers. On the one hand, it's good to include the raw data. But nobody is ever going to look at those pages and pages of
numbers, and even if someone did, they'd be hard pressed to learn anything useful from them. Hmmm... if only there was a way to present data in a way that (a) didn't take up so much space on the page and (b) was easier to look at and actually learn something from, i.e., interpret. Hey! There is such
a thing! It's called a graph! So in the future, it would be good to include
your data in the form of a graph. For example, if you have 500 different
values for some quantity that you measured 500 times, show a histogram (a
plot showing the number of times various values (grouped into some number of
small intervals or "bins") showed up in your 500 trials). Or you could even
just display a regular graph of result vs. "run number". This is often a
very revealing thing to look at -- you can almost always see just by looking
at the graph if there is some kind of funny systematic effect going on. One
of the groups, for example, measured "g" a certain way by doing 5 runs of 20
trials each. Well, their raw data made it look as if the values were all
pretty similar within each group of 20, but there appeared to be rather
large and systematic differences between the values in the 5 different
groups. I don't know if they noticed this; I don't even know if there
really was such an effect, because it's hard to tell for sure just by
staring at 5 pages of numbers. But a graph would reveal this (and save
space to boot).
There was frequent inconsistency with regard to the level of precision or "care". Here's what I mean: lots of people took literally hundreds of
measurements and carefully calculated means and standard deviations and
kurtoses and whatnot with them; this care was typically evident in the
write ups. But often the same write-ups just blazed at light-speed past
statements like "the length of our pendulum was 1.9 meters" or "the ball
was dropped from 3.01 meters". But how did you measure this? What is
the uncertainty? If your fancy statistical analysis suggests that you've
measured "g" with a 1% uncertainty, but you don't tell me how accurately
you know any of the measured parameters that went into the calculations for
g, why should I believe your 1% error bar? As an extreme example, if you
just eyeballed the length of the pendulum (as in, "Well, it looks like a
bit less than two meters... call it 1.9 meters... too bad we don't have
a meter stick to actually measure it") then maybe your value for the length
is good to, say, 10%. But if that's the case, your calculation of g -- which
is based on the length -- is also immediately afflicted by this same 10%
uncertainty (or thereabouts). This goes back to the general goal of
convincing the reader to believe your conclusion. All the care in the
world on some points is simply wasted if you're not equally careful (and
equally explicit in the write-up) about other points.
A few groups seemed overly concerned with (or even overly aware of) the "nominal" value of g -- 9.8 m/s^2 or whatever. There's of course nothing wrong with knowing that other people have measured this and that that's what they got. But that knowledge should play essentially no role whatsoever
in your experiment. Here are some examples: one group claimed that their
experiment was successful because their result was consistent with the
nominal value. It's fine to note that, but that's definitely not what it
means for an experiment to be successful -- indeed, defining success that
way means you can never learn anything new from a successful experiment, and,
conversely, a surprising result is always by definition a failure! So... try
to force yourself to redefine "success" -- for this class at least, a successful experiment is one which would convince a rational (appropriately skeptical!) reader that your result (including its uncertainty, whatever that turns out to be) should be believed.
Every write-up included at least some discussion of possible systematic errors. That's good. But most of them were all talk and no game. In fact, only one group actually identified, analyzed, and corrected for a systematic error. That's not so good (I mean for the rest of you). Talk is cheap. If
you think your experiment is afflicted with some particular systematic error
(and you all did), you can't just mention that and go on. You also can't
say: "well, there's probably some friction, which would be a systematic
error, but it's negligible." How do you know it's negligible? The answer
is: you don't know, not unless you measure it or calculate it or estimate
it or something. So... any time you recognize the presense of a systematic
error, you should make some real, concerted effort to isolate it, to
understand it, to estimate its "size", and, if possible, to correct for it.
One group's report included the statement: "Our results revealed a large amount of systematic error." I think what they meant is that the nominal value (9.8 m/s^2) was outside their 95% confidence interval -- hence, there
must have been some systematic errors. Well, that's probably true. But
I think the statement might also have been based on a confusion about the
nature of systematic errors -- namely, systematic errors are something you
(at most) identify, but you can't really do anything about them; but that's
OK because you get to find out in the end how big they were when you compare
your result to "the truth." Well, that works fine if you already know the
truth. But if you already know the truth, why are you wasting time doing
the experiment? (That's a rhetorical question... probably best you don't
answer it!) Bottom line: the time to analyze and correct systematic errors
is before your experiment is done. If you do a good job of this, you will
actually be able to believe your results. Always carry out your experiment
as if you have no idea what the outcome will be, and try your hardest to be
sufficiently careful that you believe your own results (including the error
bars) more than you believe what you read in some book or magazine.
Be wary of formulas. A couple groups got into trouble by having just a little too much faith in formulas. For example, the period of a pendulum
is only "T = 2 pi sqrt(L/g)" if the amplitude is very, very tiny, and if
the "bob" is infinitely heavier than the string which supports it. Or:
the equation for the distance a dropped object has fallen in time t is
only "d = 1/2 g t^2" if there is no friction or air resistance. I think
all the groups that used formulas like these recognized the possibility
that there might be systematic errors associated with measuring, say, "d"
and "t" (e.g., reaction-time delays). But several groups didn't seem to
recognize that, in addtion, the equation itself might simply be wrong,
because the equation is derived under some assumptions which weren't
applicable to the actual situation at hand.
Only one group looked at the AmJPhys paper I referenced. =( But that one group got, I think, a lot out of it. Let that be a lesson to the rest of
you! If you're stuck or confused or looking for good advice, it's not at
all crazy to see how people have done the thing before and learn something
from their attempts.
Feb. 28 & Mar. 4: Juglone extraction and assay with Todd
Your groups
- Group – Jesse, Billy, Melanie
- Group – Rebecca, Liz, Moses, Gabe
- Group – Patrick, Nate, Julie, Kristen
- Group – Leo, Daniel, Roy
Things you should know
- Chemical bonds: covalent bond, polar covalent bond
- How polar covalent bonds relate to molecular polarity and hydrogen bonding
- Whether a particular solvent is polar or non-polar
- Whether a particular compound is soluble in a given solvent
- The meaning and purpose of performing an extraction
- How Thin-Layer Chromatography (TLC) works
Assignment
- Attempt to extract juglone from ground black walnut hulls using a particular solvent
- Use TLC to see if your extract contains juglone
- Test whether black walnut hull extract or pure juglone will inhibit growth of tomato seeds*
- Write a laboratory report describing your methods, experiment, and results - refer to the hand-out I distributed on writing a lab report
- The lab report is due April 4th
Notes on analysis of experimental data
For this experiment, I hope that all of the seeds grow, and that some of the treatments (e.g., high and low concentrations of juglone) result in seedlings that don’t grow as tall as the control seedlings. If we do get such results then one way to analyze the data is with an Analysis of Variance (ANOVA - see below).
Another possibility is that any plants that grow will be of similar size, but that the different treatments affect germination differently, so that some treatments contain lots of plants, and other treatments contain only a few plants that grew. How would you analyze this type of data?
Analysis of variance – ANOVA
The point of the ANOVA is to test for significant differences between means. This analysis partitions the total variance in the data set into:
- Variation within a group - due to random variation
- Variation between groups - due to differences between means, i.e., changes due to the different values of the independent variable, meaning the different treatments used in the experiment
If more of the total variance is due to differences between groups, rather than to variation within a group, then there’s a high probability that the means are from different populations – that the treatments used had an effect on the dependent variable – and the null hypothesis (H0) is rejected.
In our experiment, what is the independent variable? It’s the different chemicals applied to tomato seeds. And the dependent variable? Well, we expect to measure plant height as the dependent variable. Is this a continuous or discrete dependent variable?
- Our different treatments, or different levels of the independent variable, are the solutions we applied to the seeds:
- water (control)
- low dose of pure juglone in water
- high dose of pure juglone in water
- acetone extract of walnut hulls, in water
Once the seeds germinate I’ll pick a day and measure their height. Then you can calculate mean plant height for each treatment. You’ll probably find that the means are not the same. But is the difference in mean plant height due to random variation (experimental error), or some systematic effect of the independent variable (treatment effects)? We can use an ANOVA to try and answer this type of question.
Using ANOVA
To analyze the result from our experiment we will use a ‘1-way ANOVA’, or a ‘one factor ANOVA’. Our ‘one factor’ is the different chemicals applied to the seeds. A two-factor, or two-way ANOVA would add another variable. For example, if we wanted to test for an interaction between juglone and soil composition, we would apply different concentrations of juglone to seeds in different types of soil.
But for our one-way ANOVA the null hypothesis is that all of our calculated population means, the treatment means, are equal. To test this hypothesis ANOVA partitions the total variance (as described above) and calculates an F statistic.
For ANOVA's performed using computer software, the program compares this F statistic with a table of critical values for the F distribution, which gives us the probability of arriving at a particular value for the F statistic. This probability is the p-value.
When the p-value is less than 0.05, and the null hypothesis (H0) is rejected, at least one mean is different from at least one other mean. But the ANOVA can’t tell you where this difference lies – it can’t tell you which means are different from which other means. To find this difference we use multiple comparison tests, or post-hoc tests. One method is to use a series of pair-wise comparisons – say, t-tests. But what risk do you run when you start performing lots of t-tests? You increase your chances of committing a type I error: a true Ho is incorrectly rejected. To control for this risk, multiple comparison tests require a lower p-value for rejecting H0 – i.e., lower than our normal critical value of 0.05 (5%) – where the exact value depends on the number of comparisons you perform.
To summarize, you will use an ANOVA to analyze your data. The software you use to perform the ANOVA will calculate an F statistic and a p-value. If p £ 0.05 you reject Ho. Next you examine the results of the multiple comparison test to determine which means are statistically different from which other means.
Performing ANOVA using JMP
Computers in the computer lab have a program called JMP. I recommend using this software for your analysis. Excel can also perform ANOVA’s, but has no provision for performing multiple comparison tests.
After launching JMP you get a dialogue box called JMP Starter
- click the New Data Table button in this box
- Select Column 1 and re-name it (call it something like Juglone dose)
- Double-click on the header-space next to the top of Column 1 to create Column 2
- Re-name Column 2 something like Plant Height
- Now you need to enter your data
- Juglone dose is a nominal variable – it allows JMP to see into which categories your data are going to be sorted
- Plant Height is a continuous variable – each data point can be any real number (say, plant height in mm)
From the menu at the very top of the screen select Analyze
- Select Fit Y by X
- In the new dialogue box there is a smaller box that says Select Columns
- Select Juglone dose in this window and drag it into the X, Factor window
- Select Plant Height from the same window and drag it into the Y, Response window
- Click OK in the lower right
- The computer will draw a graph and present an ANOVA table and summary data
- From the small red carrot at the top of this new page pull down a menu and choose Compare Means, and then select Student’s t
- The computer will add the results of this multiple comparison test to the graph
March 28 - April 4 : the wonders of biology with Bob
your groups
- Group 1: Roy, Kristen, Liz
- Group 2: Leo, Melanie, Moses, Julie
- Group 3: Patrick, Jesse, Rebecca
- Group 4: Nate, Billy, Daniel, Gabe
what to expect
stuff you should know by the end
your assignment
April 11 - 18 : Correlation & Causation with Jenny
your groups
- Group 1: Billy, Melanie, Gabe
- Group 2: Leo, Patrick, Nate
- Group 3: Julie, Kristen, Daniel, Liz
- Group 4: Roy, Jesse, Moses, Rebecca
SEE:
http://akbar.marlboro.edu/~allisont/correlation.xls
for an example spreadsheet with a scatter-diagram and with the calculation of the correlation coefficient. You may have to cut & paste that url; the link isn't covering the whole address, for some reason. You will need microsoft excel to open the file.
4/18/05, what Jenny wrote on the board (your assignment):
80% group
Statement of possible correlation
Basis for developing correlation
Methods for collecing data for correlation
Data presentation for correlation
- scattergram (labeled axes; figure legend)
- correlation coefficient
Interpretation of correlation analysis
- is there a correlation
- is there reasons to believe there is a causal relationship
- is it significant?
Design an experiment/set of observations to test causality
20% individual
Present one possible correlation to analyze (natural science-oriented)
Due: Monday May 2nd
April 25 & May 2 : Student Presentations
your groups
This portion of the course is not done in groups; you will be presenting your data from one of the previous labs (your choice) in more detail, as well as providing a write-up.
schedule
Presenting on Monday, April 25th are:
- Melanie, Julie, Moses, Daniel, Liz, Nathan and Kristen.
Presenting on Monday, May 2nd are:
- Gabriel, Roy, Billy, Leo, Rebecca, Jesse and Patrick.
last modified Tuesday April 19 2005 7:24am EDT