Assume that two engineers jointly conduct an experiment to determine if a particular process change affects the average process yield. One prefers to specify a significance level and selects .05, while the other prefers to take a more modern approach and reach a decision based on the p-value.
(a) If this experiment were conducted a large number of times and if the null hypothesis were true, what proportion of times would the p-value be less than the specified value for the significance level?
(b) Of course, the experiment will be conducted only once, however, and now assume that the null hypothesis is false. Can the probability that the p-value is less than the significance level of .05 be determined? If so, what is the probability? If not, explain why the determination cannot be made.