All posts by lortie

Grant proposal writing tips

So many great resources out there!
Remember, you can/should do the same topic you are studying in the lab for your systematic. The goal is to generate a second, USEFUL product from the same research. Become an expert in that topic and propose a cool, new research idea based on the research gaps you have identified.



This is just a plain amazing description of effective grant writing.
UNC Writing Center – Give me the money.

Here is a good resource on considerations for the grant title.

Finally, a good primer on how to write a grant app.

NSERC Master’s Funding instructions

To get funding for one year of a Master’s in Science funded by the Canadian Government at $17, 500 (funding call), you write  a single page!  No kidding.  You get one additional page for your bibliography and that is it. Instructions below.

Outline of Proposed Research from NSERC website:

1. Outline of proposed research (maximum of one page);
Bibliography/citations (maximum of one page).
Provide a detailed description of your proposed research project for the period during which you will hold the award. Be as specific as possible. Provide background information to position your proposed research within the context of the current knowledge in the field. State the objectives and hypothesis, and outline the experimental or theoretical approach to be taken (citing literature pertinent to the proposal), and the methods and procedures to be used. State the significance of the proposed research to a field or fields in the health sciences, natural sciences and/or engineering or social sciences and/or humanities, as appropriate.

Ensure you have a very clear title.

If the output of your degree program is an artistic creation rather than a thesis, indicate clearly the research component of your proposed work. Outline the objectives of your research, the context, methodology, and contribution to the advancement of knowledge.

If your degree program does not involve a thesis, a major research paper/essay, or a major research project, remember to outline clearly the research component of the degree.

If you have not yet decided on a specific project, you must still provide a detailed description of a research project that interests you. You are not bound by this project. You may change your research direction or activities during the course of the award. Certain restrictions may apply. Refer to the Canada Graduate Scholarships-Master’s Award Holder’s Guide.

In addition:

Applicants are expected to write their own outline of proposed research independently. Ideas and/or text belonging to others must be properly referenced.
Include all relevant information in the outline.
Do not refer reviewers to URLs or other publications for supplemental information.
In the Outline of Proposed Research, provide the requested information according to the guidelines and format standards outlined in the Presentation Standards.

Presentation Standards

In each section of the Research Portal application, you will find instructions and tips on how to complete your application. All documents uploaded as attachments must be in Portable Document Format (PDF) and must conform to the following presentation standards. See also Transcripts (Attachment) for related presentation standards.

Text must be single-spaced, with no more than six lines per inch.
The acceptable font is Times New Roman (regular, minimum 12-pt.) or a comparable font.
Condensed type is not acceptable.
Set margins at a minimum of 2 cm (3/4 of an inch) all around.
Use a left-justified, standard page layout.
Include your name in the page header on every page.
For multipage attachments, number the pages sequentially.
The maximum number of pages permitted is indicated in each section of the application.
In addition, if you have transcripts or related documentation written in a language other than English or French, you must provide a certified English or French translation.
Applications that do not meet presentation standards may be deemed ineligible, or be at a disadvantage, in comparison with those that respect the presentation standards. Pages in excess of the number permitted will not be assessed by the selection committee.


Test analytics

Hi designers
I popped you each an individual email with your breakdowns.

m = memory
c = concepts
s = synthesis question

Here are some basic analytics:
1. There was no statistical relationship between study time and total test score.

2.  Students scored highest in memory, then concepts, then synthesis  (using the means).  However, this is a bit deceptive.  See plot below. Synthesis answers were the most positively skewed.  A few individuals brought down the mean for that section because they did not complete designing the third question for that section.

3250 test

3. The test is worth 25% so ensure you scale your calculations to that. However, out of 100 points, the median was near 70.

test total

Test next week (Oct 22, 2015)

Reminder, test next week, Oct 22 (our last meeting before Halloween).


Test details
Three parts, 100 points, total worth 25% of total grade in course.
I will give you a few choices for the final section (problem solving).  Duration: 3h.
synthesis of best experimental design to solve problems.

Sample test questions

What is independence from an experimental design perspective? Explain some simple solutions to deal with this issue
What are independent data points from an experimental design perspective and provide at least 2 solutions and give a real world example.
What is random variation from an experimental design perspective and provide an example.
What is the difference between individual variation, inter-individual variation, and intra-individual variation?
What is intra-individual variation, why do we care about and how can we use it?
What is a confounding effect and provide a real world example.
What is the difference between a confounding factor and a 3rd variable?
What is a hypothesis, provide a real world example and explain the difference between a hypothesis and prediction
What are some common pitfalls associated with making good hypotheses and good predictions?
What is a pilot experiment, what are the benefits, and what are the elements of an effective pilot experiment?
What is a null hypothesis and do we ever use them?
What is the difference between a confounding and indirect variable?
What is an indirect measure and why do we ever use them?
What is the difference between mensurative and manipulative experiment? Provide a decision tree for when you would use each (2 reasons for each).
Where do good ideas come from and how does this relate to critical scientific thinking?
The global goals are an inspiration. We need them. Design an experiment that can provide the evidence for one of the 17 goals listed
You are Matt Damon stranded on Mars. Alone. Design an experiment using only potatoes to test whether Martian soil can supply necessary nutrients. You have 120 potatoes with eyes, all the soil you can lug into your space station, lights, and a water production system. You should test not only whether the potatoes can be propagated to support you but whether they are nutritious. You have all the usual equipment to analyze both soils and organic matter an astronaut scientists has access to.
VW lied to the public about TDI cars. The programmed a cheat code that allows the car to pass aircare tests.   However, when driving it produces 4-40x above acceptable emissions. Design an experiment to determine the frequency that this cheat was applied to their TDI models (3 models with TDI engines), if they are set to cheat – the amount they pollute, and the extent that the cheat activates in testing within each model (i.e. is their sensitivity in the cheat turning on).
What is replication and why is it an important consideration is designing effective experiments?
What is pseudoreplication and what are the common sources?
When is pseudoreplication not an issue in designing experiments?
What are cohort effects?
What is a randomization sampling design?
Can you use random and regular sampling concurrently in an experiment?
What are the different sampling methods available in designing sample collection?
Why do you need to randomize the order that you treat replicates?
What are some tools that you can use to decide on how the extent that you replicate?
Why are most scientific study/findings are false?
What is an effect size?
What is an effect size and how did they influence the value of the finding in science?
What is the difference between a Type I and II error. Provide a real-world example following your definitions.
What are the different types of control and do we always need controls?
What is a procedural control?
What is a placebo effect? Explain with reference to types of controls.
What is the difference between factor and levels?
What is a blind or masked procedure, why do we do them, and what are the different kinds?
Calculate the total number of sample units needed for the following two experiments. Orthogonal .. and an unbalanced/incomplete factor design
p87 figure
p89 figure
Why do we sometimes use blocks and sometimes use paired designs in experiments?
What is the primary advantange of a paired design in an experiment?
What is a within-subject design and why is not pseudoreplication?
What are the advantages and disadvantages of a within-subject design?
Why are we sometimes forced to do split-plot designs? Explain split-plot designs in your answer as well using a figure.
What is the difference between a nominal, ordinal, and interval scale?
Calibration of scales makes good sense. However, how might you calibrate other measurement devices such as human observers. Develop a calibration plan including an indication how often you would do it.
What is the difference between precision and accuracy? Explain in words and with a figure too please.
You are managing a running race. Design a validation experiment to demonstrate that the system you have devised minimizes bias and inaccuracies. You have a limited budget for equipment but many volunteers.
Intra-observer reliability can be a significant challenge in designing some experiments. Propose three solutions associated with these classes of errors.
Errors in measurement often scale by magnitude. How do you test this phenomenom and solve?
Provide a visual decision tree associated with how you decide on the resolution of the measurements you take in an experiment?
Recording and storing data from experiments also pose challenges. List two challenges, why they are important, and propose solutions.
Why are experiments that are subject to floor or ceiling effects a problem?
What is the difference between sensitivity and specificity in measurements?
What do you really need to know to most effectively interpret the outcome of an experiment/test even if you know the specificity and sensitivity? Explain and provide a real-world experiment.
You have been told you have a marker in your blood associated with arthitis.   You have completed an experimental design course. What do you need to know to effectively interpret the likelihood that you will soon develop arthitis?
How do decide between two versus many levels for a given factor in designing an experiment? Use figures to show how you decide and provide a brief explanation in text form too. Make a general recommendation on the most likely design to best capture variation across many levels but still estimates variation/protects against drop-outs too.
Subsampling wthin a set of populations or across samples is a solution to solving challenges we sometimes face in designing experiments. Explain how you decide on what extent to subsample, describe trade-offs and when it is appropriate at some levels, and use a figure to illustrate what you are describing as well.
It is sometimes useful to unbalance replicates across groups or levels for ethical or practical reasons. Provide a set of guidelines including a checklist of considerations for researchers.   Clearly explain and show when you want to unbalance in favour of controls versus treatments and the converse.
Explain the most appropriate use for sequential sampling in designing experiments.
Explain stratified sampling and the gold standard in using it to sample heterogeneous populations/systems.
What is a latin squares design? Explain and include a short table listing its strengths and weaknesses.
Covariates can interact too. No surprise. Provide an example set of plots showing statistical interactions between a factor (continuous) and covariate (continuous) and no interactions and another set showing interactions/no interactions for a factor (two levels only) and another factor (three levels). Explain each set of plots in a brief figure legend.
A version of question 6.2 but for sleep and a study aid.
Provide a plot showing Simpson’s paradox for male versus female applications to a university.
Test a few terms from the biomedical literature.
Provide a personal version of a flowchart from your time training in ‘best experimental design’ (BED).


Power & effect sizes

Here is the power and effect size material I mentioned in class. It will really help you answer the test questions we prepped in class and capture the ideas more effectively.

Statistical power analysis.

Online power calculators: many. Here is a useful one. Many more though. You can use them to calculate power or to calculate the sample size you will need given your estimate of power.Unknown

Effect size
Super important idea. Read up online.
Here is great paper why it matters!
Why most published research findings are false.


Welcome to experimental design: lecture but NO labs first week of term

Welcome to experimental design.



The primary purpose of this course is to provide you with useful instruction on how to design experiments AND give you a chance to dive deep into a research topic of your choice in doing so!

Everything you need is here on the blog.

Here are the critical innovations in this course that were intended to  make your life better.

  1. This blog. Everything here, updated weekly.
  2. Only ONE lecture test and no final exam.
  3. All other evaluation tools designed to give you set of useful products you care about.