Why should we care about research design? | What makes a strong research proposal? | 10 steps to good research design | Writing your dissertation, thesis or grant application | Further reading |
Watch the video
You can watch the video of a lecture I gave as part of the APECS Webinar series here:
Ten Steps to Good Research Design from APECS Webinars on Vimeo.
Why should we care about research design?
If you’re an undergraduate or masters student, you will need to undertake a significant piece of independent study. Your dissertation will form a large part of your degree, and it is important that it is correctly put together. Your dissertation or piece of independent research has important added value and has many ‘learning outcomes’. No matter what you go on to do in your later career, being able to put together a proposal, argue its merits, and design a project, will be valuable skills that you are likely to rely on.
PhD students should consider research design early in their first year. The way that you design and plan your research will have significant implications for the success of your project. Although you will probably write the introduction to your thesis last, a well-designed research project should first work through these steps right from the start.
You may also be writing small grant proposals for awards and fellowships to attend conferences or pay for fieldwork. Nowadays, grant writing is a significant part of my job, and I spend a lot of time writing applications for money. So, good research design is important and a vital part of your skills toolbox, and a few successful small grants on your CV is a must for anyone considering an academic career.
The scientific methodology that you use underpins your entire project, and flawed assumptions or a flawed methodology will result in a questionable integrity of your results. Poor scientific methodology can also mean that you are liable to have biased results.
What makes a strong research proposal?
Good research proposals will achieve Closure by showing how impacts and deliverables will answer their ‘Big Question’.
What is research? There are varied definitions, and they include:
- ‘a systematic investigation to discover facts or collect information’ (Collins Gem English Dictionary, 1992)
- ‘a detailed study of a subject, especially in order to discover (new) information or reach a (new) understanding’ (Cambridge Dictionaries online, 2008)
- ‘a process of investigation leading to new insights effectively shared’ (REF consultation document, 2009)
A key factor in all of these definitions is the focus on new and original facts, informationandunderstanding.
Strong research proposals must skilfully combine and blend a ‘Big Question’, with wide implications, impact and importance, with a novel approach and a sound methodology. Good proposals will have a unique question or a new method, and the methods and study area will be appropriate. Finally, good proposals will achieve closure by bringing their proposal full circle, demonstrating how their outputs and deliverables will feed back into the ‘Big Question’.
After Alon, 2009. Project difficulty versus time spent. For a PhD or Post-doc student, projects in the upper right (not too much time, but large gain in knowledge) are best. For an undergraduate or MSc dissertation, projects in the bottom right are best. But no one wants projects in the bottom right – little gain in knowledge but very difficult. Where does your project fit?
It is very important that you consider, as a starting point, whether your research is achievable. Research projects can provide us with varying amounts of information, and vary in size from small to large projects. Different problems are suitable for people to tackle at different stages of their careers (cf. Alon, 2009). Is your project of a suitable difficulty for you to tackle, and does to add enough to our body of knowledge?
The best proposals are appropriate to the career stage and time available of the person concerned. They are concise, clear and complete – not asking to do too much. They have a strong rationale and a wider justification, with a Big Question clearly situated within this. The rationale and wider justification should clearly demonstrate the importance and impact of the ‘Big Question’. Finally, the best proposals are novel, but not too novel. ‘Me-too’ science is likely to not be funded – but so is large, speculative science, that is likely to have unachievable aims and targets.
Whether you are writing a proposal or a thesis or dissertation, you should follow these steps for good research design. You should also take a look at my blog post, “Climate Change Skeptics“, which also talks about research design.
10 steps to good research design
Below, I have put together a list of 10 steps for you to think about when designing a research project. Follow these steps for good research design, and for writing a good grant application or introduction to your dissertation or thesis. Of course, I have only covered this topic briefly here; there are many further resources that you can look at that discuss research design in more detail.
10 steps to a well-designed research project.
It goes without saying that your teacher or supervisor is the real expert here and they should be consulted at every step of the way. Make use of their years of training and expertise. Discuss your ideas with them, and where you want your research to go. The joy of research is that you get to decide what to do and how to do it. But you should check with your supervisor that your methods are appropriate, that your research is relevant (and hasn’t been done before!), and that it is achievable within your timeframe.
Step 1. Why are you doing this research?
The first phase of your research design is to decide what you are doing, and why you are doing it. Many people become so immersed in their project, they cannot see the wood for the trees, and assume that everybody knows why their work is important. This is not the case; you need to be able to explain to non-specialists the importance of your work. Your examiner will be looking to see that you understand the relevance. Whether you are writing a proposal or an introduction to your thesis, you should start with a rationale.
Grants are awarded competetively on the basis of their relevance and importance. Papers are only published if they are relevant. The rationale is probably one of the most important parts of your research design, and you skim over it or ignore it at your peril.
Step 2. Identify the key unknowns
Once you have decided on the broad area of your research project, and you have established a good rationale and reason for undertaking it, you need to read up on previous work. What are the key unknowns and key research questions? What gaps can my research project fill? Write a ‘Wider Justification’ where you explore previous work, but where you identify gaps in knowledge.
If you are doing a PhD or MSc dissertation, you should expect to spend a considerable amount of time at the start of your research project reading the available literature. Make a list of research questions and key unknowns as you come across them. This is a vital step to becoming an expert in your area.
Finally, this stage is imperative to make sure that your research has not already been done!
The best projects are novel, but not too novel (which is risky and difficult – see the graph by Alon, 2009, above), and should avoid ‘me-too’ science. Don’t just jump on the bandwagon because everyone else is doing it. Your project must have clearly defined research questions. Equally, large, speculative science is unlikely to get funded.
Step 3. What is your aim and what are your objectives?
Once you have worked out a rationale for your research, you need to decide on an aim. This is the most important part of your research design, and it should address the key unknowns identified in Step 1 above. Ideally, you should be able to express your aim in one sentence, e.g.,
Aim: to reconstruct the glacial history of the NE Antarctic Peninsula on centennial to millennial timescales.
Your objectives should help you to achieve your aim. You can identify, typically, 3-5 objectives that will each bring you a step closer to your achieving your aim. Ideally, each objective should be associated with research questions so that you are always trying to achieve something new and original. This will also keep your research focussed and on the right lines.
Good aims and objectives usually have the following characteristics:
- Specific, achievable and feasible
- Clear sense of deliverables
- Specific, clear, over-arching research question
- Realistic about methods and timescale available
- Use words like Compare, determine, characterise, explain, quantify, interpret, measure
Alternatively, poor aims and objectives typically have the following characteristics:
- Vague, broad, unspecified titles
- No hypothesis or research question
- Overambitious and not realistically achievable
Step 4. What hypotheses are you testing?
For reasons that are covered in more detail in this blog post, as scientists, we need to test hypotheses. These hypotheses should be indentified by your analysis of previous work and key unknowns. We work within a Research Programme: this means that there are key things that we hold to be true (evolution, plate crustal movement, basic processes of glacier movement), and that there are areas that are continuously under development and being questioned (details of past glacier history). You need to write one or two hypotheses that you will test. A good scientist should attempt to falsify her hypotheses. Your hypotheses should be based on the literature, your identification of the key knowns and unknowns, and should move the science forward.
Step 5. Identify the key deliverables.
What are the key outcomes and deliverables of your research going to be? The deliverables should use words such as, understanding, quantification, conceptual, process, analysis, characterisation, determination. For example:
- An improved understanding of process XX
- Glacier velocity maps
- Process-based conceptual models of process YY
- Quantification of ZZ
Analysis of glacier fluctuations over time (glacier outlines submitted to GLIMS)
These deliverables should enable you to test your hypotheses and achieve your aim. They should be specific and achievable, and help you achieve closure with your ‘Big Question’.
Step 6. Identify key resources
What resources will you need to complete this research project? Will you need to do fieldwork, and if so, for how long? Will you need any specific computer resources, packages, programmes, remotely sensed images, computer codes?
Step 7. Timeframe for research
Simplistic gantt chart (they do not need to be complicated). You should identify the key outputs in your caption. Needless to say, they should match the identified project deliverables.
Your research design should incorporate a realistic assessment of the time committments for each objective. Write a Gantt Chart (you can just do this in Excel or on paper!) that outlines each objective and the amount of time you have available. Work out in detail how much time each objective will take you, and be realistic about whether it is achievable in your time available. Typically, students underestimate the amount of time a specific objective will take them, so be cautious in your estimation.
Step 8. Draw up a work-flow model
An example of a stylised workflow model.
Once you have worked through the steps above, you are ready to put it together into a coherent workflow model. I think these should be included in all dissertations and grant applications, as they clearly set out how the different objectives fit together.
Write the aim at the top, and then the hypotheses beneath it. Include your resources or inputs below this. Then, each in a separate box, outline each objective and the key deliverables associated with this objective. Finally, at the bottom, give your end result; e.g., hypotheses accepted or refuted; a general model or process XX; quantification of YY. This makes it very clear how your research project will fit together, what you will achieve, and how it fits with your aims and hypotheses.
The workflow model should illustrate your deliverablesand thus provide closure for your ‘Biq Question’.
Step 9. Risks and risk mitigation
If you are planning on doing fieldwork, you will need to do a risk assessment and clearly identify hazards and how you will mitigate or prevent them. But you should also be aware of more general risks; do you have the relevant knowledge? Are the resources that you need available? Will the costs change? Risks could include:
- Unreliable exchange rates
- Wildlife hazards (polar bears?)
- Weather (hot / cold / blizzards)
- Environmental hazards and disposal of waste
- Equipment failure
- Not being able to obtain key datasets
- Access to field areas
Step 10. Undertake your research!
Now that you have spent some time carefully planning your research, you are ready to begin. You are going to research something that is important, interesting, and something that you will enjoy. You have written a good research plan, and know that your work is relevant to society and other scientists. Have fun!
Evaluate and refine your research as you go.
Evaluate your work as you go, and be prepared to change your methodology or objectives as you go along, if you find something is not feasible or too difficult. In fact, you may find that your work needs to go full circle if your methods do not work or it turns out to not be feasible! So, your final research design might look something like the figure opposite…
Always remember Occam’s Razor: In the case of several competing hypotheses, the one that makes the fewest assumptions is most likely to be true.
Writing your dissertation, thesis or grant application
Now that you have done some excellent research design, you’re already well on your way to a top grade! So go off and complete your research. Write up your dissertation or thesis, but write your introduction last. Your introduction should follow something like the outline below (although your supervisor, teacher or advisor will give you more advice that is relevant to your project).
If you are writing a grant application, then the format should be similar to that below (check the application details carefully as they might be specificied), but you should include a timeframe for research as well.
- Rationale for the research. Why the work is important.
- Key unknowns and research questions.
- State aim, hypotheses and objectives clearly and succinctly.
- Show the Progamme of Research or Workflow Model.
- Go into each objective in more detail. Pepper them richly with research questions. Be clear about the resources and timeframe available.
- You may want to include a section on risks and risk mitigation, a wider justification of the work, and for grant applications, why it might benefit you personally.
- If you are writing a dissertation or thesis, you might want to give an outline of your thesis chapters here.
Grant applications in particular will need a careful justification or resources. You will need to account for every penny requested! Think carefully through all the things that you will need to cost in.
Here are some excellent articles to help you on your way.
Alon, 2009. How to Choose a Good Scientific Problem. Molecular Cell.
Erren et al., 2007. 10 simple rules for doing your best research. Computational Biology.
Schwartz 2008. The importance of stupidity in scientific research. Journal of cell science.
Rhoads and Thorn 1996. The Scientific Nature of Geomorphology.
Article by Bethan Davies.
Volume 3, No. 4, Art. 2 – November 2002
Encouraging Students to Think About Research as a Process
Mark A. Earley
David de Vaus (2001). Research Design in Social Research. London: Sage, 279 + xvii pages, ISBN (hardcover): 0761953469, US$ 91.00, EUR103.11, £ 63.40, ISBN (paperback): 0761953477, US$ 29.95, EUR33.93, £ 20.86
Abstract: In Research Design in Social Research, David DE VAUS presents a wonderful opportunity for research design students and practitioners to think more about the planning and process of research design. DE VAUS structures his text by presenting introductory research design tools followed by sections on experimental, longitudinal, cross-sectional, and case study designs. Each section presents types within each category, issues in conducting these types of research, and data analysis recommendations. The text is accessible to a variety of audiences, from beginner to experienced, regardless of the discipline. What makes this text stand out as highly useful for the classroom is the author's continued insistence that "there is no right way of developing ideas" (p.23), and that researchers must continue to think about what they are doing rather than blindly doing it.
Key words: research design, experimental research, longitudinal research, cross-sectional research, case study research, research methods
Table of Contents
2. Organization of Text
3. Unique Contributions to Research Design
4. Applications and Audiences for Text
It is no small task to introduce students to the mechanics and artistry of social science research. Most students in introductory courses are new to the experiences of designing, conducting, analyzing, interpreting, and writing about research. They usually call for structure and "answers" to questions they have. They are often looking for "how to do" research. With limited time, many professors are hard pressed to disappoint them, stressing mechanics and "getting the job done." The artistry many of us express through our research is often left out or minimized in the classroom experience. This artistry and thinking about research is exactly what DE VAUS stresses in his text, Research Design in Social Research. 
2. Organization of Text
The book is structured around five main sections: (a) an introduction to research design, followed by four parallel sections describing (b) experimental, (c) longitudinal, (d) cross-sectional, and (e) case study research. Each of the latter four sections discusses types of designs for that particular category, issues in conducting research in that category, and analysis options for that category. The systematic treatment of each category of research stresses their similarities, and at the same time highlights the unique aspects of each. Students are presented with the structure they need when learning new information, and seasoned practitioners can flip to a particular section for a quick refresher. 
As an example, the section on case study designs first gives an overview of thinking about case studies. Next, DE VAUS examines how to define a case. This introduction is followed by a very well-organized discussion of the relationship case studies have to theory: "without a theoretical dimension a case study will be of little value for wider generalization" (p.221). He then goes on to describe how theory can be used in the design process, including a discussion of designing case studies for theory testing, theory building, and understanding a case through theory use. DE VAUS also includes a section on descriptive case studies and their relationship to theory use. Finally, Figure 13.2 in the text lays out an excellent grid relating additional considerations in the case study methodology such as single/multiple cases, parallel/sequential studies, and retrospective/prospective studies. The succeeding chapters in this section of the text cover design issues such as internal and external validity, sampling, access, and ethics, along with a detailed description of data analysis options such as pattern matching, time series, statistical analysis, and analytic induction. 
DE VAUS states in the beginning of the text, "it is erroneous to equate a particular research design with either quantitative or qualitative methods" (p.10). This is important, as it again stresses the importance of first developing a question to address and then designing a study to address that question, without worrying about what label to place on the study (i.e., "phenomenological" or "quasi-experimental"). As the researcher considers various aspects of the study's design, these labels will emerge; the researcher can then go to sources more specific to these traditions for further assistance. This is important to consider when reading the text, as one will not find reference to a variety of qualitative paradigms (other than the case study section described above) such as grounded theory or ethnography. 
3. Unique Contributions to Research Design
The most exciting aspect of DE VAUS' introduction to research design is in his description of the basic process of research. It is in these early phases of learning about research that students are particularly hungry for "rules" and specifics. DE VAUS does an excellent job fulfilling students' needs, with one important and unique caveat. In my experience evaluating texts for class, rarely have I found an author who encourages readers to think about what they are doing with research, as opposed to thinking about how they will carry out the research. Indeed, one basic premise of DE VAUS' text is that "technology has assisted greatly in the conduct of social research. But some of the basic thinking about the logic of research is still missing in research" (p.xvii, emphasis in original). 
Frequently we present students with options in an introductory research course: options for how to write research questions, options for how to select participants, options for how to collect data, and options for how to analyze data. These options are typically geared toward helping students design the mechanics of a study, or how they will carry out the research. DE VAUS consistently encourages readers to think beyond these issues: "how the data are collected is irrelevant to the logic of the design" (p.9, emphasis in original). Readers are encouraged to explore alternatives and thinklogically about their research, so that the suggestions and mechanics DE VAUS presents are not meant to be "rules" for doing research, but rather areas for consideration, as "research design refers to the structure of enquiry: it is a logical matter rather than a logistical one" (p.16, emphasis in original). Thus students are encouraged to reflect on the process of design and the study as a whole, rather than working from step to step to just complete the study. Under this model, specific methods for collecting data are not discussed as "quantitative" or "qualitative" but rather as options for any type of design. The reader will see, for example, that DE VAUS mentions observations (typically considered a "qualitative data collection method") as one type of data used in experimental research (typically a more "quantitative" design). 
As a professor of educational research, and indeed even as an educational researcher, I find the organization of the text to be one of DE VAUS' greatest contributions to the profession. For example, most of the traditional research design texts (e.g., CRESWELL, 2002; CHARLES & MERTLER, 2002; GAY & AIRASIAN, 2000; WIERSMA, 2000) describe internal and external validity in a separate chapter. DE VAUS weaves these issues into each section of his text, covering unique questions and decisions researchers must attend to for each category of research. This helps students understand that validity is not a generic research topic, but idiosyncratic to a variety of settings. Ultimately, DE VAUS creates a reader who is ready to think about, plan, and logically create a research study that he or she will continue to evaluate and reflect on as the study is underway. This is an exciting contribution, as it encourages students and practitioners to remain artists, focusing on the overall journey of research rather than the final destination. My own thinking has changed after reading DE VAUS' text, so that I now spend more time thinking about my research studies as I am doing them. 
4. Applications and Audiences for Text
I am currently in the design phase of a study exploring the cultural aspects of the introductory statistics classroom. Individual classrooms will be observed and videotaped, students and professors will be interviewed, and artifacts such as notebooks, assessments, overheads, and textbooks will be analyzed. DE VAUS' description of case study designs in Part V has guided me as I begin to formalize my description of the study and attempt to focus my data collection and analysis plans. In particular, helping me clarify my final goal, which is to "build of a full picture of [the statistics classroom], its subunits ... and its context" (p.231), help to give me an overall structure to what can often be a complex type of study. 
DE VAUS continues in his treatment of case study design to discuss specific issues in case study design such as internal and external validity, selection and screening of cases, cost, access, and ethics. These sections have been particularly helpful for me for a number of reasons. First, DE VAUS clarifies that case studies are not generally developed to achieve statistical generalizability (i.e., through random sampling of a large number of units), but can strive for theoretical generalizability through replication and the use of a guiding theory or set of theories in the design phase. Further, depending on my accessible population of introductory statistics classrooms (44 in my pilot study), careful attention to the selection and screening of potential cases is important. Initially I worked more toward the (perhaps naive) goal of capturing as many classrooms as I could. My current access challenge is securing instructor permission to be in the classrooms. Looking forward to the ways in which data can be analyzed and the theoretical implications of my work, I have been able to be more clear in my proposal writing – in part because DE VAUS has focused my reflections on the overall purpose of case study research as well as the details faced along the way. 
It is not my place to say who should and should not read this text, but I must encourage those who teach introductory research design courses in any field to consider DE VAUS' text. I currently use the text along with examples of published research studies from fields relevant to my students' majors. For example, a large percentage of my students are special education majors, so I incorporate one or two published articles in special education that fall under one of DE VAUS' four categories (experimental, longitudinal, cross-sectional, or case study). Each article is used to demonstrate not only what a completed study might look like, but also how readers can evaluate a research study to determine if it successfully addressed its questions and to determine if it meets their own needs as background for their own research. These evaluations are done using DE VAUS' text as a guide, with ample time given for students to consider and discuss alternatives to the research that was done. 
As a practicing educational researcher, I also find DE VAUS' text a helpful refresher for the varieties of research I conduct. I have already used various sections of DE VAUS' book to help guide my thinking about the logic of a study I am planning. Each step along the planning path, I now refer to DE VAUS for suggestions and considerations. When planning my own research, DE VAUS' reminder that "there is no right way of developing ideas" (p.23) helps me remember that I am not simply writing a recipe for conducting a research study. 
As a researcher and professor, I have found comfort in DE VAUS' suggestions and consistent writing style throughout the book. It is a highly accessible and reader-friendly text that encourages us to remember what research is really about: the process, not just the product. It is this process that I enjoy introducing to my students in educational research courses, and it is the logic of this process that is the most important lesson I could impart on them. DE VAUS has become, for me, an indispensable partner in my teaching and research efforts through his clear and organized format. I find it especially valuable to have a partner there to remind me to think about what I am doing before I do it, and together we help students develop this thought process. 
Charles, C.M. & Mertler, Craig A. (2002). Introduction to Educational Research (4th Ed.). Boston, MA: Allyn & Bacon.
Creswell, John W. (2002). Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research. Upper Saddle River, NJ: Merrill.
Gay, Lorraine R. & Airasian, Peter (2000). Educational Research: Competencies for Analysis and Application (6th Ed.). Upper Saddle River, NJ: Merrill.
Wiersma, William (2000). Research Methods in Education: An Introduction (7th Ed.). Boston, MA: Allyn & Bacon.
Mark A. EARLEY is currently an adjunct assistant professor in the College of Education at Bowling Green State University, Bowling Green, Ohio, USA. His research interests currently involve statistics education, statistics classroom environments, and student understandings in statistics. He teaches courses in introductory statistics and research design as well as an introductory qualitative research course. EARLEY has also presented qualitative research papers at the Midwest Qualitative Research meeting in Minneapolis, MN and at the 2nd Statistical Reasoning, Thinking, and Literacy Forum in Armidale, NSW, Australia.
Dr. Mark A. Earley, Ph.D.
Dept. of Educational Foundations & Inquiry
Bowling Green State University
Bowling Green, OH 43403, USA
Earley, Mark A. (2002). Encouraging Students to Think About Research as a Process. Review Essay: David de Vaus (2001). Research Design in Social Research [12 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 3(4), Art. 2, http://nbn-resolving.de/urn:nbn:de:0114-fqs020420.