Posted: May 12th, 2015

FACULTY OF HEALTH AND LIFE SCIENCES

Masters Programmes

PP0191 – Systematic Appraisal of

Published Research

Dissertation Guidelines[1] and Support Materials

Contents

Contents list 1

Systematic Appraisal of Published Research 2

A step by step guide

Choosing a topic 2
Research Aims 3
Research Questions 4

Scoping exercise 4

Translating your research question into a search strategy 6
Inclusion & exclusion criteria 8

Quality check list 9

Data extraction & analysis (incl. tables) 11

Findings / Data synthesis 13

The Discussion Chapter – 14

Linking your findings to the wider Health context

Conclusion 14

References & resources 15

Appendix 1 – Outline of Dissertation Chapters 18

Writing tips & submission process

Appendix 2 – Sample form to record searches

Appendix 3 – Gantt Chart

Appendix 4 – Flow chart example

Appendix 5 – Final checklist questions

Appendix 6 – Template for front cover

PP0191 – Systematic Appraisal of Published Research.

PP0191 – Systematic Appraisal of Published Research

A step by Step Guide

Students are reminded that what follows is a deliberately oversimplified account of the Systematic Appraisal (SA) process. In many senses, the dissertation presents you with an opportunity to integrate all previous learning on this course e.g. evidence-based practice, health knowledge and policy, research processes and paradigms and intentions. The tutorial sessions and group exercises in this module are intended to further develop your Systematic Appraisal skills. Before embarking on your own project you will have the opportunity to read a number of Systematic Appraisals completed by Northumbria students on similar courses. You should also take the opportunity to obtain and read a published Systematic Review, which addresses similar concerns to your own (see reading resources for suggestions). Although you are not undertaking primary data collection, you are engaging in RESEARCH (using secondary data).

The stages addressed in the guidebook are designed to enable you to use given examples and then to apply the same principles to your own selected appraisal intentions. The processes described mostly reflect the content and context of the Methodology Chapter of your dissertation. The sections will help you construct your protocol document, and it is one objective of this handbook to support you to completea protocol document.

This whole endeavour will require a disciplined and systematic approach. You will need to manage your time effectively to complete a worthwhile project on time.

You may find the Gantt chart (Appendix 3) useful to manage the stages of your project.

Choosing a Topic

This must be located within the broad field of your discipline i.ePublic Health, Health, Nursing, Social Care, Education.

You are advised to select a topic which is of current relevance to [a] your practice or [b] the national context or [c] of international concern (or all of these things). It has been said in previous modules, there is no substitute for selecting a topic that (at least) interests you, and will be able to sustain your interest over the course of your writing.

Of course, there ought to be a reasonable amount of existing literature about your topic – otherwise you will find little to appraise!

The selected area of interest may be restricted to one country, one age group etc. Decisions regarding to time, place and persons must be must be clearly accounted for (see developing and applying inclusion and exclusion criteria – below).

Who will use the results of your appraisal? How will they use the information? You need to be clear about your SA’s contribution to Nursing / Health practice, policy, methodological and/or theoretical development.

Initial Thoughts: What are your initial options for the Broadsubject of your dissertation?

Research aims

You must be quite clear what your aims for the SA are. The overall aim of most SA projects will begin:

“To collate and critically appraise published evidence in relation to x / y / z &c.”

You are advised to read the critical literature related to Systematic Reviews (see resources and reading at end of booklet) as well as the material which simply describes them. The following questions will help you formulate your aims and arrive at a specific research question or questions.

What is it you aim to investigate in your Systematic Appraisal?
Secondary questions – What (exactly) do you want to learn about the selected issue?

Obviously, the main aim of the review must be reflected in the title of your project.

Aims must be specific. Petticrew and Roberts (2006:29) use the example of a poor

aim, ‘A systematic review of stress at work’ to illustrate the danger of proposing to engage in a field which is too large and impossible to approach (within the time and resource constraints) systematically.

Research Question(s)

Ask a poor question and you will get a poor review (Counsell, 1997). What questions do you need to ask to achieve your aim(s)? Andrews (2003) reminds researchers that questions can take time to develop. You will need to familiarise yourself with some key literature before your question(s) emerge(s). It may be appropriate to discuss your ideas with your student peers.

The terms used in your question will provide direction for your Systematic Appraisal. Andrews (2003:10) talked about the tightness of the ‘aperture of your study’. He used the example of whether the focus is on ‘influence’ (wide aperture), ‘impact’ (smaller) or ‘effect’ (tightest). Measuring ‘effect’ would require appraisal of experimental studies whilst exploring ‘impact’ would enable the appraiser to include studies which examined strategies and processes as well as outcomes. Exploring influences would involve qualitative approaches and could include analysis of policy and cultural practices. At this stage you should be familiarising yourself with the literature in your chosen field.

Scoping Exercise

Your research question and specific aims will suggest certain concepts. Just as in all research, Systematic Appraisal demands that you fully conceptualise your research question.

deVaus (1986) outlined the process of conceptualisation – systematic clarification of the meaning of a research topic. He called this process ‘descending the ladder of abstraction’.

Frequently, in Nursing / Health research, concepts are both partial and contested e.g. they involve [contestable] choices of certain theoretical perspectives / facets / definitions above others.

The choice of particular concepts over and above other [competing] concepts always requires theoretical justification, and by-in-large, you would usually explain why you have chosen to focus on particular aspects of a research question during the introduction section of your work.

The process of conceptualisation – defining, framing, and operationalising the research problematic should always occur very early in the research process – and the results of this exercise will invariably inform your initial literature search.

Source: de Vaus D. (2001), Research Design in Social Research. London: SagePage 5

Translating a Question into a Search Strategy

You now have some specific focus for your Systematic Appraisal and will be nearer to beginning to search for related literature. The construction of a search strategy will obviously be based upon the components of your question.It’s quite usual to revise your intentions as you discover more ideas in the literature.

One useful device to help you think more specifically about your research question (and potential search terms) is the PICO model (CASP 2005). In particular, where the search focuses on intervention (effectiveness) studies, the PICO model will be useful. The ‘people, intervention, comparison and outcomes’ approach encourages the researcher to consider the components of the question.

Example :

‘How effective are critical incident debriefing sessions [Intervention]in decreasing [comparison] Intensive Care Unit nurses’ [people] perceptions of work-related stress [outcomes]?’

Using the PICO model, refine / clarify your initial search ideas and place in the box below.

Population/ People:

Intervention:

Comparison:

Outcomes:

Petticrew& Roberts (2006) emphasised the need for another component – the context in which the intervention is delivered must be considered. It is seldom enough to report that some intervention or another worked without taking into account the wider context. Context in this case might refer to:

Rural – urban;
Developed country – developing country;
Geographical context – e.g., not all findings from Health research in Asia will have global relevance;
Social geographies e.g. Rich / poor; male-female; ethnic dimensions &c.

Pawson (2006) reminds researchers that interventions are always inserted into pre-existing conditions. Critical appraisal of context and its inter-relationships may be an important component of your synthesis and discussion chapters.

Initial Thoughts about Search Strategy

Initial Research Question:
Key Concepts	Possible alternative search terms

After you have decided on search terms, it’s simply a matter of deciding what databases to access e.g.

Web of Knowledge;
Cinahl;
Medline;
NORA (to name but a few)

It is important that you document your search criteria and strategy and also mention any changes that you might make along the way.When you begin searching, it is all but essential practice to list each database searched, the numbers of references or ‘hits’ obtained and the order in which you refine your terms. A form is provided (appendix 1) which you may find useful when conducting your searches.

You will find that you will have far too many papers from the initial searches and you will need to further refine your inclusion & exclusion criteria. A flow diagram is helpful to illustrate the direction of your search pathway (see appendix 4). The funnel analogy is useful to help visualise the move from a wide range (funnel top) of papers which may involve only a scan of the title, to an appraisal of the study abstracts and then through further refinement you retrieve the full published papers. At the much narrower (neck) you will make your final decision on which of the primary research papers (between 6 – 12) you will carry out your detailed systematic appraisal. The final decision can be assisted by the help of a ‘quality check list’ (see section 7 below).

Be warned! This is typically a very lengthy process and you must set aside plenty of time (check Gantt chart appendix 3)

Inclusion and Exclusion Criteria

At this stage you are moving down the funnel mouth. Your search strategies will also have to address other criteria which further refine your material. You will make decisions regarding which studies to include and exclude in your final systematic appraisal. For example, you may wish to confine your search to: England, or the UK,or your home country or a specific time period (e.g. since 1990), a specific kind of research design e.g. qualitative, survey or experimental design. The studies included in the final SA must be written in English (Northumbria regulations for the award of higher degrees).However it may be relevant to draw on papers published in other languages (understood by you) in your literature review, if they are relevant to Nursing / Health and its context.

A list (which is explained and justified) of inclusion criteria is an important part of your methodology chapter. For your chosen appraisal list your inclusion & exclusion criteria in the table below. NB that not all of the following will be of relevance to your study.

Initial thoughts about Inclusion and Exclusion criteria

Possible Inclusion (& Exclusion) Criteria

Why?

Publication language (e.g. English language).

Essential to meet exam regulations for the award of higher degrees at Northumbria University

Publication scrutiny (e.g. Peer reviewed studies)

Essential to meet quality assurance of the primary papers cited.

Geographical context (see above)?

Years of publication?

Study design(s)?

Population of interest?

Intervention type?

Sampling procedures?

Data collection methods?

Outcome measures?

The above exercise, by default, ends up with a preliminary exclusion list. This will be all studies that do not meet your inclusion criteria (and will be tabulated in the appendices of your dissertation).However, you may need to compromise on your inclusion list as you may not find an appropriate number of primary research studies which match the criteria exactly. At this stage you may have between 20 and 30 potentially useful studies. You can develop a rudimentary scoring system based on your inclusion criteriain order to aid the final selection process. The next section offers a template for this process.

Quality check list

The table below provides an example of a checklist to address the quality of studies to be included in the final appraisal. You will need to read and critically appraise all your potential studies to complete this exercise. Each study must be ‘checked’ in this way. Each ‘met’ answer receives a score of one, some ‘met’ answers must be achieved i.e. no compromise possible. These are asterisked in the criteria column.

The example below relates to our earlier example: ‘How effective are critical incident debriefing sessions in decreasing Intensive Care Unit nurses’ perceptions of work-related stress?’

Criteria

Met

Not Fully Met

Weighting/

Score

In English*

Published in refereed journal since 2000 *

Full text available*

Methodology e.g. Qualitative or survey-based study.

Based in England/UK/home country

Qualified Public Health professionals with over 2 years professional practice experience

Specific debriefing intervention described

Informed consent clearly described*

Cultural issues acknowledged

Purposive sampling methodology explicit*

Total Score

10 / 14

There are 9 categories to ‘meet’. Not all categories, however, will be of equal importance. You may deem that a score of 10 is acceptable.

Three of the ‘quality criteria’ listed are not sufficiently met. As an appraiser you may find that your search provides one or two excellent studies but they are based in Australia. You could compromise and include both Australian and Malaysian research (provided that the other key criteria were met).

Once you have arrived at your final selection (6 – 12 papers) you are ready to work on detailed data extraction, analysis and synthesis.

Please note: you are not permitted to include full copies of research papers in the appendices of your dissertation as this constitutes a breach of copyright law

Data extraction & analysis

Further detailed work is necessary to appraise each study. Essentially, this phase of your study consists of [a] Description and [b] Critique / evaluation. A tabular format (landscape) is useful here. Each one of your selected papers is presented in the table. There are various ‘off the shelf’ tools to aid you with your critical analysis. For instance, the CASP tools offer a series of ten-question instruments analysing a range of both quantitative and qualitative studies. There is also a suggested template offered below. At this stage you will be moving on to the material for your ‘Reporting Findings’ Chapter, although some detail of your extraction, analysis and synthesis approaches must be given in the Methodology Chapter.

For each type of study there are general and specific areas that need to be addressed. The table on the next page are presented with column headings only for three different types of studies relevant to Nursing / Health. The far right hand column enables you to add your own commentary regarding any element of the study which will require more in depth discussion. For example, the funding for the study may have come from an organisation which has a vested interest in the results, thereby creating a possible conflict of interests. Another comment may refer to sampling processes which may not be explicit or the staff involved in delivering the interventions may have different backgrounds and ethnicity (which may or may not be relevant).

Normally one study would be presented on one landscape page. These will appear either at the end of your Methodology Chapter or at the beginning of the Findings Chapter.

Thomas et al’s(2004) paper on integrating qualitative research with trials may be helpful if you plan to mix designs.

The British Sociological Association Medical Sociology Group (1996) offer a general set of questions that might be used in order to ‘interrogate’ published research findings:

Are research methods appropriate to the question being asked?

Is there a clear connection to an existing body of knowledge/wider theoretical framework?

Are the criteria for/approach to sample selection, data collection and analysis clear and systematically applied?

Is the relationship between the researcher and the researched considered and have the latter been fully informed?

Is sufficient consideration given to how findings are derived from the data and how the validity of the findings was tested?

Has evidence for and against the researcher’s interpretation been considered?

Is the context for the research adequately described and accounted for?

Are findings systematically reported and is sufficient original evidence reported to justify a relationship between evidence and conclusions?

Are the researchers clear about their own position in relation to the research topic?

Data Extraction – Experimental (Intervention) Study
Authors, date & country and focus of study	Study design	Participants, recruitment & sampling methodology	Intervention		Outcome measures	Results		Comments
Look for conflicts of interest, potential bias. Justification for the study.	RCT, Quasi-Exp, B&A? Clearly explained. Is it appropriate for the aims of the study? Theoretical framework explicit?	Randomisation & Representation Descriptions of participants’ characteristics. Informed consent issues	What? When? Where? By whom? How? Are appropriate behaviour change theories included?		Data collection methods Subjective or objective measures? Validity & reliability of instruments (eg records, Questionnaires) Confounding factors	Analysis clearly reported. Statistical inferences?		Informed consent issues. Experimental & control groups – homogeneity?
Data Extraction – Survey Study
Authors, date & country and focus of study	Study design	Participants, recruitment & sampling methodology		Exposure – disease or risk factors	Outcome measures	Results		Comments
Look for conflicts of interest, potential bias. Justification for the study	Retrospective or prospective? Clearly explained. Length of follow up. Is it appropriate for the aims of the study? Theoretical framework explicit?	Representative of the defined population? How was the cohort population recruited? Was there something special about this cohort? Informed consent issues.		Clarification of exposure factor(s) with relevant theoretical models underpinning relationships	Data collection methods. Subjective or objective measures? Validity & reliability of instruments, laboratory tests. Questionnaires. Confounding factors	Analysis clearly reported. Follow up complete and long enough? Strength of association between exposure and outcome – RR		Informed consent issues. Numbers lost to follow up? Do the findings match available evidence?
Data Extraction – Qualitative Study
Authors, date & country and focus of study	Study design	Participants, sampling methodology			Data collection & Analysis	Findings	Comments
Look for conflicts of interest, potential bias. Justification for the study	Appropriate to answer the research question & aims? Is the context clear? Theoretical frameworks included?	How, why and by whom was the sample obtained? Can the methods be justified?			How were data obtained? Storage? Confidentiality & anonymity? Informed consent issues. Clear analytical pathway?	Valid, internally coherent and trustworthy? Researcher’s own perceptions? Reflexivity? Valid quotations used?	Audit trail from design – methods – analysis – interpretation clear? Validation of findings? Cultural issues?

Data Synthesis/Findings

This is where you categorise and critically comment on the key findings/results from all the included studies. A useful guide is to view the rows in your tables as individual extraction (each paper’s contribution) and the separate columns as the headings for your synthesis. The following table (incomplete) illustrates the processes:

Synthesis – differences/similarities/anomalies across ALL studies

Place/time Design/sample What/how/who/where? Measures

Authors date, country

Study Design

Sampling methods

Intervention

Outcome measures

Data

capture

Smith, et al (2006)

USA

Quasi-experimental

Children 6 – 10yrs in 4 schools in different towns

N = 200 (80 girls, 120 boys)

Daily activity sessions with drama teachers over one month

-Behaviour modification

-fewer tantrums

-willingness to participate

-improved reading

for each

study

in sep. tables

Ochachi, et al (2001)

Sweden

Cluster RCT

340 4 – 6 yr old children in 6 nursery settings in one Town

Controls = 180 (100 boys, 80 girls)

Experimental = 160 (60 boys, 100 girls)

Exp Grp =Daily creative play with performing arts students

Controls = Normal play activities

Half term period

-Behaviour modifications

-improved sleeping

-fewer tantrums

-willingness to participate

Findings

Do issues relating to reliability (where relevant) and validity emerge?
Are there ethical concerns arising from the studies?
What differences are apparent in the studies’ findings? Can these be accounted for?

Full scale Systematic Reviews of quantitative studies will normally include a meta-analysis (aggregated statistics) of the results (see www.cochrane.org). You may (if it is relevant and you have the ability and inclination) attempt to carry out a meta-analysis. The software programme to do this can be downloaded free from the Cochrane website. However you need to be wary of the futility of a meta-analysis on results from a limited number of studies. Your final ‘results’ cannot be relied upon to provide valid conclusions. Meta-analysis has no place in Systematic Appraisals of qualitative studies, although some ‘meta-ethnographic’ collation of themes may be possible (see Atkins et al. 2008 for example).

In whichever paradigm your Systematic Appraisal is situated, your dissertation will need to include relevant theoretical debates.

The Discussion Chapter – Linking your findings to the wider NURSING/health context

Discussion chapters are notoriously difficult to predict at the outset of the study: They are also difficult to write. The principle danger in a review-type project is that of Repetition.

A very useful way of organising this section of your dissertation is to use your original findings as sub-headings. Under these sub-headings you might then consider:

How well do the studies appraised address your SA aims and research questions?
What issues regarding the research processes appraised emerge?

In addition, the following bullet points offer only a brief summary of what could possibly be covered:

A critical discussion of the Systematic Appraisal methodology process itself – underpinning philosophy and strengths and potential limitations/weaknesses;
A critical discussion of the findings in relation to the wider body of Nursing/ Health literature.

You may end up focussing upon only one or two of these points and yet produce a perfectly defensible conclusion.

Conclusion

The following bullet points offer only a brief overview of possible contents:

The limitations of your particular SA should be summarised.
A summary / restatement of the main points raised in the discussion section.
Tentative theoretical proposals.
Implications for Nursing / Health professional practice.
Identification of gaps, silences and the silenced in the evidence.
Implications for future research.
A critical discussion about the problematic nature of ‘evidence’ and its collection, collation and interpretation.

(Think of this in terms of a ‘recommendations’ section that you might find in an official report).

A final ‘checklist’ for you to use when compiling your Systematic Appraisal Dissertation can be found in appendix 5

References, Suggested Reading & Resources – please also see the weblinks tab on the elp

Andrews, R. (2003) Research questions. London: Continuum.

Atkins, S. Lewin, S. Smith, H. Engel, M. Fretheim, A. &Volmink, J. (2008) Conducting a meta-ethnography of qualitative literature: Lessons learnt, BMC Medical Research Methodology,8, Article No. 21.

BSA Medical Sociology Group (1996), Criteria for the evaluation of qualitative research papers.Medical Sociology News 22.

Counsell, C. (1997) ‘Formulating questions and locating primary studies for inclusion in systematic reviews’, Annals of Internal Medicine 127(5) pp380-387.

Higgins, J.P.T 7 Green, S. (2010) Cochrane Handbook for Systematic Reviews of Interventions. Chichester: Wiley Blackwell

Petticrew, M., Roberts, H. (2006) Systematic reviews in the social sciences. Blackwell

Popay, J, Rogers, A & Williams, G (1998) Rationale and standards for the systematic

review of qualitative literature in health services research, Qualitative Health Research, 8(3) pp341-351

Thomas, J., Harden, A., Oakley, A. et al. (2004) ‘Integrating qualitative research with trials in systematic reviews’, British Medical Journal, 24^th April, 328(7446) pp1010-1012 [

Torrance, H. (2004) ‘Systematic reviewing –the ‘call-centre’ version of research synthesis.Time for a more flexible approach.Invited presentation to ESRC/RCBN seminar on Systematic Reviewing June 24^th University of Sheffield’. Available at: www.esri.mmu.ac.uk/respapers/systematic-reviewinght.php (Checked: Jan 26th 2011)

Weingarten, M. A., Paul, M. &Leibovici, L. (2004) ‘Assessing ethics of trials in systematic reviews’, British Medical Journal, 328(7446) 24^th April, pp1013-1014

White, D. (2001) ‘Evaluating evidence and making judgements of study quality: loss of evidence and risks to policy and practice decisions’, Critical Public Health, 11(1) pp3 – 17.

Indicative Reading List or Other Learning Resources:
Research governance and ethical guidelines
British Educational Research Association (BERA) (2004) Revised ethical guidelines for educational research. Available at: http://www.bera.ac.uk/blog/category/publications/ (Checked: Jan 26th 2011)

British Sociological Association (BSA) (March 2002, appendix updated May 2004) Statement of ethical practice for the British Sociological Association. Available at:

(Checked: Jan 26th 2011)

Department of Health (DoH).(2005) Research governance framework for health and social care.2^ndedn. Available at:

http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_4122427.pdf(Checked: Jan 26th 2011)

National Patient Safety Agency(2007) Facilitating and Promoting ethical research.Available at http://www.nres.npsa.nhs.uk/ (Checked: Jan 26th 2011)

Pertinent Websites of interest

Cochrane Library (2007)

Evidence for policy and practice information and co-ordinating centre (EPPI-centre) (2006) Available at: http://eppi.ioe.ac.uk

University of York (2007) Centre for reviews and dissemination.Available at: http://www.york.ac.uk/inst/crd/

National Coordinating Centre for Health Technology Assessment (2007) NHS health technology assessment programme. Available at: www.ncchta.org

Texts, journal articles and electronic publications

Aveyard, H. (2007) Doing a literature review in health and social care. A practical guide OUP/McGraw Hill Education

Ballinger, C. (2004) ‘Writing up rigour: representing and evaluating good scholarship in qualitative research’, British Journal of Occupational Therapy, 67(12), pp. 540-546.

Blaxter M. (2000) ‘Criteria for the evaluation of qualitative research papers’, Medical sociology news 26(2), pp. 34-37. Available at: http://www.britsoc.co.uk/user_doc/MSNVol26No2.pdf

Boslaugh, S. (2007) Secondary data sources for public health: A practical guide. Cambridge University Press Available at:

http://assets.cambridge.org/97805218/70016/excerpt/9780521870016_excerpt.pdf
Dixon-Woods, M., Fitzpatrick, R. (2001) Qualitative research in systematic reviews.BMJ 323 pp. 765-766

Dixon-Woods, M. et al (2004) Integrative approaches to qualitative and quantitative evidence. Health Development Agency

http://www.nice.org.uk/nicemedia/documents/integrative_approaches.pdf

Egger, M., Davey Smith, G.& Altman, D.G. (2001) Systematic reviews in health care. London: British Medical Journal Books.

Green, J. & Britten, N. (1998) ‘Qualitative research and evidence based medicine’, BMJ, 316 (7139) pp.1230-2.

Khan, K.S. &Kleijan, J. (2003) Systematic reviews to support evidence-based medicine: how to review and apply findings of healthcare research. London: Royal Society of Medicine Press.

Lyne, P. Allen, D., et al. (2002) ‘Improving the evidence base for practice: a realistic method for appraising evaluations’, Clinical Effectiveness in Nursing, 6, pp.81-88.

Mulrow, C.D. (1994) Systematic Reviews: Rationale for systematic reviews. British Medical Journal 309(6954) 3 September pp597-599
Moher Det al. (1999) ‘Improving the quality of reports of meta-analyses of randomised controlled trials: the QUORUM statement’,The Lancet,27^th November354 (9193), pp.1896-1900.

Oakley, A. (2003) ‘Research evidence, knowledge management and educational practice: early lessons from a systematic approach’, London Review of Education, 1 (1), pp.21-33.(please ask at helpdesk)

(An earlier version of this article (2002) is available at: https://www.oecd.org/dataoecd/47/38/2074395.pdf (Checked: Jan 26th 2011)

Pearson, A. (2004) ‘Balancing the evidence: incorporating the synthesis of qualitative data into systematic reviews’. JBI reports, 2(2) pp.45-64.

Popay, J. &Roen, K. (2003) Report 3: Using evidence from diverse research designs. Report No. 3. Social Care Institute for Excellence [Online] Available at: http://www.scie.org.uk/publications/reports/rep03.asp
Sutton, A. J. et al.(1998) ‘Systematic reviews of trials and other studies’, Health technology assessment, 2(19) [Online]. Available at: http://www.hta.ac.uk/project/942.asp

Appendix 1

Systematic Appraisal (SA) Dissertation Chapter Outlines

NB the actual content will vary according to the particular Specific Appraisal area

Word lengths given are approximates only

Title & cover information including word count

Acknowledgements

Contents including Tables, References & Appendices

Glossary of terms – (could be placed in an appendix)

Abstract (400 words max)

Think of an abstract as being each section of your review reduced to one or two sentences;

You can insert subheadings into the text e.g.

Objectives: Aim of SA with your research question

Methods: Search criteria statement including no. of papers selected with briefest details of origins/dates etc + methodology(ies) & method(s) appraised

Setting:

Findings/results: Key statements only

Conclusion: Implications for Nursing / Health practice / education / research / policy etc.

Chapter One – Introduction (Roughly 1200 words)

The overarching theme is framed around answering the question of ‘Why this SA is important / significant’. In order to answer this question, you might include:

An introduction to the whole project;
Definitions /clarifying terms /relevant policy / statistical information;
Key supporting literature – remember you’ll have to expand on this in the literature review section;
Why your topic needs a review in order to clarify certain questions

It is normal to conclude this section with a statement of:

Your main research question(s)?
The specific Aims for YOUR

Chapter Two – The Literature Overview (Roughly 2500 words)

It is usual to start this section with a short(ish) account of the search methodology, e.g.

Search terms;
Databases searched;
Number of ‘Hits’ for each combination of search terms.

This can information can be tabulated.

It’s usual in most searches to:

Have all major theoretical perspectives on a topic represented;
Have a mixture of ‘classics’ and more up-to-date work;

Present the literature in ‘themes’ that are pertinent to the research question e.g. don’t summarise one study, then go to the next, then the next &c. Use sub-headings;
Give short critical comments upon the pieces reviewed;
Summarise the main points at the end of the review (I prefer bullet points, but its a matter of personal preference).

NB a critical issue in writing this section for a review-type dissertation is that you concentrate on material that doesn’t appear later as the subject of your detailed review. You can however, mention the material that you intend to review in detail. If you do this, just a few lines will do here together with a note that this work is subjected to detailed review later.

Chapter Three – Conducting the Appraisal – Methodology & Method (Roughly 3000 words)

A sizeable section on Methodology should follow a brief Introduction / overview.

This section could include:

A discussion of Evidence-based practice (the nearest thing that one can get to an underpinning ‘philosophy’ for these types of project);
A definition / description / discussion of Systematic Reviews / Narrative Reviews &c.
A discussion relating to ‘secondary data analysis’;
The advantages / disadvantages of doing reviews, including the advantages / disadvantages of secondary data analysis.

NB a notable ‘trap’ here is to get drawn into a discussion about the methods that the primary papers have used – but you should normally comment on these in following sections.

Further note that, at this point, you are discussing reviews in the abstract / in general – you are not discussing your particular review.

The sizeable remainder of this section should outline your Methods e.g. what you have actually done

Return to / restate your research question(s)
Outline the search terms;
Tabulate search terms / databases and numbers of hits;
Explain your Inclusion / exclusion criteria;
Explain how you arrived at your final selection of studies;
A flow chart outlining this process (see appendix 4 below);
An explanation of data extraction & analysis (including your critical reading checklist / criteria); and,
Short conclusion & link to next chapter

ChapterFour – Reporting the Findings (Roughly 2,500 words) (WHAT DID YOU FIND?)

Introduction to chapter

Tabular presentation of ALL included studies;

Some general comment on the characteristics & quality of Nursing / Health research studies selected;

This then should normally be followed by a detailed appraisal of studies –

Which would normally be presented in themes e.g.

Theoretical issues;
Methodological limitations,

Sampling;
Approaches to analysis;
Outcomes/findings;
Quality assurance issues e.g. validity / reliability / trustworthiness / rigour of the studies;
Conclusion & link to next chapter

ChapterFive – Discussion (Roughly 3000 words) (WHAT DO THE FINDINGS MEAN?)

A very useful way of organising this section of your dissertation is to use your original AIMS as sub-headings. Under these sub-headings you might then consider:

How well do the studies appraised address your SA aims and research questions?
What issues regarding the research processes appraised emerge?

In addition, the following bullet points offer only a brief summary of what could possibly be covered:

A critical discussion of the appraisal process itself;
A critical discussion of the findings in relation to the wider body of Public Health / Nursing / Health literature.
Tentative theoretical proposals ;
Implications for Nursing / Health practice / education / management;
Identification of gaps in the evidence;
Implications for future research;
A critical discussion about the problematic nature of ‘evidence’ and its collection, collation and interpretation.

Effectively, this section conducts an ‘academic conversation’ between your findings and others’ claims in their published research. You will have probably covered some of the relevant ‘others’ work in your literature review section

Chapter 6 – Conclusion (Roughly 1500 words) (SO WHAT?)

The following bullet points offer only a brief summary of what could possibly be covered:

The limitations of your particular SA should be summarised;
A summary / restatement of the main points raised in the discussion section.
Tentative theoretical proposals ;
Implications for Nursing / Health practice / education / management;
Identification of gaps in the evidence;
Implications for future research;
A critical discussion about the problematic nature of ‘evidence’ and its collection, collation and interpretation.

You may end up focussing upon only one or two of these points and yet produce a perfectly defensible conclusion.

(Think of this in terms of a ‘recommendations’ section that you might find in an official report).

References

All papers/references listed – Harvard style

Appendices

Appendices can be many – depending upon what you think might be useful and / or relevant in order to support your study.

However, beware of a trap – do not simply ‘lump’ material in the appendices in order to reduce that overall length of your dissertation. Material that appears in appendices, technically cannot be marked – therefore if it’s that important, it shouldn’t be in the appendices.

One appendix however, would be given over to a tabular presentation ofALL excluded studies. Studies in your included table must appear in your list. (Do not include copies of original research papers)

Writing Protocols:

Word length:15,000 words (normally) with 10% leeway in either direction.

The word length includes the references in the text (authors’ names/dates) but does not include the list or the material contained in the appendices.

Presentation:Dissertations must be typed, 1.5 or double spaced (single-spaced for abstract, references and appendices) on A4 white paper. Margins should be as follows: 4cm at the left (to allow for binding), 2.5cm at the top, bottom and right. Pages must be numbered.

Front cover:See appendix 6 below for a template of the front cover.Please use card covers (back & front)

Binding:You are responsible for arranging your own binding – comb style please.

Do get someone to proof read your work for you – it’s amazing the number of minor errors you miss.

Submission & marking processes

Submission date: check with programme leader / supervisor

Two bound copies should be submitted via the SSAC office (C011) at Coach Lane Campus.

Each copy will be marked by two members of academic staff. One marker will be (normally) the student’s designated group supervisor and the other a member of staff familiar with Masters level work. The two markers meet to discuss the work and agree a final mark. Where a discrepancy cannot be readily resolved a third marker may be involved. The External Examiner will review a sample of work. S/he may ask to review dissertations other than those she has been sent. The grading framework appears in the course handbook and is the same that has been applied to already submitted work.

The student may be given a provisional mark, but marks are not confirmed until ratification at the Examination Board. All students will receive written notification of their marks and final award.

There are four possible outcomes at the end of the course:

Masters with Distinction (average marks >70)

Masters with Commendation (average marks 60 – 69)

Masters (average marks 50 – 59)

Refer (average mark is below 50)

Appendix 2

Sample form to record information on your searches

Description of data source / search strings

Author(s) Name(s):

Country:

Funding/Sponsors:

Journal Name:

Study Title:

Data source searched:

Details of search strings used:

Abstract details:

Issues emerging from search for your SA: Not relevant?

Obtain full text? Revisit search criteria? New areas to consider for SA aims? Time taken to locate?

Appendix 3 Systematic Appraisal Project Management Gantt Chart – an example

Task

March

April

May

June

July

August

September

Read relevant literature. Choose & justify a topic

Clarify aims for SA

Develop & refine research question

Carry out scoping exercise by reviewing relevant literature from range of sources

Translate your research question into a search strategy

Complete Protocol Document

Searching – inclusion & exclusion criteria

Data extraction & analysis

Synthesis & appraisal of findings

Discussion

Conclusion

Individual Tutorials

Editing & proof reading – Binding

Submit Dissertation

X 2 Copies

Appendix 4

FLOW CHART (Generic example – you would include the topics/terms etc.)

Appraisal Question

18,600 citations – most not research or unrelated

Scoping exercise

Initial search

1,007 citations

Many not research

Refine research terms

10 suitable papers

Appendix 5

Final Checklist Questions

Is the research problem to be explored clearly presented and justified?þ

Are underpinning theoretical frameworks evident and clearly expressed?þ

Is the choice of methodology/methods justified and explained?þ

Is the supporting literature presented critically?þ

Is the search strategy evident with clear and appropriate use of inclusion & exclusion criteria?þ

Are the selected studies appraised according to a clearly defined framework? þ

Does the analysis and subsequent synthesis reflect Masters level work? Is there a strong theoretical & conceptual methodological critique?þ

Does the discussion locate & contextualise the review findings within the wider academic field?þ

Are the limitations of your SA acknowledged?þ

Do the conclusions relate to the original research question(s)?þ

Are your recommendations for Nursing / Health policy/practice/theory/research/ education (not necessarily all areas) in keeping with the appraisal findings?þ

If you can answer confidently & positively to all – you may hand in your Dissertation – and look forward to graduation.

APPENDIX 6

The title page of the dissertation should be set out as shown as follows:

NORTHUMBRIA UNIVERSITY

FACULTY OF HEALTH AND LIFE SCIENCES

[name of masters programme]

[PROJECT TITLE]

[STUDENT NAME]

DATE e.g September 2015

Word count

A DISSERTATION SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS OF THE DEGREE OF [name of masters programme]

[1]Adapted from those guidelines prepared by Dr.Michael Hill

Expert paper writers are just a few clicks away

Place an order in 3 easy steps. Takes less than 5 mins.

Appendix 3

Systematic Appraisal Project Management Gantt Chart – an example

Submit Dissertation

X 2 Copies

Expert paper writers are just a few clicks away

Calculate the price of your order