Click for Great Web Sites

by Dr PHUA Kai Lit

School of Medicine and Health Sciences

Monash University Sunway Campus

Bandar Sunway, Malaysia

Click for Interview with Prof Michael Marmot on social determinants of bad health

The ability to read and evaluate scientific and scholarly material (especially articles published in scientific journals) in a sophisticated manner. To do so, you need the following skills:

1. Know where and how to search for scientific information

2. Know about the main types of research design (methodology)

3. Able to detect flaws in the research design of published scientific papers

4. Know how to interprete data published in research papers (including results of statistical tests)

It is important to possess critical appraisal skills and to read scientific literature because of the following:

a. Constant advances in diagnosis and treatment of diseases (new drugs, new medical procedures etc)

b. Important to know about latest news on adverse effects of drugs and procedures

c. Need to know about emerging and re-emerging diseases

d. Need to know about emerging drug-resistant infections

e. Be able to distinguish between high quality and low quality research

f. Be aware of existence of scientific fraud

Short article on scientific fraud

Read long article on scientific fraud

Fraudulent letters sent to academic journals!

Public Citizen's problematic pharmaceutical drugs website

Drug safety and recalls (article by Phua Kai Lit)

Another version of this article will be appearing in the British journal Clinical Ethics soon

1. Outright fabrication of data

2. Falsification of data ("massaging of data")

3. Suppression of negative findings (suppression of data that does not support the research hypothesis)

4. Plagiarism (Example 1: using other people's ideas without acknowledgement. Example 2: using other people's written text and passing it off as your own i.e. without proper quotation)

**General Journals**

New England Journal of Medicine

Journal of the American Medical Association

BMJ (British Medical Journal)

Lancet

Medical Journal of Malaysia

Singapore Medical Journal

American Journal of Public Health

Asia-Pacific Journal of Public Health

* Focus especially on "review articles" and "concensus recommendations of expert panels". These will give you information on current knowledge and the state of the art.

**Literature Review Journals**

Medical Letter on Drugs and Therapeutics

**Books**

Summary-type books e.g. "Current Medical Diagnosis and Treatment"

Classic textbooks e.g. Cecil's, Harrison's, Harvey's textbooks (medicine)

e.g. Maxcy-Rosenau-Last (public health)

Review volumes e.g.

Annual Review of Medicine

**The Internet**

General information

Google

Enclose your phrases (can be more than 1 phrase) in quotation marks

e.g.
To search for info on HIV in Malaysia, type in "HIV" "Malaysia" simultaneously

Local websites

Foreign websites

**Clinical Practice Guidelines**

U.S. Agency for Healthcare Research and Quality

Clinical Practice Guidelines Online

Ministry of Health, Singapore

Clinical Practice Guidelines Online

**Computerised Literature Searches**

**Other HIGH QUALITY Sites**

Centers for Disease Control (CDC), U.S. Public Health Service

1. Experiments

2. Quasi-experiments

3. Correlational Studies

4. Prospective Studies (Cohort Studies or Longitudinal Studies)

5. Retrospective Studies (Case-Control Studies)

6. Other e.g. meta-analysis

EXPERIMENTS

Researcher has control over the treatment/intervention e.g. RCT (Randomized Controlled Clinical Trial)

There should be at least one Experimental Group and one Control Group

Randomization i.e. reseach subjects are equally likely to end up either in the Experimental Group or in the Control Group

Blinding e.g. double blind experiments where research subjects do not know which group they are in (Expt or Control Group) and researchers do not know which group an individual is in either

Placebo effect (to correct for this problem, the Control Group given inert substance/sham treatment)

QUASI-EXPERIMENTS

This method is used when experiments cannot be done. Quasi-experiments are common in public health research.

Researcher cannot control characteristics of subjects. Randomization is not possible e.g. a public health campaign is held in City A (experimental group) but not in City B (the "control group")

CORRELATIONAL STUDIES

Study relationships or associations between variables.

CAUTION: Correlation does not necessarily mean causation!

PROSPECTIVE STUDIES (COHORT STUDIES OR LONGITUDINAL STUDIES)

e.g. observe 2 groups of people over time (Group 1 is exposed to the suspected disease-causing agent while Group 2 is not exposed).

Are people in Group 1 more likely to get the disease than people in Group 2?

RETROSPECTIVE STUDIES (CASE-CONTROL STUDIES)

"Retrospective" = backward-looking
e.g. Individuals with the disease (Group 1 of "cases") are compared to those without the disease (Group 2 of "controls"). Try to determine what could have caused the disease in Group 1 (identify possible risk factors).

Calculate the Odds Ratio. Is this statistically significant?

OTHER e.g. META-ANALYSIS

Combine the findings of several RCT to arrive at a definite conclusion

a. Identify studies on a similar research question

b. Use statistical techniques to get one conclusion

**IMRAD Format**

**I**ntroduction

**M**ethodology

**R**esults

**A**nd

**D**iscussion

**(1) ABSTRACT AND KEYWORDS**

Scan the abstract (summary). It should summarise the contents of the paper properly.

Note the keywords

**(2) INTRODUCTION**

What is the stated goal of the article?

What is the rationale (why was the study done)?

Read the literature review. Is it done properly?

**Is the research question clearly stated?**

Are the research hypotheses very clear and explicitly laid out?

Is the research question important?

Is the research original or a replication of work done by other reseachers?

**(3) METHODOLOGY**

What is the research design?

Randomised controlled clinical trial?

Quasi-experiment?

Observational study e.g. cohort study, case-control study?

**Subject Selection**

What is the sampling method? What is the response rate? Any bias? What is the control group?

What is the dependent variable (outcome variable)?

What are the independent variables (predictor variables)?

**(4) RESULTS**

Presented in the form of graphs, charts, tables

What are the statistical tests used?

Is the statistical test misused (i.e. are the assumptions of the statistical test violated?

What are the results of the statistical tests?

**(5) DISCUSSION**

Evaluate the summary of findings

What are the limitations of the study?

Do the conclusions match the research question declared at the beginning of the article?

How do the findings compare with other research?

What further research is needed?

**(6) REFERENCES**

The references section should be carefully prepared in a well-written scientific paper. Different journals may use different formats BUT if you submit a paper to a particular journal, you should follow its format for refences STRICTLY!

**(7) APPENDIX**

The appendix often contains highly technical details on the research methodology

1. **The p-value and "Statistically Significant" research findings**

**The p-value is the probability that the observed relationship (e.g., between variables) or a difference (e.g., between means) in a sample occurred purely by chance, and that in the population from which the sample was drawn, no such relationship or differences actually exist.**

"Statistically significant" means the probability that an observed result has occurred by chance is very low. Because the probability is so low, the result seen is very likely to be real (not due to chance) and meaningful (significant).

If the p-value is > 0.05, it is NOT CONSIDERED statistically significant (strictly speaking, "there is not enough evidence to reject the null hypothesis H_{0}"). This is because the probability is more than 0.05 that it could have occurred by chance.

If p < 0.05, it is considered statistically significant (written as --> p < .05 *)

If p < 0.01, it is considered highly significant (written as --> p < .01 **)

Decision Rule:

Accept ("fail to reject")H_{0}if p > .05

Reject H_{0} if p< .05

2. **Null Hypothesis and Research Hypothesis**

For the chi-square test:

H_{0} : there is no association between Variable X and Variable Y. Any association seen is due to chance alone

H_{1} : there is a statistically significant association between X and Y

For the t-test:

H_{0} : there is no difference between the two underlying population means. Any difference seen is due to chance alone

H_{1} : there is a statistically significant difference between the two underlying population means

3. **Relative Risk (used for prospective studies)**

**How the Relative Risk is Calculated**

Becomes Diseased | No Disease | |
---|---|---|

Exposed to risk factor |
a | b |

Not exposed to risk factor |
c | d |

Risk of Developing Disease for Those Exposed to Risk Factor over Time = a divided by (a+b)

Risk of Developing Disease for Those Not Exposed to Risk Factor = c divided by (c+d)

The Relative Risk is a/(a+b) divided by c/(c+d) i.e. risk of exposed group is compared to risk of unexposed group. Hence the term **relative** risk.

If the Relative Risk = 1, this means that the risk of developing disease for the exposed group is the same as the risk for the unexposed group. Therefore, there is no association between a risk factor and a disease

If the Relative Risk > > 1, there is a strong positive association

If the Relative Risk < < 1, there is a strong negative association

e.g. In a study on smoking and lung cancer, the Relative Risk (estimated from a sample) is 1.62

This means roughly that "People who smoke are 1.62 times more likely to develop lung cancer than those who do not smoke"

This means strictly that "The risk of developing lung cancer for people who smoke is 1.62 times the risk for people who do not smoke"

The next step is to determine if this estimated Relative Risk is statistically significant.

3b. **Testing if a Relative Risk is Statistically Significant **

To do this, we need to construct a 95% Confidence Interval.

(A 95% CI means roughly that the probability is 0.95 that the real value of a parameter lies within the interval calculated)

Strictly, it means that if you take 100 samples and calculate the statistic 100 times, 95 of them will fall within the 95% CI)

The estimated RR is used to construct the 95% CI for the true RR

Example:

H_{0} : Relative Risk = 1 (no association between exposure to risk factor and Disease X)

H_{1} : Relative Risk is not equal to 1 (association between exposure to risk factor and Disease X)

If 95% CI does not include 1, reject H_{0}

(prob. is 0.95 that true RR falls within the interval i.e. 95% sure that true RR is not equal to 1)

If 95% CI includes 1, accept ("fail to reject") H_{0}

Example 1

Estimated Relative Risk = 1.62

95% CI for true Relative Risk = 1.15 to 2.28

H_{0} : RR = 1

H_{1} : RR is not equal to 1

What is your conclusion?

Since the 95% CI does not include 1, we are 95%
sure that the real Relative Risk is not equal to 1.

Therefore, we accept H_{1} and conclude that (1)there is
an assoc between exposure to the risk factor and chances
of getting a Disease X and (2)People who are exposed to the risk factor are 1.62 times more likely to develop the Disease than those who are not exposed.

4. **Odds Ratio (Often used for retrospective or case-control studies)**

** What is meant by the term "Odds"?**

An example will clarify:

The probability of getting a 6 when a dice is thrown is 1/6

The odds of getting a 6 when a dice is thrown is 1:5 or 1/5

That is, when you throw a dice, there are 6 possible outcomes (1,2,3,4,5,6)
and one outcome is the desired outcome while the other five outcomes are what you don't want to get.

Thus, there is: 1 possible "success" compared to 5 possible "failures" (1:5 or 1/5)

**How the Odds Ratio is Calculated**

Has the disease now (CASES) | No disease (CONTROLS) | |
---|---|---|

Exposed to risk factor in the past |
a | b |

Not exposed to risk factor in the past |
c | d |

Odds of Exposure for Cases = a:c or (a/c)

Odds of Exposure for Controls = b:d or (b/d)

The Odds Ratio is (a/c) divided by (b/d) --> This is the same as (a*d) divided by (b*c)

The Odds for the Cases are compared to the Odds for the Controls. Hence the term Odds **Ratio**

If the Odds Ratio = 1, there is no association between a risk factor and a disease (i.e. if cases and controls are compared, there is no difference in odds of exposure to a risk factor in the past.

If the Odds Ratio > > 1, there is a strong positive association

If the Odds Ratio < < 1, there is a strong negative association

e.g. In a study on lung cancer presence now and smoking in the past, the estimated OR is 2.14

This means roughly that "Lung cancer victims are 2.14 times more likely to be smokers than people without lung cancer"

(This means strictly that "Lung cancer victims' odds of exposure to smoking are 2.14 times the odds for those without lung cancer")

The next step is to determine if this estimated OR is statistically significant.

4b. **Testing if an Odds Ratio is Statistically Significant **

To do this, we need to construct a 95% Confidence Interval.

(A 95% CI means the probability is 0.95 that the real value of a parameter lies within the interval calculated)

The estimated OR is used to construct the 95% CI for the true OR

Example:

H_{0} : Odds Ratio = 1 (no association between exposure to risk factor and Disease X)

H_{1} : Odds Ratio is not equal to 1 (association between exposure to risk factor and Disease X)

If 95% CI does not include 1, reject H_{0}

(prob. is 0.95 that true OR falls within the interval i.e. 95% sure that true OR is not equal to 1)

If 95% CI includes 1, accept H_{0}

Example 1

Estimated Odds ratio = 2.5

95% CI for true Odds ratio = 2.0 to 3.0

H_{0} : OR = 1

H_{1} : OR is not equal to 1

What is your conclusion?

Since the 95% CI does not include 1, we are 95%
sure that the real odds ratio is not equal to 1.

Therefore, we accept H_{1} and conclude 1) that there is
an assoc between exposure to the risk factor and chances
of getting a Disease X and 2) People with Disease X are 2.5 times more likley to have been exposed to the risk factor in the past than people without the Disease X

Example 2

Odds Ratio = 1.3

95% CI = 0.9 to 1.7

H_{0} : OR = 1

H_{1} : OR is not equal to 1

What is your conclusion?

Since the 95% CI includes 1, accept ("fail to reject") H_{0}

Conclude that 1) there is no assoc between exposure to suspected risk factor and chances of getting Disease X and 2) People with the Disease are no more likely (equally likely) to have been exposed to the risk factor in the past compared to people without the Disease (hence we conclude there is no association between exposure and chances of getting the Disease. Therefore the suspected risk factor can probably be ruled out as an actual risk factor).

**Interpreting the Relative Risk versus Interpreting the Odds Ratio**

If Relative Risk = 2.1

Interprete roughly as "People who are exposed to the risk factor are 2.1 times more likely to develop the disease than people who have not been exposed"

Interprete strictly as "For people exposed to the risk factor, their risk of developing the disease is 2.1 times that of people who have not been exposed"

If Odds Ratio = 2.1 (in a case-control study)

Interprete roughly as "People who have the disease are 2.1 times more likely to have been exposed to the risk factor in the past compared to people who do not have the disease"

Interprete strictly as "For people who have the disease, their odds of exposure to the risk factor in the past are 2.1 times the odds of exposure for people who do not have the disease"

**Thus, Relative Risk concerns the statement (as used in prospective studies i.e. looking forward in time)
"If people are exposed to a risk factor, how likely are they to develop a disease (compared to those not exposed)?"**

**
Odds Ratio concerns the statement (if used for retrospective, case-control studies i.e. looking backward in time)
"If people have a disease now, how likely are they to have been exposed to a risk factor in the past (compared to those without the disease now)?"**

**NOTE**: Odds Ratio analysis **CAN** be used for prospective studies too. But since they are harder to interprete, it is better to use Relative Risk analysis for prospective studies.

**Sensitivity versus Specificity (of Screening Test)**

Truly has the disease | Does not actually have the disease | |
---|---|---|

Tested positive |
a | b |

Tested negative |
c | d |

Total | a + c | b + d |

Sensitivity (%) = a/(a+c) X 100

Sensitivity = ability of test to correctly identify persons who HAVE the disease

Specificity (%) = d/(b+d) X 100

Specificity = ability of test to correctly identify persons who DO NOT HAVE the disease

**PPV versus NPV (of Screening Test)**

Truly has the disease | Does not actually have the disease | Total | |
---|---|---|---|

Tested positive |
a | b | a+b |

Tested negative |
c | d | c+d |

Positive Predictive Value (PPV) (%) = a/(a+b) X 100

Negative Predictive Value (NPV) (%) = d/(c+d) X 100

* Using the same test in a population with higher prevalence increases positive predictive value. Conversely, increased prevalence results in decreased negative predictive value.

(Sensitivity and specificity of the clinical test are unaffected by prevalence of the disease).

**

**