Do your own research: How to evaluate scientific papers and abstracts in smoking cessation

One of the most impactful ways you can take control of your health is to make informed choices based on the most up-to-date research.

Journalists strive to write stories about the latest scientific papers in a way that a larger audience without a scientific degree can understand. Sometimes, they’re excellent and present relevant, factually accurate information for their readers. But other times, flashy headlines or rushed reporting leads to stories that lack the detail and nuance you need to decide the best steps to take for your health.

If you’re looking up the latest research on how to quit smoking or use less harmful forms of nicotine, the best place to start is with the research itself. Credible news outlets will link to the original paper they are writing about so all readers can access it.

Ideally, you’d be able to evaluate the full text of the research to determine how much weight to give it. But if you can’t access the paper because of a paywall – or don’t have time to – you can often gain a lot from reading the scientific abstract.

The abstract is a brief section at the beginning of every scientific paper that provides an overview of the study and its conclusions. Abstracts should be available over the paywall. Most follow a similar structure, often broken down into the following sections:

Take this paper, published in the New England Journal of Medicine in February

BACKGROUND

By reading the background and methodology, we see that the research team is trying to evaluate if e-cigarettes are better at helping adults quit smoking than standard counseling and a voucher participants could use for any purpose, including traditional nicotine replacement therapies.

We also quickly see that this is a recent study, meaning that its findings are likely still relevant. Older studies may still carry important knowledge but may miss newer developments.

METHODS

If we look further at the methods, we see that this is a randomized controlled trial. In these types of studies, researchers randomly assign participants to either “control” groups, who receive either no intervention or standard care, and “treatment” groups, who receive a new intervention.

Randomized controlled trials are an extremely high-quality form of research design. They minimize confounders and ensure that participants cannot self-select into one group and agree to follow the study protocol. The gold standard of research trials are “double blinded” randomized controlled trials, in which neither the research team nor the participants know which group they have been assigned to. In this case, though, it would be obvious to both parties if they had received an e-cigarette or not.

Types of studies you may see are observational that observe different factors or behaviors without changing other conditions, or lab-based studies that look at cell or animal models. Observational studies, sometimes called cohort studies, can show if factors are related to certain outcomes – like smoking cigarettes and developing lung disease – but it takes lab studies to confirm how smoking can lead to cancer. In other words, correlation does not imply causation.

In this case, scientists measured chemical signals in participants’ blood to assess whether they were successfully able to quit smoking, which means that there is less of a chance of participants misremembering or misreporting whether they were able to quit during the study period.

RESULTS

The results in this abstract shows us more about the methodology. This team collected data on more than 1,200 individuals.

Typically, the larger the study size, the more robust the results. Smaller numbers of participants may still be credible and appropriate, but it depends on the study design and the types of statistical analysis the researchers usedt. For example, this paper explained their sample size calculation in the full text.

The team gives us a mix of figures to explain the results. They present percentages (fractions of the whole group assessed) and absolute values (how many people the effect happened to). These are the most important figures to understand, because they show the breakdown of how e-cigarettes performed compared to the control group, which received standard counseling and vouchers.

Based on this study’s results, we can see that about 13 percent more adults who used e-cigarettes to quit were able to stay tobacco-free after 6 months. But more adults who used standard counseling hadn’t used nicotine entirely in the week before their 6-month follow-up than those who used e-cigarettes. In other words, participants who used e-cigarettes to quit were more likely to use some form of nicotine than those who used traditional methods, even if they were more likely to quit smoking.

Other figures, like the relative risk (RR) and confidence intervals (CI), are statistical measures. For example, CIs help measure how precise the results are. Relative risk is a ratio of risks that tells us how likely something is to happen in one group compared to another. While it is important for the team to be transparent about the statistics for the validity of the paper, the specifics of the statistical analyses are often less important to the general public. The fact that the paper is published in a peer-reviewed journal at all indicates to us that its methods have been reviewed by other scientists in the field and journal editors for validity.

CONCLUSIONS

The conclusion of this paper says that the use of e-cigarettes led more individuals to stop using tobacco after six months. This matches the results that the authors presented previously.

The abstract of this paper shows us a lot – but there are some questions we’d have to read the full paper to get clarity on. For example, do we have any more details about the participants – what is the breakdown among genders, race, levels of education, or socioeconomic status? Is the sample representative of the population? Additionally, the research team mentions “serious adverse events,” or a situation requiring medical care that may be unrelated to the study, which is intriguing. These questions are answered in the full text and supplementary material; their answers are most important for those who are committed to doing a much deeper dive on the matieral.

No one study, even if you had the full paper, is going to give you conclusive evidence on what you should or shouldn’t do on your cessation journey. All studies have strengths and limitations. This is why it’s important to see many studies, or even collections of studies called systematic reviews or meta-analyses to get a better sense of the whole scientific field.

Want to learn more about reading scientific papers? Check out our Get the Facts page.