ISSN Online: 2177-1235 | ISSN Print: 1983-5175

Bibliometric analysis of articles published in the Brazilian Journal of Plastic Surgery between 2005 and 2012. Part III: study designs and levels of evidence
Análise bibliométrica dos artigos publicados na revista brasileira de cirurgia plástica entre 2005 e 2012. Parte III: desenhos de estudo e níveis de evidência

Previous Article Next Article

Original Article - Year2014 - Volume29 - Issue 1

Rafael Denadai1; Dov Goldenberg2; Cassio Eduardo Raposo-do-Amaral3

http://www.dx.doi.org/10.5935/2177-1235.2014RBCP0004

ABSTRACT

INTRODUCTION: Bibliometric methods have been used to analyze study designs and levels of evidence of articles published in various periodicals. Such information does not exist in the field of Brazilian plastic surgery. This study aimed to evaluate the study designs and levels of evidence of articles published in the Brazilian Journal of Plastic Surgery ("Revista Brasileira de Cirurgia Plástica," RBCP), the only Brazilian journal devoted exclusively to plastic surgery.
METHOD: Study designs and levels of evidence of articles published in the RBCP between 2005 and 2012 were bibliometrically and quantitatively analyzed. The articles published in two periods (2005-2008 and 2009-2012) were compared, in order to characterize the evolution of the scientific production of the RBCP.
RESULTS: During the period analyzed, 603 scientific articles met the inclusion criteria. The overall analysis revealed a significant predominance of retrospective studies and articles with level of evidence III (p < 0.05). The comparison between the periods showed a significant increase (p < 0.05) in the proportion of case series and scientific articles with level of evidence IV (2005-2008 < 2009-2012), and a significant reduction (p < 0.05) in the proportion of articles with level of evidence V (2005-2008 > 2009-2012).
CONCLUSION: This study showed that a predominance of retrospective studies and articles with level of evidence III published in the RBCP between 2005 and 2012.

Keywords: Bibliometrics; Plastic surgery; Evidence-based plastic surgery; Study design; Brazilian Journal of Plastic Surgery; Level of evidence

RESUMO

INTRODUÇÃO: Os desenhos dos estudos e os níveis de evidência dos artigos publicados em diversos periódicos vêm sendo analisados bibliometricamente. Como não existem tais informações no campo da cirurgia plástica brasileira, o presente estudo tem o objetivo de avaliar os desenhos dos estudos e os níveis de evidência dos artigos publicados na Revista Brasileira de Cirurgia Plástica (RBCP), o único periódico brasileiro destinado exclusivamente à cirurgia plástica.
MÉTODO: Foi realizada uma análise bibliométrica, quantitativa dos desenhos dos estudos, e níveis de evidência dos artigos publicados na RBCP, entre 2005 a 2012. Dois períodos (2005-2008 versus 2009-2012) foram comparados para caracterizar a evolução da produção científica da RBCP.
RESULTADOS: No período analisado, 603 artigos científicos preencheram os critérios de inclusão. A análise global revelou um predomínio significativo (p<0,05) de estudos retrospectivos e artigos com nível de evidência III. A comparação entre os períodos revelou aumentos significativos (p<0,05) nas proporções de séries de casos e artigos científicos com nível de evidência IV (2005-2008<2009-2012), e uma redução significativa (p<0,05) na proporção de artigos com nível de evidência V (2005-2008>2009-2012).
CONCLUSÃO: Este estudo demonstrou que houve um predomínio de estudos retrospectivos e artigos com nível de evidência III, publicados na RBCP entre 2005 e 2012.

Palavras-chave: Bibliometria; Cirurgia plástica; Cirurgia Plástica baseada em evidências; Desenho dos estudos; Revista de Cirurgia Plástica; Nível de evidência.

INTRODUCTION

In the 1990s, the term "evidence-based medicine" (EBM) was introduced and defined as the conscious, explicit and judicious use of current best evidence for making decisions about individualized care for the patient1. Since then, the influence of EBM in the practice of medicine is such that it is been considered to be the fourth revolution of American medicine2; the British Medical Journal (BMJ) listed EBM among the top 15 medical advances since 18403.

Thus, numerous efforts have been made to incorporate EBM in various areas of medicine, including plastic surgery. The American Society of Plastic Surgeons (ASPS) and the journal Plastic and Reconstructive Surgery (PRS) launched an initiative to improve understanding of the concepts of EBM, and provide tools for its implementation in the practice of plastic surgery4. To that end, since 2007, the ASPS has published evidence-based practical guidelines4 and, in 2009, the editorial "Introducing evidence-based medicine to Plastic and Reconstructive Surgery"5 addressed the importance of EBM in plastic surgery, and provided an invitation to a "new initiative", outlining levels of evidence and ways to incorporate high-level studies in the practice of plastic surgery. As part of this initiative, the ASPS proposed a level of evidence grading system (ASPS Evidence Rating Scales) to help plastic surgeons to assess the literature critically6,7. In Brazil, editorials published in the Brazilian Journal of Plastic Surgery ("Revista Brasileira de Cirurgia Plástica," RBCP)8,9 and the Brazilian Journal of Craniomaxilofacial Surgery10 also warned of the need to apply EBM in the practice of plastic surgery, and related areas of activity. In this vein, leaders of various societies, scientific journals, and academic institutions participated in 2 evidence-based plastic surgery summits; the first (2010) signaled a change in approach of plastic surgery to EBM, and was organized to promote EBM within the specialty, while the second summit (2012) culminated in a consensus in terms of the principles, priorities, and recommendations for the maintenance and dissemination of EBM in plastic surgery11,12. In 2013, the Journal of the American Medical Association Facial Plastic Surgery (JAMA FPS; formally, Archives of Facial Plastic Surgery), in partnership with the EBM committee of the American Academy of Facial Plastic and Reconstructive Surgery (AAFPRS), reported the inclusion of a new section, "Facing the Evidence", which contains a summary of the best evidence available for specific clinical situations13.

It is important to mention that in order to practice evidence-based plastic surgery successfully, plastic surgeons should have an understanding of both the hierarchy of evidence (known as levels of evidence) and different study designs, and be able to apply 2 fundamental principles: (1) decision-making in evidence-based plastic surgery to be undertaken with reference to a hierarchy of evidence (Figure 1); however, (2) consideration of the evidence in isolation is not enough because plastic surgeons should weigh the risks, benefits, drawbacks, and costs of any given approach, and consider their own clinical experience and preferences, and the patient's values (Figure 2)1,4,14-16. In many situations in plastic surgery, the best available level can be characterized as low impact, but its mention in this perspective tends to emphasize the study. Added to this, the practice of plastic surgery with reference to levels of evidence allows plastic surgeons to place the results of a study in context, and to consider the benefits and limitations inherent in different study designs16,17.


Figure 1 - The evidence hierarchy pyramid. Study designs that represent the highest level of evidence (studies potentially with less bias) are at the top and those representing lower levels of evidence (studies potentially with more bias) are placed at the bottom. Randomized controlled trial (RCT); Systematic review (SR) with or without meta-analysis.


Figure 2. Overlapping components (best available scientific evidence, preferences, and values of the patient, and clinical experience and knowledge) of evidence-based medicine (EBM). These 3 elements must be used simultaneously to successfully employ EBM in clinical practice.



As levels of evidence have been increasingly used to emphasize the importance of appropriate study designs17, several bibliometric analyses have been conducted to characterize the study designs and the hierarchy of evidence of scientific articles published in numerous Brazilian journals, including Acta Ortopédica Brasileira, Revista Brasileira de Ortopedia, Revista Brasileira de Medicina do Esporte, Arquivos Brasileiros de Cardiologia, and Revista Brasileira de Cirurgia Cardiovascular18-20. In the field of plastic surgery, such analyses have considered various international journals, including PRS, JAMA FPS, the Journal of Plastic, Reconstructive, and Aesthetic Surgery (JPRAS), the Aesthetic Surgery Journal (ASJ), Aesthetic Plastic Surgery (APS), the Canadian Journal of Plastic Surgery (CJPS), Annals of Plastic Surgery (AnPS), and the European Journal of Plastic Surgery (EJPS)17,21-27. However, although evidence-based plastic surgery has been addressed previously8-10, no such analysis has been undertaken with reference to Brazilian plastic surgery.

Therefore, the present bibliometric study (part III) aimed to analyze the study designs and levels of evidence of articles published in the RBCP, the official journal of the Brazilian Society of Plastic Surgery (BSPS).


METHODS

A bibliometric, quantitative analysis28 was performed to evaluate the study designs and levels of evidence of scientific articles published in the RBCP between 2005 and 2012 (volume 20, number 1-volume 27, number 4). The study period was divided into 2 periods (2005-2008 vs. 2009-2012) in order to characterize the evolution of the scientific production of the RBCP. Since the present investigation was not intended to ascertain the appropriateness of individual study designs, or judge the quality of the information included in each article, no further analysis was performed. Additional information, such as search strategies, data extraction, and inclusion and exclusion criteria, has been detailed in Part I of the bibliometric analysis.

Study designs

All articles included were categorized according to study design (randomized controlled trial [RCT], systematic review, simple review, prospective study, retrospective study, case series, or case reports) using a previously published system29. Some study designs, such as expert opinions, were classified as "other studies" to facilitate the compilation and presentation of data, as the level of evidence was unknown6,7. Furthermore, care was taken to define RCTs (methodological bases and appropriate statistical analyses) and systematic reviews (analysis with well-established methods, such as the PICO, STARLITE, and PRISMA strategies), as these are important factors in defining the hierarchy of evidence4,7,16. Classification of the epidemiological study design of each article was carried out by assessment of the elements reported in each study. Articles were not assessed for quality, and as such, no analysis tool for assessing study design quality was applied4.

Levels of evidence

The main pillar of EBM is the hierarchical system of classification of evidence, often referred to as "levels of evidence"11,16. According to this system, studies are hierarchically classified according to the perceived probability of bias16. Since the Canadian Task Force on the Periodic Health Examination described the first system of classifying levels of evidence in the 1970s30, several systems or scales of classification for levels of evidence have been proposed. Adapted versions of these systems have been adopted by several medical societies (for example, ASPS and AAFPRS) and periodicals such as PRS, JAMA FPS, ASJ, APS, CJPS and EJPS6,7,13,17,31,32. Although there are some differences between the systems, they are for the most part similar to the scale developed by the ASPS4,7,16. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group33 concluded that there is no universal classification system for levels of evidence; as such, we adopted the scales developed by ASPS4,6,7 as tools for standardizing the analysis of levels of evidence of all articles included, because this is a classification system adapted and applied specifically in the field of plastic surgery4,6,7,27. The categorization of levels of evidence (I, II, III, IV or V)4,6,7 was carried out by evaluating the information reported in each article. The levels of evidence I and II were considered high levels of evidence, while the levels of evidence III, IV and V were considered low levels of evidence24,26,27. The overall weighted mean level of evidence was calculated according to the following formula24: Percentage of articles of level of evidence × level of evidence/100.

Statistical analysis

All information was compiled in the Excel 2013 program for Windows (Office Home and Student 2013, Microsoft Corporation, USA) and subjected to statistical analysis using analysis of variance (ANOVA), equality of 2 proportions, and confidence interval for the mean, with the aid of the Statistical Package for Social Sciences for Windows, 16th version (SPSS, Chicago, IL, USA). Values were considered significant at a confidence interval of 95% (p < 0.05).


RESULTS

A total of 603 articles published in the RBCP between 2005 and 2012 were included in this bibliometric study, equivalent to 83.52% of total articles published during the period. Of these, 187 (31.01%) were published in 2005-2008 and 416 (68.99%) in 2009-2012.

Study designs

The overall analysis showed a significant predominance of retrospective studies (278 articles), followed by case reports (103 articles), case series' (76 articles), prospective studies (60 articles), other studies (58 articles), and simple reviews (28 articles) (p 0.001 for all comparisons) (Figure 3). Comparison between the 2 periods showed a significant increase in the proportion of case series' (p 0.05) and no significant differences in the proportions of prospective studies (p > 0.05), retrospective studies (p > 0.05), simple reviews (p > 0.05), case reports (p > 0.05), and other studies (p > 0.05) (Table 1 and Figure 4).


Figure 3. Study designs of the scientific articles published in the Brazilian Journal of Plastic Surgery between 2005 and 2012 (n = 603). All comparisons (retrospective studies > case reports > case series = prospective studies = other studies > simple reviews) showed a statistically significant difference (p < 0.01), with the exception of prospective studies versus number of cases (p = 1.0), prospective studies versus other studies (p = 0.268), and case series' versus other studies (p = 0.268).




Figure 4. Study designs of the scientific articles published in the Brazilian Journal of Plastic Surgery, distributed according to year of publication (n = 603)



Levels of evidence

The weighted mean of the level of evidence of the articles published in the RBCP between 2005 and 2012 was 3.65. During this period, there was a significant predominance of scientific articles with level of evidence III (278 articles; p < 0.001), followed by articles with levels of evidence V (189 articles; p < 0.001), IV (76 articles; p < 0.001) and II (60 articles; p < 0.001) (Figure 5). The analysis also revealed a significant predominance of articles with low level of evidence (levels of evidence III, IV, and V, 90.05%) when compared with articles with a high level of evidence (level of evidence II, 9.95%; p < 0.05). The comparison between the periods showed a significant increase in the proportion of articles with level of evidence IV (p < 0.05) and a significant reduction in the proportion of articles with level of evidence V (p < 0.05). The proportion of articles with levels of evidence II and III did not differ significantly between the 2 periods (p > 0.05; Table 1 and Figure 6).


Figure 5. Levels of evidence of scientific articles published in the Brazilian Journal of Plastic Surgery between 2005 and 2012 (n = 603). All comparisons (level of evidence III > level of evidence V > level of evidence IV = level of evidence II) showed a statistically significant difference, with the exception of level of evidence II versus level of evidence IV (p = 0.145).


Figure 6. Levels of evidence of scientific articles published in the Brazilian Journal of Plastic Surgery, distributed according to the year of publication (n = 603).



DISCUSSION

For decades, there has been an awareness of the existence of a gap between the results of scientific research and their application in medical practice, and the consequence of such gaps terms of inefficiency, costs and, even harm to patients4. EBM offers the opportunity to question practice, and locate and evaluate the relevant literature, enabling the utilization of scientific results in day-to-day medical practice4.

Despite the clear need for EBM in clinical practice, it was negatively received in the field of plastic surgery, with initial reluctance to adopt it, both due to the fear of the unknown, and a sense of outrage that surgical results would be subjected to perhaps undue scrutiny, leading to a belief that "EBM only serves non-surgeons"11,14. In addition, it may seem difficult to incorporate EBM into plastic surgery practice, given that it is not always possible to evaluate the results of interventions in a specialty that depends on artistic creativity1. However, EBM should be appreciated as a welcome scientific addition to the art of plastic surgery that complements clinical experience through an efficient and judicious application of surgical literature, rather than replacing the judgment of the individual surgeon6. In fact, EBM can benefit not only patients, but also surgeons themselves11,14; by applying the principles of EBM, it is possible to objectively analyze scientific results and determine the best treatment options, while enabling continual evaluation of the results of interventions, thus ensuring that the best available approach is offered to the patient34.

With this in mind, editors of scientific journals have stressed the need to incorporate and maintain the practice of EBM within plastic surgery8,10,11,14,34. Goldenberg and Baroudi presented some principles of EBM to the readers of the RBCP and encouraged its application8. Rohrich stated that EBM is the key to the future, for those who really want to become better doctors and plastic surgeons, providing appropriate patient care34. Rohrich et al. highlighted that the practice of EBM is a life-long process that should be incorporated into the daily routine of all plastic surgeons11. Similarly, Alonso stated that surgeons should always seek the best evidence to support their surgical decisions, and suggested that further research should be undertaken in this area10. Recently, Rhee and Daramola argued that EBM is not a fad, but a tool that should be adopted in the present and the future14.

Reflecting these ideas, the participants of the first evidence-based plastic surgery summit established that the incorporation of EBM in plastic surgery is essential to the provision of continuous improvements in safety and quality of care, and should be a priority for organizations, leaders, researchers, educators, surgeons, and the public11. At the same summit, it was concluded that standard definitions of levels of evidence should be incorporated and applied by the specialty; this is considered a key factor for the full inclusion of EBM in plastic surgery11. Consistent with these goals, the characterization of levels of evidence of articles published in scientific journals is intended to improve the overall quality of medical care, therapeutic outcomes, and patient safety, as well as help editors, reviewers, authors, and readers in the critical assessment of the profile of scientific evidence of articles published in specific journals7.

Thus, given that the analysis of study designs and levels of evidence in the field of plastic surgery field are limited to the international arena17,21-27, we evaluated 603 scientific articles published in the RBCP, the only Brazilian periodical devoted exclusively to plastic surgery, between 2005 and 2012.

Since the vast majority of the articles included in the full bibliometric study (Parts I, II and III) originated from Brazil (98.67%), the information presented in here is specific to national plastic surgery, undertaken in Brazilian institutions/by Brazilian plastic surgeons. In addition, the evaluation of articles published in plastic surgery journals allows reflection on advances in the specialty35; to the best of the authors' knowledge, this is the first national research that provides an evolutionary profile of Brazilian evidence-based plastic surgery, with reference to study designs and levels of evidence. As notes in Parts I and II, such work also makes it possible to set goals for the future.

While the goal is to improve the level of evidence in plastic surgery16,21, articles with a high level of evidence (I and II) and those with low level of evidence (III, IV and V) both have notable roles in decision-making in plastic surgery6,7,16. We shall therefore discuss in detail the results of this analysis, as well as the main study designs and their respective levels of evidence, highlighting important aspects relating to evidence-based decision-making, and its main limitations. We will also include an overview of the main results of the most relevant research on this subject.

Study designs

The present bibliometric analysis revealed a significant predominance of retrospective studies (46.10%) over the other study designs; this is distinct from the proportions reported in international bibliometric studies (Tables 2 and 3). Comparison between the 2 periods showed a significant increase only in the proportion of case series' (2005-2008 < 2009-2012). This result is similar to the proportion described in a previous analysis of cosmetic surgery from the PRS, AnPS, JPRAS, APS, and ASJ (43.7% in 2000; 49.4% in 2005; and 53.9% in 2009-2010) 26. Another study, which also assessed articles on cosmetic surgery published in the PRS, AnPS, and ASJ, showed an increase in the proportion of retrospective studies (4.6% in 1998-2002 and 7.8% in 2002-2007), while the proportion of prospective studies was relatively stable (4.3% in 1998-2002 and 4.2% in 2002-2007)23.






In the present investigation, we found no RCTs or systematic reviews, with or without meta-analysis. Bibliometric studies, both national18-20 and international (Table 3), have revealed that the proportion of such study designs varies according to the journals and the periods evaluated. For example, national analyses19,20 (Acta Orthopaedic Brasileira, Revista Brasileira de Medicina do Esporte, among others) showed that 0-1.38% and 1.38-9.75% of the published articles were systematic reviews and clinical trials, respectively. Furthermore, bibliometric analyses that included different study designs showed an increase in the proportion of RCTs over time (PRS, AnPS and ASJ, 1.2% in 1998-2002 and 4.8% in 2002-200723; JAMA FPS, PRS, Laryngoscope, Otolaryngology-Head and Neck Surgery, 0.8% in 1999 and 1.2% in 2008)25; however, this increase was not significant (p = 0.982)25. An analysis exclusively dedicated to RCTs published in PRS, AnPS, and JPRAS, meanwhile, revealed a significant increase in recent decades (1966-2003 < 2004-2008)36.

Levels of evidence

The present bibliometric study showed that the weighted mean of the level of evidence of the articles published in the RBCP in 2005-2012 (3.65) was higher than that reported for the PRS (3.05), ASJ (3.11), AnPS (3.31), and JPRAS (3.35) in 200724, but lower than the mean reported in an evaluation of the PRS in 1983 and 2003 (4.42 and 4.16, respectively)21. The weighted mean of the present study was also higher than that reported for orthopedics (2.9) and ophthalmology (3.1)24, but similar to that described in otorhinolaryngology (3.6). Although the weighted means differed between journals21,24, our overall result (3.65) was very close to the level of evidence III, as reported in an international analysis of the fields of plastic surgery, otolaryngology, and ophthalmology24.

In evaluating the proportion of levels of evidence, we found that the vast majority (90.05%) of the included articles had a low level of evidence (levels of evidence III, IV and V), while only 9.95% had a high level of evidence (level of evidence II). Other studies of journals of plastic surgery (PRS, ASJ, AnPS, APS, JPRAS, and CJPS) showed a similar pattern, but with different percentages (5.9-32.5% and 67.5-94.1% of the analyzed articles had high or low levels of evidence, respectively), with results depending on the year and the classification system adopted17,21,24,26,27.

Between 2005 and 2012, no article in the RBCP was rated with a level of evidence I; this is similar to the findings of analyses of journals of plastic surgery in isolated years (ASJ in 2007, PRS in 1983 and 2000, and PRS, JAMA FPS, and Laryngoscope and Otolaryngology-Head and Neck Surgery in 2002)21,24-26. The proportion of articles with a level of evidence I in international plastic surgery (Table 4) is less than that reported in specialties such as orthopedics (21%), ophthalmology (18%), and otolaryngology (7%)24.




S in 1983 (0%), 1993 (0.4%), and 2003 (1.5%)21; in PRS, AnPS, JPRAS, APS, and ASJ in 2000 (0%) and 2009-2010 (2.6%)26; and in PRS, JAMA FPS, and Laryngoscope and Otolaryngology-Head and Neck Surgery in 1999 (0.8%), 2002 (0%), 2005 (0.4%), and 2008 (1.2%)25. However, these increases were not statistically significant (p = 0.28 and p = 0.982)25,26. A study that assessed only articles with level of evidence I published in PRS, AnPS, APS, and BJPS/JPRAS found a steady increase in recent decades (1978-2009)37; again, this was not statistically significant.

In the present study the proportion of articles with a level of evidence II (9.95%) was lower than that of some previous investigations in the field of plastic surgery (12.8-31%; Table 4) and orthopedics (15%), but higher than that described in analyses of CJPS (5.81%)17 and in other areas including ophthalmology (8%) and otolaryngology (7%)24. The proportion in the subfield of cosmetic surgery is variable (4.2-13.7%)23,26. Furthermore, studies evaluating articles in plastic surgery journals over more than 1 period revealed variable proportions of level of evidence II (PRS, 3.4% in 1983, 5.0% in 1993, and 4.4% in 2003; PRS, AnPS, JPRAS, APS, and ASJ, 15.9% in 2000, 12.4% in 2005 and 13.5% in 2009-2010)21,26. These results are similar those of our investigation, in which the comparison between periods showed no significant difference in the proportion of articles with level of evidence II (2005-2008 = 2009-2012).

Our analysis showed a predominance of articles with level of evidence III (46.10%), similar to that reported in general plastic surgery (31-41%; Table 4), and higher than that reported in an evaluation of CJPS between 2007 and 2011 (1.74%)17, in cosmetic surgery (6.3-10.8%)23,26, and in other surgical specialties including otolaryngology (6%), orthopedics (16%), and ophthalmology (16%)24. In the present study, in the comparison between the periods, we did not detect any difference in the proportion of articles with level of evidence III (2005-2008 = 2009-2012). Other investigations have shown an increasing trend in the proportion of articles with level of evidence III (PRS, 4.3% in 1983 and 7.2% in 2003; PRS, AnPS, and ASJ, 4.6% in 1998-2002 and 7.8% in 2003-2007; PRS, AnPS, JPRAS, APS, and ASJ, 10.3% in 2000 and 12.2% in 2009-2010); however, no statistical analysis was performed to confirm these findings21,23,26.

Our study also showed that a substantial proportion of articles (43.94%), published in RBCP in 2005-2012 were low in the hierarchy of levels of evidence, i.e. level of evidence IV or V. This proportion is greater than that described in an analysis of the articles published in the PRS, APS, JPRAS, and AnPS between 2008 and 2012 (30.5%)27, but lower than that reported in other analyses in the field of plastic surgery (Table 4). Similarly, this ratio is also variable in other specialties (otorhinolaryngology, 80%; ophthalmology, 58%; and orthopedics, 48%)24. In the comparison between the periods, there was a significant increase in the proportion of articles with level of evidence IV, and a significant reduction in the proportion of articles with level of evidence V. Two previous studies also showed this trend, but without statistical confirmation21,26.

The discrepancies between the proportions of levels of evidence found in plastic surgery and other specialties is deserving of discussion. The diversity of diagnoses and procedures found in the daily practice of plastic surgeons, and the frequent need to adapt or tailor treatments for patients with particular medical conditions can lead to subsequent publication of case series' or case reports (level of evidence IV and V), which may explain this discrepancy24. In addition, the lack of academic plastic surgeons who are exclusively dedicated to research can result in a dependency upon methods of research that require minimal funding and time, namely retrospective studies or reviews of medical records (levels of evidence III and IV); this may result in the skewed level of evidence present in the literature24.

Based on this understanding, we propose that the mentality of plastic surgery requires change: hypotheses should precede the collection of data in studies, and the correct sequence of submission to the ethics review committees, prospective implementation of data collection, analysis, and publication must prevail.

Evidence-based plastic surgery: beyond study designs and levels of evidence

When faced with a patient with a specific deformity, plastic surgeons should seek answers based on the best evidence available15. However, culturally, plastic surgery has been based on expert opinion (level of evidence V), mainly because the common sense and technical expertise of senior plastic surgeons are essential for the success of a cosmetic intervention6,34. While this current attitude will be maintained for the near future, it is important that plastic surgery evolves as a culture based on the critical application of the principles of EBM6. In this sense, according to the results of the present bibliometric analysis and other studies (Tables 2-4), plastic surgery has been slow with regard to the proportion of articles with a high level of evidence (RCTs and systematic reviews), compared with areas such as orthopedics and ophthalmology24. However, as noted above, the nature of plastic surgery means that all well-written articles have an important role in decision-making, independent of study design and respective level of evidence6,7,16.

Thus, while plastic surgeons may benefit from understanding the bibliometric data from this study and previous international investigations (Tables 2-4), caution should be exercised when embarking on this "new journey" toward evidence-based plastic surgery4. Plastic surgeons should master the limitations and biases of EBM4. This will allow them to carefully evaluate the available literature and the hierarchy of levels of evidence before making clinical decisions, and to improve the methodological rigor of studies and their resulting scientific articles6,13,14,16.

Well-conducted and reported RCTs occupy a prominent place in the hierarchy of levels of evidence (level of evidence I)5,36,37, and it is interesting to note that a recent investigation revealed that the majority (82%) of the RCTs published in international journals of plastic surgery have been conducted by North American or European researchers; the number of RCTs published by authors from countries of South America (10%), including Brazil, is limited. There is a need to increase the proportion of articles with level of evidence I (RCTs and consequently systematic reviews, with or without meta-analyses) in plastic surgery, especially in Brazil, as reinforced by the data reported in this study.

We propose that Brazilian plastic surgeons must undertake a commitment, similar to that reported by others5,22,36: "as plastic surgeons, we can assume a leading role in order to increase the production of unbiased evidence on the effectiveness of our surgical interventions"; "We should encourage plastic surgery to produce publications of high-level evidence, thereby promoting plastic surgeons as leading scientists in surgical disciplines."

However, it is important to note that there are limitations to the performance of RCTs in plastic surgery, including the learning curve involved in a practical specialty; difficulties standardizing surgical techniques; technical differences between surgeons; issues with randomization, and blinding of surgeons and patients; and loss to follow-up16,21,36. In addition, the results of these studies cannot be generalized to the entire population, as they tend to be conducted in highly specialized academic settings, under extremely rigorous inclusion and exclusion criteria5. Some studies have revealed methodological limitations, including a failure to report a sample size calculation, analyze on an intention to treat basis, or perform randomization or concealment of allocation; such limitations may be present in the conception, execution, and publication of RCTs. Thus, plastic surgeons should also consider observational studies, including case-control studies and prospective or retrospective cohort studies, with levels of evidence II and III, and participate in such studies. This is particularly the case when an RCT is not feasible (only 40% of treatment questions involving surgical procedures may be subject to RCTs5) and/or there is no RCT for a given specific issue1,16.

Although the results of observational studies are often criticized for their vulnerability to the influence of unpredictable confounding factors38, their level of evidence when designed, conducted, and reported properly can approach or exceed that of an RCT, and, in some situations, may achieve similar results14,16,38. For these reasons, observational studies have become increasingly popular in the investigation of the relationship between exposures (for example, risk factors or surgical interventions) and outcomes (for example, complications)38. With the increase in observational studies, systematic reviews of these studies will also increase, thus improving global levels of evidence in plastic surgery16. Additionally, observational studies can generate hypotheses, establishing issues that may form the basis for performing an RCT15,38.

An interesting analysis of the journals PRS, AnPS, BJPS, and Scandinavian Journal of Plastic and Reconstructive Surgery and Hand Surgery revealed that all 50 most-cited articles in plastic surgery have a level of evidence IV or V,19 demonstrating that plastic surgeons tend to more frequently cite case series', case reports, the opinions of experts, and other studies considered to have a low level of evidence19. From this perspective, it should be noted that the case series', case reports, and expert opinions also have a role in evidence-based plastic surgery, predominantly in the context of cosmetic surgery23,26. These study designs have been valuable to the practice of plastic surgeons as they may address technical refinements7 and permit the formulation of hypotheses that can support the development of controlled studies7,15,16.

Retrospective studies (level of evidence III) have been considered the most appropriate for assessing rarely-reported diseases, because they increase the chance of finding the patients with these diseases, as opposed to depending on such patients appearing in a prospective study (level of evidence II). Therefore, plastic surgeons can retrospectively analyze extended periods, and involve multiple institutions, in order to accumulate a large number of unusual cases. Consequently, isolated case reports (level of evidence V) can be transformed into a case series or retrospective study (level of evidence IV or V), thus increasing the overall level of evidence in plastic surgery.

Although the opinions of experts (level of evidence V) seem obsolete in the field of EBM, they can be very useful when no other evidence is available4. However, in order for the opinions of experts to be of use when making treatment decisions, consideration has to be given to how such opinion pieces are developed, particularly, the impartiality of the method by which the facts are evaluated, rather than the extent to which experts are persuasive4.

Although the present study is strictly a quantitative bibliometric analysis, another point for discussion concerns the distinction between the level of evidence and quality of study designs and publications13,24. Decision-making cannot be based only on the level of evidence, as the hierarchy of these levels does not address the external validity/generalizability of the results of a particular study13. Since this hierarchy is not absolute, articles with a high level of evidence do not necessarily equate to studies of high quality6,13,24. For example, a low-quality RCT (level of evidence II) may be more likely to contain misleading mistakes than a well-designed cohort study (level of evidence III)6; similarly, a case series (level evidence IV) of a given intervention may have more of an impact than a poorly designed RCT (level of evidence II)24.

In addition, limitations in the presentation of information, even in well-conducted studies, can have a negative impact on the overall quality of an article4,16,38. Thus, plastic surgeons cannot assume that the level of evidence I is always the best or most appropriate choice for a specific question24. Both for the critical analysis of published studies and for the execution of new studies, plastic surgeons should be aware of various methodological aspects38 and should adopt, whenever possible, appropriate tools (for example, QUOROM, STROBE, CONSORT) to outline the fundamental peculiarities of a particular study design, and improve the transparency and presentation of information4,17,24,38.

The results and discussion of part III of the present study raises the need for educational initiatives to motivate evidence-based plastic surgery among Brazilian plastic surgeons. Thus, RBCP and SBCP can focus their efforts on introducing and maintaining the concepts of evidence-based plastic surgery for SBCP members and for RBCP authors and readers, as other journals and plastic surgery societies have done (for example, PRS/ASPS, JAMA FPS/AAFPRS)5,7,13. To that end, strict rules, such as requiring the members of SBCP to publish scientific articles in RBCP, may be accompanied by initiatives to improve the understanding and applicability of the principles of EBM among Brazilian plastic surgeons. This should result not only in an increase in the number of articles, but also an improvement in the quality of submissions, therefore improving the overall level of evidence of RBCP.

Included among the list of possible educational initiatives are:

  • Assign levels of evidence for all articles published in RBCP, as has been adopted in various journals6,7,13,17,31,32;
  • Include a special section (or publish articles regularly) dedicated to EBM, to educate authors and readers on the critical evaluation of the surgical literature, and also to disseminate tips (for example, methodological and statistical principles) on the practice of evidence-based plastic surgery;
  • Request critical and unbiased discussions from experts in the field of EBM, to determine if the interventions/technologies proposed in selected articles can be applied to patient care;
  • Offer courses on evidence-based plastic surgery5,7,26.


    • Similar measures have been adopted by periodicals in the field of orthopedics and urology, including The Journal of Bone and Joint Surgery and the British Journal of Urology International; as a result, within a few years, the proportion of articles with a high level of evidence almost doubled26. Therefore, we believe that RBCP and SBCP should follow these examples, aware that achieving EBM in plastic surgery will take some time11, and will be achieved only with training and education7.

    The present bibliometric study has limitations that should be addressed. An important limitation is that the studies conducted by Brazilian plastic surgeons published in other journals were not considered, and therefore, the actual proportion of study designs and levels of evidence published by Brazilian plastic surgeons is likely to differ from that presented here. However, as we restricted the study to articles published in RBCP, which are generally authored by Brazilian plastic surgeons, our results allow specific conclusions at the national level, despite a lack of generalizability to other journals. In addition, the articles published by Brazilian plastic surgeons in international journals are part of other investigations36 and are therefore beyond the scope of bibliometric analysis of RBCP.

    We classified all included items according to the definitions set out in the chosen scales, used as standardization tools4,6,7,27,29, but are aware that other authors19,20 have adopted different definitions and subcategories of, for example, a clinical trial (clinical trial, a quasi-randomized clinical trial, or an RCT). However, it is important to note that, as there are differences in the definitions of study designs and in the hierarchy of levels of evidence, as well as lack of confirmation of findings with statistical analyses in studies similar to ours17,21-27, comparisons between studies also have limitations.

    As with other investigations17,21,23,24,26, we did not perform a qualitative assessment of the included articles and, therefore, any extrapolations discussed in this study (for example, whether studies with levels of evidence III can be compared with those with level II evidence16,38) should be examined case-by-case. Another limitation is that we only evaluated study designs and levels of evidence, although there are numerous other aspects that are relevant to the practice of evidence-based plastic surgery, including the validity, impact, and applicability of studies4. Therefore, future research should be conducted to address these limitations, with the aim of further increasing our understanding and dissemination of Brazilian evidence-based plastic surgery.


    CONCLUSION

    This bibliometric study (Part III) revealed that the global average of evidence of scientific articles published in RBCP between 2005 and 2012 was 3.65 (level of evidence III), with a significant predominance of retrospective studies. Over the period studied there was a significant increase in the proportion of case series' (level of evidence IV), and an equally significant reduction in the proportion of articles with level of evidence V.


    REFERENCES

    1. Kowalski E, Chung KC. The outcomes movement and evidence-based medicine in plastic surgery. Clin Plast Surg. 2013;40(2):241-7.

    2. Chung KC, Ram AN. Evidence-based medicine: the fourth revolution in American medicine? Plast Reconstr Surg. 2009;123(1):389-98.

    3. Godlee F. Milestones on the long road to knowledge. BMJ. 2007;334(Suppl 1):s2-s3.

    4. Swanson JA, Schmitz D, Chung KC. How to practice evidence-based medicine. Plast Reconstr Surg. 2010;126(1):286-94.

    5. Chung KC, Swanson JA, Schmitz D, Sullivan D, Rohrich RJ. Introducing evidence-based medicine to plastic and reconstructive surgery. Plast Reconstr Surg. 2009;123(4):1385-9.

    6. Eaves F 3rd, Pusic AL. Why evidence-based medicine matters to aesthetic surgery. Aesthet Surg J. 2012;32(1):117-9.

    7. Sullivan D, Chung KC, Eaves FF 3rd, Rohrich RJ. The level of evidence pyramid: indicating levels of evidence in Plastic and Reconstructive Surgery articles. Plast Reconstr Surg. 2011;128(1):311-4.

    8. Goldenberg DC, Baroudi R. Medicina Baseada em Evidências: como e quando em cirurgia plástica (Editorial). Rev Bras Cir Plást. 2009;24(1):1.

    9. Goldenberg D, Baroudi R. Artigos originais versus relatos de caso. Hierarquia dos níveis de evidência científica. Rev Bras Cir Plást. 2011;26(3):378.

    10. Alonso N. Cirurgia crânio-maxilo-facial baseada em evidências. Rev Bras Cir Craniomaxilofac. 2010;13(1):1.

    11. Rohrich RJ, Eaves FF 3rd. So you want to be an evidence-based plastic surgeon? A lifelong journey. Plast Reconstr Surg. 2011;127(1):467-72.

    12. Eaves FF 3rd, Rohrich RJ, Sykes JM. Taking evidence-based plastic surgery to the next level: report of the second summit on evidence-based plastic surgery. Plast Reconstr Surg. 2013;132(1):227-35.

    13. Rhee JS, Larrabee WF. Facing levels of evidence: the JAMA Facial Plastic Surgery initiative. JAMA Facial Plast Surg. 2013;15(3):174-5.

    14. Rhee JS, Daramola OO. No need to fear evidence-based medicine. Arch Facial Plast Surg. 2012;14(2):89-92.

    15. Sprague S, McKay P, Thoma A. Study design and hierarchy of evidence for surgical decision making. Clin Plast Surg. 2008;35(2):195-205.

    16. Burns PB, Rohrich RJ, Chung KC. The levels of evidence and their role in evidence-based medicine. Plast Reconstr Surg. 2011;128(1):305-10.

    17. Thoma A, Ignacy TA, Li YK, Coroneos CJ. Reporting the level of evidence in the Canadian Journal of Plastic Surgery: Why is it important? Can J Plast Surg. 2012;20(1):12-6.

    18. Moraes VY, Belloti JC, Moraes FY, Galbiatti JA, Palácio EP, Santos JB, et al. Hierarchy of evidence relating to hand surgery in Brazilian orthopedic journals. Sao Paulo Med J. 2011;129(2):94-8.

    19. Torloni MR, Riera R. Design and level of evidence of studies published in two Brazilian medical journals recently indexed in the ISI Web of Science database. Sao Paulo Med J. 2010;128(4):202-5.

    20. Riera R. Designs of studies published in two Brazilian journals of orthopedics and sports medicine, recently indexed in the ISI Web of Science. Sao Paulo Med J. 2009;127(6):355-8.

    21. Loiselle F, Mahabir RC, Harrop AR. Levels of evidence in plastic surgery research over 20 years. Plast Reconstr Surg. 2008;121(4):207e-11e.

    22. McCarthy CM, Collins ED, Pusic AL. Where do we find the best evidence? Plast Reconstr Surg. 2008;122(6):1942-7.

    23. Chang EY, Pannucci CJ, Wilkins EG. Quality of clinical studies in aesthetic surgery journals: a 10-year review. Aesthet Surg J. 2009;29(2):144-7.

    24. Sinno H, Neel OF, Lutfy J, Bartlett G, Gilardino M. Level of evidence in plastic surgery research. Plast Reconstr Surg. 2011;127(2):974-80.

    25. Xu CC, Côté DW, Chowdhury RH, Morrissey AT, Ansari K. Trends in level of evidence in facial plastic surgery research. Plast Reconstr Surg. 2011;127(4):1499-504.

    26. Chuback JE, Yarascavitch BA, Eaves F 3rd, Thoma A, Bhandari M. Evidence in the aesthetic surgical literature over the past decade: how far have we come? Plast Reconstr Surg. 2012;129(1):126e-134e.

    27. Tahiri Y, Kanevsky J, Vorstenbosch J, Mok E, Gilardino M. Disclosure of funding source and conflict of interest: exposure of biases affecting evidence and clinical utility of plastic surgery publications. Eur J Plast Surg. 2012;35(6):457-62.

    28. Durieux V, Gevenois PA. Bibliometric indicators: quality measurements of scientific publication. Radiology. 2010;255(2):342-51.

    29. Maran AG, Molony NC, Armstrong MW, Ah-See K. Is there an evidence base for the practice of ENT surgery? Clin Otolaryngol Allied Sci. 1997;22(2):152-7.

    30. The periodic health examination Canadian Task Force on the Periodic Health Examination. Can Med Assoc J. 1979;121(9):1193-254.

    31. Spinelli HM. Introducing Aesthetic Plastic Surgery to Evidence-Based Medicine. Aesthetic Plast Surg. 2011;35(2):145-6.

    32. Mayer HF. Commentary on "Disclosure of funding source and conflict of interest: exposure of biases affecting evidence and clinical utility of plastic surgery publications" by Y. Tahiri et al. Eur J Plast Surg. 2012;35(6):463.

    33. Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, et al. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res. 2004;4(1):38.

    34. Rohrich RJ. So you want to be better: the role of evidence-based medicine in plastic surgery. Plast Reconstr Surg. 2010;126(4):1395-8.

    35. Huemer GM, Bauer T, Gurunluoglu R, Sakho C, Oehlbauer M, Dunst KM. Analysis of publications in three plastic surgery journals for the year 2002. Plast Reconstr Surg. 2004;114(5):1147-54.

    36. Veiga DF, Veiga-Filho J, Pellizzon RF, Juliano Y, Ferreira LM. Evolution of reports of randomised clinical trials in plastic surgery. J Plast Reconstr Aesthet Surg. 2011;64(6):703-9.

    37. McCarthy JE, Chatterjee A, McKelvey TG, Jantzen EM, Kerrigan CL. A detailed analysis of level I evidence (randomized controlled trials and meta-analyses) in five plastic surgery journals to date: 1978 to 2009. Plast Reconstr Surg. 2010;126(5):1774-8.

    38. Song JW, Chung KC. Observational studies: cohort and case-control studies. Plast Reconstr Surg. 2010;126(6):2234-42.

    39. Loonen MP, Hage JJ, Kon M. Plastic Surgery Classics: characteristics of 50 top-cited articles in four Plastic Surgery Journals since 1946. Plast Reconstr Surg. 2008;121(5):320e-327e.










    1 - MD - Aspiring member in training of the Brazilian Society of Plastic Surgery (Sociedade Brasileira de Cirurgia Plástica - SBCP), Resident Physician in Plastic Surgery, Plastic Surgery Service "Prof. Dr. Cassio M. Raposo do Amaral" of the Craniofacial Plastic Surgery Institute of the Hospital SOBRAPAR, Campinas, SP, Brazil
    2 - MD, PhD - Member of the SBCP, Associate Professor Faculdade de Medicina da Universidade de São Paulo (FMUSP), CoEditor of the Brazilian Journal of Plastic Surgery (Revista Brasileira de Cirurgia Plástica - RBCP)
    3 - MD, PhD - Member of the SBCP and the Brazilian Association of Cranio-maxillofacial Surgery (ABCCMF), Doctorate by the Clinical Surgery Program at the University of São Paulo (USP), Vice-President of the Craniofacial Plastic Surgery Institute of the Hospital SOBRAPAR, Campinas, SP, Brazil

    Institution: Study conducted at the Institute of Craniofacial Plastic Surgery of Hospital SOBRAPAR, Campinas, SP, Brazil.

    Corresponding author:
    Rafael Denadai
    Hospital de Crânio e Face SOBRAPAR
    Av. Adolpho Lutz, 100 - Cidade Universitária
    Campinas, SP, Brazsil - CEP 13083-880; Caixa-postal 6028
    E-mail: denadai.rafael@hotmail.com

    Article received: August 17, 2013
    Article accepted: July 14, 2014

     

    Previous Article Back to Top Next Article

    Indexers

    Licença Creative Commons All scientific articles published at www.rbcp.org.br are licensed under a Creative Commons license