Assessment of an electronic mailing list for orthopaedic and trauma surgery

GEORGE J MCLAUCHLAN, MIKE CADOGAN, CHRISTOPHER W OLIVER
Edinburgh Orthopaedic Trauma Unit (http://www.orthopaedic.ed.ac.uk/eotu.htm), Royal Infirmary of Edinburgh, Edinburgh

Medical electronic mailing lists have grown rapidly with the evolution of the Internet. The purpose of this paper was to assess the content and academic quality and of an electronic mail discussion list “Orthopod” dedicated to research in orthopaedic and trauma surgery. A retrospective analysis was undertaken of four months of webbased archive of communication through the Orthopod mailing list http://www.mailbase.ac.uk/lists/Orthopod At the end of this study period there were 505 members of the Orthopod list. The main outcome measures of this paper were number of messages, type of messages, number of contributors, designation of contributor and number of topics referenced. On average 276 messages were posted each month. The mean number of messages per contributor per month only varied between 2.3 and 2.5. On average 54% of message threads contained at least one reference. At least 77% of the patient history and general threads had input from senior orthopaedic staff. The quantity of information on the Internet need not be a substitute for quality in orthopaedic and trauma surgery mailing lists. The Orthopod Electronic mail discussion list is another medium for the circulation of good quality surgical information. Orthopod provides a unique international electronic forum list for trauma and orthopaedic surgeons worldwide.

Keywords: Internet, electronic mailing list, orthopaedics, trauma communication

J.R.Coll.Surg.Edinb.44, Feb 1999, 36-39

INTRODUCTION

Internet technology is rapidly expanding and now provides access to an enormous volume and variety of health information. This information may be from a leading expert with excellent documentation and a complete bibliography or it may be in the form of encouragement from a friendly support group. This makes it difficult for the user to determine which information is usable and credible; how it can be evaluated, critiqued, or verified; when it should be ignored, rejected, debunked, or erased; and whether to read, print, file, or down-load it.¹ Thus, while there is a wealth of health information available, there is no guarantee of its quality.^2-6

The Internet as a means of acquiring health information for medical professionals is rapidly expanding. In trauma and orthopaedic surgery it is now impossible to identify the total number of Websites providing specialist information. To tackle this problem initiatives such as the Orthogate⁷ project have attempted to define a single portal for all trauma and orthopaedic surgery information on the Internet. Electronic mailing lists for health professionals in trauma and orthopaedic surgery have been running since 1993 when the Orthopaedic Mailing List was created. This list has now closed, due to lack of technical support. In July of 1997 the electronic mailing list Orthopod⁸ was established. The Orthopod list was intended to be: an international academic mailing list for discussion, collaboration, sharing ideas and the dissemination of research in orthopaedics and trauma surgery, to provide an information sharing service for basic science and clinical practice research as well as to be a forum for reports of meetings, courses and workshops. Orthopod was intended only for surgeons, surgical healthcare workers and healthcare students. It was not intended to be a forum for patient information. The list is archived and searchable by author, thread, date and topic. A person who joined the list would receive a copy of any electronic mail placed on the list by a contributor. A reply could be made if so wished and this was automatically distributed to all other members on the list. The combination of a question or statement and any replies to it made up a thread.

Mailbase⁹ provided the core administration and Internet server support for Orthopod list. Online documentation to support the administration of the list is provided by Mailbase. The Orthopod list established its own user support WebPages.¹⁰ Members joining the Orthopod list were all sent an electronic introduction file that outlined the “netiquette” of the list. Members were requested to voluntarily register their membership details on an electronic sign-up form.¹⁰ Funding for the list is provided by the United Kingdom Joint Information Systems Committee¹¹ of the Higher Education Funding Councils for England, Scotland and Wales and the Department of Education of Education for Northern Ireland. Mailbase provides over 2134 Electronic mail discussion lists and has over 144,000 members¹² (July 1998). Mailbase discussion topics are very diverse. For Mailbase to provide an electronic mailing list the list must support United Kingdom higher education and research. The Orthopod list was included by the Organised Medical Network Information¹³ as a quality biomedical resource in August 1997.

The purpose of this study was to try and ascertain the activity of the mailing list and whether the Orthopod mailing list had any academic credibility.

METHOD AND MATERIALS

Four consecutive calendar months from January to April 1998 were chosen for the analysis. All messages had been previously archived¹⁴ by Mailbase. At this time the list had been running for six months. All messages over this four-month period were assessed. The general category of each message patient case history, general orthopaedic or trauma question, personal viewpoint, meetings, jobs, humour and other was recorded. The number of replies, the number of referenced threads and the length of discussion were recorded. Note was also made of the number of contributors in any month. If a discussion crossed over months it was counted in the month in which it started. It was not unusual for one thread to lead into another connected theme. For the purpose of this analysis if there was an obvious connection to the initial point or question then it was counted as one thread. Using information from the voluntary sign-up form¹⁵ note was made where possible of the designation of the most senior of the participants in a thread.

RESULTS

Over the four-month period a total of 1105 messages were posted. There were 114 duplicate or erroneous messages leaving 991 unique relevant messages. The number of subscribers grew from 200 to 505 over this period. Table 1 shows the number of contributors, the number of messages and the numbers and types of thread over the four months. The percentage of the members of the list contributing over the four months remained constant at around 30% despite a 250% increase in the membership. The mean number of messages per contributor per month only varied between 2.3 and 2.5. A small number of people were more prolific over the study period which left the median number of messages varying between 1 and 2 per person. Five per cent of contributors supplied 5 or more messages in January but this fell to 4% by April. Those supplying 10 or more messages remained at 1% over this time. By the end of April 1998 there were representatives from at least twenty-five countries on the list, although the majority of communication came from the UK, USA, Canada, Australia and Spain.

The two major types of discussion occurring were specific patient histories and general orthopaedic questions. These groups made up an average of 47% of the threads over the study period. Table 2 illustrates the average numbers of replies for each of this type of thread as well as the length of discussion. In these groups the number of referenced threads was examined. On average 54% of threads contained at least one reference. However, over the four months the percentage of referenced threads fell slightly from 58% to 51%.

Despite the difficulties in ascribing a designation to all the contributors, at least 77% of the patient history and general threads had input from senior orthopaedic staff, i.e. professor, consultant or senior lecturer. This level was unchanged over the four months.

Table 1: Contributors, threads and replies over study period

Table 2: Number of replies per thread and length of discussion.

DISCUSSION

The growth of the Internet has been phenomenal⁵ and this has been shown by the enormous growth of Orthopod during this study period. The central aim of the list was to produce a high quality academic electronic mailing list within orthopaedics and trauma. Other than this broad agenda and some points on Internet “netiquette”¹⁶ there was no particular control over members of the list.

The need to measure, and the ways one can go about measuring, the quality of health information on the Internet sites has been described.^1,2.4 A major difficulty is that, in assessing the quality of a Web site, one is often trying to apply methods applicable to a peer-reviewed journal to a completely different medium. Silberg described four core standards to allow readers to decide whether what they were reading was credible, reasonable or useful.³ These are authorship, attribution, disclosure and currency (Table 3). Other more detailed quality measures have been described by the Health on the Net Foundation¹⁷ (HON) and the Organisation for Medical Networked Information (OMNI).¹⁸ It is not possible to generally apply these criteria to an electronic mailing list, as these criteria were designed more for web-based media.

Table 3: Silberg’s core criteria for measuring quality

The 50% of the messages concerning cases and general orthopaedic and trauma topics almost invariably came from either senior or junior orthopaedic surgeons, although a small number of questions came from patients. At least 77% of threads had input from senior orthopaedic surgeons. Throughout the study period the percentage of members taking part in the discussions remained consistent at around 30%.

Most members made one or two contributions with only a few higher-level users. This is similar to the experience of others.⁵ Applying the Silberg principles, the Orthopod list is deficient when it comes to identifying authorship and, more particularly, their credentials. This would readily be addressed by having such details compulsorily registered before a member was able to join, always assuming their honesty. It would be impossible to verify the professional status of every surgeon or healthcare worker that joined the list worldwide.

The forum does lend itself to international discussion format between colleagues and, perhaps not surprisingly, the majority of advice given was anecdotal, based on personal experience, albeit from some surgeons with a lifetime’s experience of orthopaedics. Over the study period, however, 54% of the threads concerning patient cases and general points or queries contained at least one reference to the peer reviewed literature. Again, if measured against the Silberg principles, the majority of correspondence fails the attribution quality criteria. However, laudable as it may be for every piece of correspondence to be referenced, this is arguably not essential in a discussion group, where a combination of opinion and experience as well as references to the literature may be of use.

It is difficult to know whether any of the discussion that has taken place on this list has influenced patient management and outcome in any discernible way. There has been little in the way of feedback so far to the list. It may, of course, be too early to expect this, as outcomes in orthopaedics can take months or years to know. Information about membership of the list is freely available on the Orthopod member’s web-site,¹⁰ as are the archives.¹⁴ This publicly available information has not caused any discernible problems to the list members.

Approximately half of the discussions on the Orthopod list concentrated on areas of the list in which quality could be readily, if imperfectly, gauged. The other half of messages covered a wide variety of topics, but in the main were concerned with orthopaedic meetings, Internet related questions and, sometimes, merely conversation between colleagues. This side of an Electronic mail list is hard to quantify in terms of quality. It would be difficult to argue that it contributed significantly to the fund of human knowledge but, equally, a forum that allows professional colleagues to communicate across international barriers can be argued to be uniquely positive.

Since this study has been completed, five further orthopaedic sub-speciality Electronic mail lists have been created through Mailbase^19-23 and in this it has followed the profession and the paper journals. The biggest barriers, however, to this type of forum taking an important place in the orthopaedic world will be the number and the seniority of surgeons who use it, as well as the number of orthopaedic institutions that embrace the technology. Despite the rapid growth in the size of the list, it contains only a tiny fraction of orthopaedic surgeons in the developed world. It is notable that the list had input from surgeons in India, where advice on orthopaedic matters is less readily available. As a result of Orthopod, the Indian orthopaedic surgeons have developed a mailing list, IndiaOrth,²⁴ based on the principles of Orthopod. From the messages on the Orthopod list, two further major website developments occurred; firstly, the Orthogate⁷ project, which aims to be a virtual Internet gateway for all trauma and orthopaedic information was initiated. Secondly, the Internet Society of Orthopaedic Surgery and Traumatology²⁵ was formed. Without Orthopod neither of these developments would have occurred.

CONCLUSION

This paper suggests that quantity of information on the Internet need not be a substitute for quality in orthopaedic and trauma surgery, and that the electronic mail discussion list is another medium for the circulation of good quality information.

ACKNOWLEDGEMENTS

Thanks to Professor Angus Wallace for help and advice preparing this paper, and to Dr Myles Clough, Eve Hollis, and Dr Randale Sechrest for their advice and for co-running the Orthopod list.

REFERENCES

1. Heathfield H, Pitty D, Hanka R. Evaluating information technology in health care: barriers and challenges. BMJ 1998: 1959-1960
2. Blumental D. The future of quality measurement and management in a transforming health care system. JAMA 1997; 278: 1622-1625
3. Stilberg WM, Lundberg GD, Musacchio RA. Assessing, Controlling, and Assuring the Quality of Medical Information on the Internet. JAMA 1997; Vol 277(15): 1244-1245
4. Wyatt JC. Commentary: Measuring quality and impact of the World Wide Web. BMJ 1997; 314: 1879-1881
5. Culver JD, Gerr F, Frumkin H. Medical Information on the Internet. J Gen Intern Med 1997; 12: 466-470
6. Jadad AR, Gagliardi A. Rating Health Information on the Internet. Navigating to Knowledge or to Babel? JAMA 1998; 279(8): 611-615
7. Orthogate web site http://www.orthogate.com
8. Orthopod list http://www.mailbase.ac.uk/lists/Orthopod
9. Mailbase web site http://www.mailbase.ac.uk
10. Orthopod owner’s pages web site http://www.orthopaedic.ed.ac.uk/Orthopod.htm
11. Joint Information Systems Committee http://www.jisc.ac.uk
12. Mailbase Annual Report 1997-1998 http://www.mailbase.ac.uk/docs/report-1998.html
13. Organisation for Networked Medical Information http://www.omni.ac.uk
14. Orthopod Archive http://www.mailbase.ac.uk/lists/Orthopod/archive.html
15. Registered Members of Orthopod List sign up form http://www.orthopaedic.ed.ac.uk/Members%20List.htm
16. Mailbase Guidelines http://www.mailbase.ac.uk/docs/guidelines.html
17. Health on the Net Foundation Code of Conduct for Medical and Health Web Sites web site Honcode
18. OMNI Advisory Group for Evaluation Criteria web site http://www.omni.ac.uk/agec/agec.html
19. Hand http://www.mailbase.ac.uk/lists/hand
20. Arthroplasty http://www.mailbase.ac.uk/lists/arthroplasty
21. Limbrecon http://www.mailbase.ac.uk/lists/limbrecon
22. Sportmedorth http://www.mailbase.ac.uk/lists/sportmedorth
23. Spine http://www.mailbase.ac.uk/lists/spine
24. IndiaOrth http://www.indiaorth.org/orth/index.htm
25. Internet Society of Orthopaedic Surgery and Traumatology http://www.isost.org

Date of copyright: 16th October 1998

Correspondence: Mr Christopher Oliver, Edinburgh Orthopaedic Trauma Unit, Royal Infirmary of Edinburgh, Lauriston Place, Edinburgh EH3 9YW (Email [email protected])

©1998 The Royal College of Surgeons of Edinburgh, J.R.Coll.Surg.Edinb., 44, February, 36-39