Reach the Right Editor and Pass Peer Review
By Nancy Hays, EDUCAUSE Review, EDUCAUSE
Do you have research results you want to publish in a peer-reviewed journal? If yes, you have some thinking to do about the audience you want to reach and your reason for choosing peer review. Perhaps your field has key journals that carry the most weight during tenure or personnel reviews, and you want the credit of publishing in one of them. Or you need peer review for those reasons and don’t care as much which journal publishes your work as long as you can get it done before a certain deadline.
What if you don’t need peer review to accomplish your goals? Would a magazine in your field give your work the visibility you want among the community you have targeted?
As an editor in technical fields for 35-plus years — 15 at the IEEE Computer Society and 17 at EDUCAUSE most recently — I have worked with multiple peer-review boards and committees and the authors who submit work to them, and I’ve also collaborated with authors to tell their stories, research-based or not, from an initial idea to a published article. I’ve divided my recommendations and experience into those two major sections for this article for those who know exactly which approach they want to follow. If you don’t know, reading both sections will help you decide.
A caveat before you begin: I am an editor who loves working with volunteers, whether authors or reviewers, and I am paid to do this job. My experience resembles few academic journal editors or reviewers, many of them volunteers, for whom these tasks fall on top of sometimes horrendously busy and stressful schedules. The editor you encounter might not react to queries and submissions as I have over my career, with enthusiasm and gratitude that you contacted *MY* publication. Even an unhappy editor will want you to submit your work to them, however — they don’t have a publication without you and your colleagues submitting papers to them.
Part 1: The Peer-Review Process
First off, if you want an abbreviated guide, watch the video. If you’d like more detail, keep reading. You can use the video as a recap.
Second, in case you think the EDUCAUSE reviewers and I have taken an easy hand over the years, the acceptance rate has held steady at 25–30 percent even as the review committee changed members regularly. Reviewers devoted time and effort to provide feedback that focused on the submissions’ strengths, pointed out the weaknesses, and suggested ways to improve the work. They also rated each submission based on its quality and its value to the audience. Some excellent papers were turned away because they fit a different audience much better. I kept a list of places to publish, and you should feel free to ask for alternatives if your submission is turned down. Another editor might welcome your work as a boon to the readers.
Literature Review
Too early to bring this up? Nope. A thorough literature review can help guide and structure your research plan, so you’ll want to conduct it early in the process. Do include all relevant information for each source you note, as you’ll need it later. See “References” for the rest of the advice on sources.
Your Research
Yes, you should design your research carefully if you want to publish one or more papers on it. I’ve seen the same problems repeatedly, and when they occur in the research phase, the author has no way to recover and proceed with another draft of the paper — the foundation is flawed.
- Unless your research is qualitative, make sure you have a decent sample size. You can’t reasonably draw conclusions based on a small sample. If that’s all you have, point out the problems introduced by the small numbers and use care in drawing conclusions. You can also conclude with the “need for more research” that indicates you think the work important and worth pursuing.
- If at all possible, include a control group and a test group. If this isn’t possible, can you compare your test group to a “before” group of appropriate similarity? You might need to do some digging to find an appropriate “before” group, as demographics count.
- Determine basic demographics at a minimum, especially age. This will be especially important if you want to compare online (usually adult) learners to face-to-face students (more often “typical” college-age learners). You should also acknowledge the possible influences of different characteristics on the research results. If you don’t, the reviewers will.
- Collect more than basic data from the research. In other words, don’t limit yourself. If you intend to go through a rigorous research process, collect as wide a variety of data as you reasonably can. Not only will this give you multiple opportunities to publish, it will probably give you the additional information you might need to address a reviewer’s comments.
- Ask one or more colleagues to critique your research plan. They might suggest a survey question or type of data you hadn’t considered, and they might just have a control or “before” group to support your analysis.
Planning before you begin not only creates a higher quality research project, it helps you evaluate the different types of publications you might want to pull from the results.
Analysis and Collaboration
You know your field best when determining the types of analysis to conduct on your cleaned up data. Consider taking varied looks at the data, as it might answer questions you hadn’t originally asked. As long as you’ve collected a good body of information, you’ll have the flexibility to explore the data in different ways, perhaps even across fields. Say you want to evaluate student performance with and without video lectures, either online or face-to-face. A colleague from the media production department might be interested in measuring student reactions to length of the video clips offered, production quality, views of a person vs. content, etc. If those factors don’t make a difference to students or their performance in a class, the videographer and lecturer might change their approach to lecture capture. And, you or maybe the videographer have another story to share, from the same data collected to measure student performance.
Again, if you follow some basic recommendations, your analysis will support your results in a way reviewers will find persuasive.
- Target the type of analysis to your preferred journal outlet. If your first choice of journal in which to publish regularly includes papers with rigorous statistical analysis of data, make sure your paper does as well, and use the same approaches. If your preferred journal has a more readable style, present your analysis in terms that an educated but not expert reader can understand. The details might appear in an appendix or be offered to interested readers, or you could post them on your university’s website and provide a link to them in the paper.
- If you get indeterminate results, ask a different question. A controversial example involves a peer-reviewed article by Michal Kosinski with Yulin Wang published in The Economist about an AI program that identifies people’s sexual orientation based on images of their faces, with statistically significant results. The original research was on facial image recognition; the neural network program answered a question the researchers hadn’t asked.
- Analyze the comments for trends. You did obtain feedback from your test group, right? You can use their comments individually during presentation of your results and conclusions about the results, but you might learn additional interesting information about the students by looking for themes. The personal comments not only enrich your presentation, they can also suggest new research or new conclusions about the data collected.
Results and Discussion
I can’t stress this enough: Avoid making assertions or stating conclusions your data and analysis do not support.
- Use charts and graphs to illustrate the data. Make axes labels clear and informative, keep text to a readable level, and use colors that still look different if printed in black and white. You can explain in the surrounding paragraphs — you don’t need to include everything in one graphic.
- Keep tables and graphics concise and readable. Reviewers don’t appreciate trying to understand a lot of information crammed into a too-small space. Break the data into multiple tables and graphics if you have a lot to report, then create a “summary” to show the overall result if that’s appropriate.
- Consider summarizing key results in a bullet list or box. A handy design element for important information highlights your key results, whether you choose a bullet list or a “sidebar” box set off from the surrounding text. You want to make the reviewers’ job as easy as possible, and this type of summary provides an easy reference.
- Organize your discussion around the key results. Many authors get carried away writing paragraph after paragraph about their results and ruminations, making it hard for reviewers to evaluate the actual support for any conclusions drawn. If you keep your discussion and thoughts about each key point together, with an identifying subhead or bullet, reviewers will find their evaluation easier and probably faster. Both are good — you want reviewers not just to understand the work but also to approve of the way you’ve presented it.
- Indicate the degree of confidence for each result. Even non-expert readers expect to see the confidence level included with statistical analysis of results. For example, if your result has a large confidence interval (25 points, say, for a small number), you’re admitting it can’t support a statistically valid conclusion. If a reviewer discovers that weakness when you didn’t report it, expect the reviewer to doubt all of your analyses. Be transparent, and speculate on why that analysis produced a suspect result. You might tempt reviewers (and readers) to chime in with their own interpretations.
- Temper your conclusions to match your analysis. If the data support a statement about the significance of your results, make it. And remind the reviewer of the statistical validity of the analysis behind the statement. If the data don’t support a conclusion you had expected, point that out and theorize why the mismatch occurred. In case of a rather weak result, use the necessary caveats to explain why the conclusion lacks strong statistical validity but might indicate a relationship — which requires more research.
- The results and discussion sections benefit from student quotes. Here you can intersperse appropriate student feedback to illustrate or support specific results. For example, you might include a neutral/negative comment and a positive comment about the inclusion of video lectures in the learning management system. Resist the temptation to use only positive comments, as reviewers won’t believe that none of the students had negative opinions or that all of them watched and benefited from the videos. They understand student behavior better than that.
- Include unexpected results. Say the students only watched videos under five minutes long and expressed indifference to video quality — topics not part of your initial research question — which you discovered through measuring their access of the videos in the LMS or through evaluation of the comments. You want to share this information to inform producers of video lectures what students do and don’t care about. And perhaps to suggest further research on this question by interested parties.
Conclusion
First of all, avoid using the word “conclusion” if possible. It’s boring. And you really don’t want to simply summarize or repeat what you’ve already covered, surely. I guarantee the reviewers won’t want to read it (unless, of course, that’s the journal model, in which case — follow it).
Think about the final message you want to leave with readers. The key point(s) of your research results? An appeal for others to research the same questions and advance knowledge of the topic? A request for collaborators on other campuses? Only one concept can dominate the final statement of your paper. Decide which one is most important to you, and use it as a brief lead into the final paragraphs and as a strong concluding statement. In between, cover your other messages in a way that supports the main one framing them.
Using the video lecture example, let’s say you had a fairly small sample size (26 students in a single class of 30 who replied to the survey) and a “before” control group of 29 students (same lecturer, same course and content) for whom you have performance only (grades in the course). Given the research question — Does video lecture capture made available to students improve their performance in a class compared to no use of video lectures? — the high percentage of respondents is less important than the low total number. Given these weaknesses in the sample size and the likelihood of no or minor differences in grades between the two groups, your strongest conclusion would either suggest more research or invite collaborators to pursue further research with you.
- Do stress your key findings somewhere near the end of the paper, either at the end of the discussion section or within the concluding section. After all, that’s what your research focused on. If the results are weak, say so (an argument for more research). If the results showed statistically valid improvements in student performance with the availability and use of video lectures, hurray! Your small sample size argues for more research.
- Mention your future research plans (if any). Especially if you plan further research into the topic explored in your paper, include that near the end of your concluding section, either before asking for collaborators or after urging others to research the question on their campuses. Your commitment to the research and belief that it is worth pursuing make a fine argument for its value to the community. Use it.
- Aim for a strong, active final statement. Generally, the concluding sentence(s) should make an assertion (“this topic merits further research”) or call for action (“please contact me to collaborate on a larger study”). Authors who simply repeat their results or comment on the interesting nature of their findings lose an opportunity for a strong close. Don’t let your paper fizzle out in a tangle of weak sentences — challenge the reader (politely, of course).
References
Despite their ancillary, although important, role in an academic paper, references can rouse scorn among reviewers, irritate editors, and drive the author to despair. Strong words? Let me explain.
- Reviewers look for thorough, directly relevant, and current sources. Relying on the Wikipedia entry won’t do. Reviewers want to see the seminal and most current research behind statements you make in your paper. If you fail to include an important work (especially one authored by the reviewer or reviewer’s colleague), it will be pointed out to you. Equally important, you should include the most current work in the area rather than having all publication dates five or more years earlier. And don’t waste your time or the reviewers’ by including sources for generally known information or listing 10 sources for a single point. Use your judgment — and see what the already published papers in your target journal do with references.
- Editors look for complete source information. Although the editor will also look at your references and wince at a lack of current or seminal work cited, the editor wants to see complete information for each source as well. The format is less important than providing each author’s full name, for example; a name can be reduced to a letter according to the reference style for the journal, but guessing a name from a single letter is impossible. Make life easier for yourself (and the copy editor if the journal accepts your paper) — note complete information for each source during your literature review. Otherwise, you might have to go back and look up the missing information later.
- Formatting depends on journal style. The author guidelines should tell you the preferred reference style to use. As long as you include complete information for each source, you can always format appropriately if the paper is accepted, and it won’t hurt to mention that to the editor. Many authors use an automatic reference program, and editors understand that you won’t want to change that until necessary.
- For online publication, provide hyperlinks. Many journals now publish online to save printing and mailing costs, and that means you can link directly to a relevant online source for most of the references you cite. For books, link directly to the publisher’s or author’s website, not a retail outlet. Databases like the government-supported ERIC can also be helpful when the work is behind a paywall. The ERIC listing includes the title, authors, publication name, publication date and volume/issue numbers, publisher, and ISBN.
You can see that with 20 or more references in your paper, this section can cause headaches for everyone involved in the process — beginning with you. Do it right the first time and save yourself trouble later.
Part 2: Publishing without Peer Review
Suppose you don’t want or need peer review. Or your target publication and audience don’t offer peer review. Or, depressingly, your paper failed peer review and you still want to publish in your target publication. What now?
Remember that peer review matters to a relatively small audience of academics and researchers. Many or even most readers just want to learn about new projects and results. Consider your goals and target audience in deciding your approach.
Peer Review Not Needed
OK, you’ve decided that you don’t need peer review for various reasons and want mainly to reach a particular audience. Pull together a list of potential publications and put them in priority order based on:
- The audience you want to reach
- Acceptance of outside submissions
- Fit of their published articles with your planned article
- Whether they pay for published work (if it matters)
Now, read the author guidelines for your top journals or magazines and evaluate whether you can or want to create an article meeting their requirements. You might find that some of your choices don’t look like a good fit for your work after you understand what they ask of authors. For example, you want to write an article supported solidly by your research data and analysis, and your top choice discourages inclusion of details — they want a conversational story with clear advice for readers, and most of their authors are paid freelance journalists, not academicians. Maybe you’d better rethink submitting to this publication — unless you are willing to write your story at a more general level.
After determining which publication best fits alls your goals (readership, level of detail, fit with your work), consult the author guidelines again to determine how best to contact the magazine:
- Submit a proposal to the editor or guest editor for a specific theme for feedback
- Send an already written article to the email indicated in the author guidelines or a call for papers announcement
- Telephone or email the editorial staff to ask for advice on submissions
- Ask a colleague already published by the magazine to “introduce” you via email to the editor or guest editor for a specific theme
- Arrange to meet the editor, perhaps at a conference in your field, or stop by for any “office hours” the editor offers while attending a conference
I’ve used all these methods with authors. EDUCAUSE has an editorial team email for just such contacts: editors@eduause.edu. Generally, the editor would prefer to interact with you before you submit an article, either responding to a proposal, talking with you by phone or in person, or even answering a question about the publication’s interest in a specific topic. If you speak to an editor at a conference,
- Introduce yourself and your title and institution; don’t provide a full biography
- Mention your topic of interest, and go into more detail if the editor shows interest
- Ask if the magazine would like to see an article or proposal/outline on that topic
- Give the editor your business card with complete contact information
- Make a note to yourself about the encounter, preferably on the editor’s business card
- Don’t monopolize the editor’s time at a busy conference — get your message across and move on
- Email (preferred) or call the editor in the week or two following the conference to reiterate your interest and ask for guidance
If you are a presenter at a conference and an editor approaches you to write an article, be realistic about your ability to produce something on the requested schedule. If you agree to the request, call or email the editor after the conference to reiterate your interest and verify the schedule; this is also your chance to ask questions about length and approach of your article. Don’t ask the editor for examples — find them yourself on the publication’s website. And pay attention to what the editor asks of you rather than modifying the topic without checking. If you send an article the editor considers off the requested topic, you might find yourself revising or even declined. Make sure you and the editor agree on what you will deliver and when in order to avoid conflict later. Being solicited to provide content does not guarantee that whatever you send will be accepted.
OK, you’ve got a “go” from the editor. Now what?
- Read the author guidelines again and skim already published articles
- Pull together an informative proposal, outline, or draft and ask for editorial feedback; this will save effort and possibly disappointment later
- Use that feedback in writing your article, and keep an eye on the deadline
- If work duties or family emergencies delay you, let the editor know and negotiate a new schedule or even drop the project
- If questions come up, contact the editor and get answers before you continue
- Send the promised article by the deadline or tell the editor you expect a short delay
- If the publication is online, include hyperlinks, audio/video, tutorials, screen captures, and other media that print can’t replicate to take advantage of the online medium; the editor will thank you, and if you don’t, the editor will probably ask for those things anyway
- About graphics — make sure you provide clear, easily understandable, and relevant graphics to illustrate points in your article, or explain what you plan for graphics and ask for feedback before investing the effort into producing high-quality images
Your article has been accepted, assigned to an editor, and scheduled for publication. Meet your commitments! If you ignore the editor’s messages and requests, miss deadlines, and provide inadequate responses to editorial queries, you will definitely find yourself on a black list and might even find your article cancelled. Assuming you plan to continue writing about your work, developing a mutually respectful and cordial relationship with the editorial team of your favorite publication will obviously serve you better than lackadaisical interactions. And, if you become a preferred author for that title, your opportunities for future publications is excellent. I know I have a group of repeat authors who will always get a quick and positive response from me, and many of those folks also sit high on the list of experts I consult for articles on specific topics.
Your Article Failed Peer Review
If your article failed peer review at your field’s most important journal, you are entitled to a bout of misery and self-pity — it’s an awful feeling. Don’t give up! You still have options.
- First, you can submit the same or a revised (to fit the author guidelines!) paper to another journal for their peer review.
- Second, you could ask the editor if you may revise your paper to meet the reviewer objections and suggestions and re-submit to the same journal.
- Third, consider revising your paper to address the reviewers’ comments and submit to another journal, especially if your first choice indicated a poor fit between your work and their goals.
- Fourth, ask the editor if a subset of your paper might make an acceptable article, not part of the peer-reviewed content, for publication in that journal.
When articles submitted to EDUCAUSE Quarterly and later EDUCAUSE Review failed to pass peer review and I really liked the topic, I used one or more of these options in responding to authors. Sometimes a major revision will solve the problems the reviewers raised — but only if the author’s research collected all the information needed or the author can consult a statistician to help with the analysis. Sometimes, adding a co-author who can “fill in the gaps” with their own work and insights will solve the problems. Often, though, the authors don’t have time for or interest in making major revisions, and then I will share a list of other places to publish. If one or more alternative publications might fit their work, I’ll point those out. If the research is weak for peer review but interesting for our audience, I’ll ask the author to write a shorter case study or good advice article based on their work. They can always publish the full research article elsewhere while reaching the EDUCAUSE community with a more accessible presentation of what they learned and can recommend. I consider this a truly positive result for both the editor and the author, and ultimately for readers.
What’s Collaboration Got to Do with It?
I’ve tried to keep my advice on publishing, with or without peer review, targeted. Now I want to stress a social skill that can benefit your research and publishing opportunities: collaboration.
Don’t think of this as a recommendation to always include others in your research or as co-authors, as that’s not what I mean. Consider the following partnerships and how they might benefit you:
- In your research: Using the video lecture example, you’ll need the help of the lecturer and the students at a minimum. If the instructor refuses to have lectures recorded and posted in the LMS, you don’t have a research project. If students don’t respond to your survey, you can’t collect the data you need. You might require approval from the Institutional Research Board to conduct research on students, and experts in creating surveys could increase your response rate. Secondarily, a media producer can ensure consistent video quality to avoid introducing a variable you don’t want to address in this particular project. Finally, and importantly, colleagues can offer feedback on your research plan, survey questions, analysis, and conclusions. Their contributions can help you achieve the highest quality possible for your paper and avoid embarrassing mistakes.
- During analysis: Do you consider yourself completely qualified to analyze your data, unaided? Or would you like help from a statistician during this stage? What about cleaning the data before you begin — would somebody else do a better job or perhaps an equally good job while saving you time? A graduate student, for example, could clean the data and look for potential themes in the comments. Just having an experienced colleague checking your work could avoid mistakes that might weaken your results.
- Discussion hazards: This step raises the greatest challenges for the researcher. Most importantly, does your analysis of the data support whatever conclusions you draw? If you check your conclusions with expert colleagues, they can tell you where you’ve gone astray or where you should add a caveat or disclaimer. Enthusiasm about your results can blind you to flaws that reviewers and editors will spot, and your colleagues can provide the final content polish your paper needs.
- Conclusions: See “Discussion hazards.” Also, ask one or more colleagues to read your paper and tell you if your concluding section is too long, repetitive, or could offer a stronger end to your paper if reworked. Ask them what message they get from the final paragraphs, and revise if it doesn’t match what you intended.
- From a publishing standpoint: To obtain the most favorable evaluation of your submission, take advantage of editorial services offered by your department or institution. Get feedback on your writing and grammar, clarity of language, organization of the material, use of graphics and other media, and focus on key points. The closer your submission gets to the journal’s author guidelines, the better. And if the editor and reviewers find it clear, readable, and to the point, they won’t be cursing you for making them work hard to understand your research and conclusions; they will appreciate the effort you put into making the presentation professional.
This advice applies across your professional life: collaboration can help you do better work and produce better results, and the relationships established will continue to support you and your colleagues as your career matures. Others call this developing a network, while I consider it developing mutually beneficial partnerships that can pay off in ways you can’t anticipate — including in publishing top-notch work in peer-reviewed journals and wider-interest magazines.
About the Author
Nancy Hays retired at the end of 2017 as Editor and Manager of Publishing at EDUCAUSE, which she joined in 2000. From 1985, she served in editorial roles at the IEEE Computer Society, leaving when she was a group managing editor of five technical magazines, including IEEE Computer Graphics and Applications and IEEE Multimedia. Between graduation with an MA in English from UCLA in 1980 and 1985, she held editorial roles in medical publishing, software evaluations, and aerospace. She earned three BAs from Oregon State University in 1977, in English, Economics, and Liberal Studies. Her email is nhays@earthlink.net.
She recently published "My Editorial Life: Immersed in Higher Education IT" in EDUCAUSE Review.
| Previous page on path | Cover, page 3 of 26 | Next page on path |
Discussion of "Reach the Right Editor and Pass Peer Review"
Add your voice to this discussion.
Checking your signed in status ...