4th Int'l AAAI Conference on Weblogs and Social Media
May 23-26, 2010, George Washington University, Washington, DC
ICWSM 2010 Blog
Friday, March 5, 2010
Response to Ed Chi about the review process
In a comment to a previous post Ed Chi asked:
Can you give some transparency to how decisions are made to reduce a full paper to a poster / short paper?
In answer - the review process this year was somewhat different from previous years:
- For each paper, one SPC-member meta-reviewer and three PC-member reviewers were assigned, primarily based on bidding, with some manual balancing of the reviewers by the PC chairs (in part based on requests from the SPC).
- The PC members submitted reviews independently (i.e., without seeing the other reviews)
- The SPC member then led a discussion of the paper, in the case of any substantive disagreement, and wrote a fourth review summarizing the results. Sometimes extra reviews were asked for at this point. The discussion is not available to the authors, although the SPC members were encouraged to summarize it in the meta-review.
At this point, if everyone's done their job - there's a lot of information about each paper, but all the decisions were based on a very local view of the pool of submissions (as each reviewer sees only 8-10 papers), and reviewers tend to calibrate their scores differently in different sub-areas. In the final stage, a small group - Sam and I, with some help from Marti and previous ICWSM organizers - looked over all the papers, reading the reviews for many of them, the meta-reviews for all of them, and sometimes the papers themselves - to give a final ranking. Aside from the raw scores for the paper, we considered the overall topical balance of the conference, and the perceived breadth of interest in a paper, but these were secondary considerations. This discussion is also not provided to the authors.
BTW, my comment about this year having a strong pool of submissions wasn't polite conversation. There were a half-dozen papers with an average score of 8.5 or above, 16 papers with an average score of 8 or above, and 30 with an average score of 7 or above. After looking the papers over one of our first steps was to figure out what was the maximal amount of papers we could present as plenaries (25), given the time constraints of the conference, and we spent a lot of time discussing the papers that were close to the cut-off. We also considered but rejected the possibility of introducing parallel sessions this year.
William Cohen
co-PC chair
# posted by ICWSM @ 7:37 AM 
Comments:
«Oldest ‹Older 1001 – 1020 of 1020
<< Home
Everything you need to know about Golden Doodles! From care tips to fun facts, we've got all the insights to keep your furry friend happy and healthy https://goldendoodleinfo.com/black-goldendoodle/
Everything you need to know about Black Goldendoodle! From care tips to fun facts, we've got all the insights to keep your furry friend happy and healthy
Film Jackets are iconic pieces that blend fashion with character identity, leaving a lasting impression on viewers. From edgy leather to bold statement pieces, they define cinematic style and pop culture.
Great tool! The SIP Return Calculator really helps in understanding how small, regular investments can grow over time. It is especially useful for beginners who want to get a clear picture of the power of compounding in mutual fund investments. Would love to see more features like comparison between different fund categories or tax impact on returns. Keep up the good work!
Whether you're a beginner in MATLAB or tackling advanced simulations, it’s normal to encounter roadblocks. The syntax, toolboxes, and computational functions can be confusing without proper guidance. That’s where expert MATLAB help becomes invaluable. These services are designed to support students through their toughest assignments and most complex projects. When you get help from professionals, you gain access to clean, well-documented code and easy-to-follow explanations that align with your academic needs. This kind of personalized assistance allows you to better understand your assignments and perform more confidently in class. You’ll also save valuable time, avoid unnecessary errors, and even discover best practices you can use in future projects. With the right support, MATLAB doesn’t have to feel intimidating—it becomes a tool for success.
Just explored White Centre Fabrics latest collection — and wow, what a treat! 💫 So many gorgeous prints and premium-quality fabrics to choose from. Perfect for summer stitching and festive wear! Can’t wait to shop again. 🌸🧵 #WCFabrics #FreshFabricFeels #SummerReady
This really is a nice post that you've shared with us!!
whirlpool dishwashers parts
appliance parts
amana washer parts
whirlpool dishwashers parts
appliance parts
amana washer parts
Really appreciate the honest explanation it is refreshing. Kind of like designing children furniture when it is done with care and thought it truly makes a difference.
Thanks for sharing this insight. Just like in thoughtful reviews luxury pret also thrives on attention to detail and both are truly an art.
This was an interesting read about the review process and how feedback is handled. Clear and fair processes matter in every field. Just like in fashion, where ready to wear dresses make it easier to find quick and reliable choices.
Interesting perspective on the review process. Just like academic systems vary traditional clothing like kameez shalwar also reflects cultural uniqueness in its own way.
This was an insightful read about the review process. While my main focus is on writing about modest fashion and abayas I appreciate transparent discussions like this that help improve how content and ideas are evaluated.
Interesting insights on the review process transparency truly matters. Just like clear feedback helps improve work well-designed shirts for women bring clarity to everyday style.
Really insightful post appreciate the openness around the review process. On a creative note I have been exploring a new ready to wear collection that highlights how structure and detail matter just as much in fashion as they do in research.
An insightful discussion on the review process transparency like this really adds value to the academic community. On a lighter note I recently came across some beautiful unstitched lawn collections that are perfect for summer styling.
Really insightful discussion transparency in the review process is so important for building trust in academic communities. I happened to be reading this over my morning coffee and an almond croissant and it made me appreciate how meaningful these conversations are. Thanks for sharing your perspective.
Appreciate the transparency in how the review process is handled it is always insightful to learn how things work behind the scenes. On a different note I have been working on a creative project focused on unstitched lawn fabrics and the freedom they offer in personal styling.
«Oldest ‹Older 1001 – 1020 of 1020
Subscribe to Post Comments [Atom]
<< Home
Sponsored by the Association for the Advancement of Artificial Intelligence. For more info: icwsm10@aaai.org
Post a Comment