Advice to Students on Mastering Communication Skills

I found that some graduate students and undergraduate students (even native English speakers) often have difficulties in their communication with me in describing what they intend to convey. After some recent observations and thinking, I found that these students are lacking in their communication skills especially in terms of the logical organization of their thoughts. In particular, the issues lie in two parts:

(1) bottom-up issue: failing to describe things in a top-down way.

(2) context-less issue: failing to connect the background/context of the listener (me) with what they intend to convey.

Let me illustrate these issues via two concrete scenarios that I extracted from my real interactions with students.

*. A student tried to describe to me a problem encountered in his/her research project.

The student immediately dived into the details of a particular experimental subject in his/her evaluation and immediately got me lost. I had no ideas what the student was talking about for a while when the student tried hard to describe so many low-level details to me. I had to stop the student and asked "can you summarize the problem in one or two sentences that I can understand with the granularity level of my existing understanding of your project so far?" The student couldn't come up with such sentences. I further asked "why do I care about the problem in terms of the success of the research project?" The student still failed to answer this further question.

After some more non-trivial interactions with the student, I finally figured out that the problem that the student intended to convey can cause not-good values for a particular metric (among multiple metrics) for evaluating the proposed approach.

The bottom-up issue here is that the student failed to organize the conversation in a top-down way: I would expect the student to start the conversation with me like below:

"I would like to discuss with you on an encountered problem that can negatively affects the effectiveness of our approach, particularly in terms of the MMM metric. The problem is caused by XXXXXX (high-level description of the problem). For example, YYYYYY. ZZZZZZZ. ...."

Instead, earlier the student started the conversation on "YYYYY.. ZZZZZ, ...." or even "ZZZZZZ, ... YYYYY...". At that time, I had no clue or got lost on why the described phenomenon is a problem and why I would care (if I indeed care about the effectiveness of the approach).

The context-less issue here is that the student failed to connect the background and context that I have on the project (i.e., I know what the high-level idea of the approach is and what the evaluation criteria for evaluating the approach are) with what the students tried to convey (i.e., the problem). To make sure that I have the expected knowledge or context to understand the student's problem, besides the expected top-down way described earlier, the student can better start the conversation like "Let me first remind you that the effectiveness of our approach is measured based on n metrics, among which the MMM metric ..... However, I encountered a problem that can cause bad values for the MMM metric. The problem is due to XXXXXX..."

* A student tried to answer my question in a qualifier exam or thesis defense.

Let's assume that the question is "what metrics do you use in evaluating your approach?" The student would start the answer with "ZZZZZZZ, .... YYYYY....". For quite some seconds and minutes, I had no clue or got lost on why what the student spoke had anything to do with my question. Or even worst, in some situations, after the student finished minutes of talking, I still had no clues on what the answer is (for those questions expecting a "Yes" or "No" answer, I still couldn't figure out whether the student intended to say "Yes" or "No"!).

The bottom-up issue here is that the student failed to organize the conversation in a top-down way: I would expect the student to start the answer to my question like "I used two metrics. The first metric is X/Y where X is ... and Y is ... In particular, I measure X with faults seeded with mutation testing, .... The second metric is... " Instead, earlier the student started the talking w.r.t. my question with "Mutation testing is a commonly used way for measuring fault detection capability, ... YYYYY.. ZZZZZ, ....". For a while since the student's talking started, I had no clue or got lost on why mutation testing has anything to do with the metrics. Even when there are occassions where some background information needs to be laid out before giving out the answer, the student still should give strong signals as starting sentences for making clear how what the students will say first is related to the answer. Some examples can be "Before I describe the metrics that I use, I would like to first describe mutation testing, which is related to three out of four metrics that I use, and then after that I will list these four metrics....."

For those questions expecting a "Yes" or "No" answer, the student can start with "My answer is Yes, with three reasons. First, ...." or "My answer is both Yes and No. The reason for me to say Yes is .... The reason for me to say No is ..."

The context-less issue here is that the student failed to connect the background and context that I have (i.e., my question) with what the students tried to convey (i.e., what the student talked about w.r.t. the answer).

* Final remark

I would suggest students to realize these communication anti-patterns and address them. Students who have these anti-patterns in oral communications often have problems in their writing (especially in terms of logical thinking and writing) such as not laying out enough background or assumptions for readers to understand what the students write next, or not clearly and concisely organize the content to make the writing easy to understand.

The Minto Pyramid Principles (e.g., top-down writing style) are good ones to follow as starting points for students. Below are some slides on some key ideas there:
http://www.dbai.tuwien.ac.at/staff/gatter/work/051104_The_Minto_Pyramid_Principle.pdf
More are described in the Minto text book:
http://www.barbaraminto.com/textbook.html

Update: my colleague Macneil Shonle recommends the following book: The Sense of Structure: Writing from the Reader's Perspective, which may worth checking out. Thank Macneil for the recommendation!

This post more or less follows the top-down writing style, if you realize.:)

Graduate Student Survival/Success Guide

Yesterday, I gave a CSC 600 lecture to new graduate students at NCSU CSC. The slides can be found here.

How not to keep your advisor up all night till last minute before the paper submission deadline?

Posted on Friday Nov 07, 2008

I recently had been through a case where a student submitted the student's draft for my review of writing two days before a submission deadline. Since I had another more urgent task during the period, the required effort for helping improve the draft to a fine shape "forced" me to stay up the whole night till early morning 5am the submission deadline.

When I tried to recall several past submissions of this student, I found that all these submissions kept me up all night in the morning till last minute before the deadline whereas many other students' drafts were often already ready to submit the night preceding the submission deadline.

I asked myself "why would this situation happen for the student?" For this immediately past submission, in fact the student did a very good job in preparing the abstract, introduction, and approach (high level description) sections early on (for my review of ideas). But the problem is that the student submitted the draft for my review of writing late, only two days before the submission deadline. Several possible reasons may contribute to this late submission: the student might think that the student needed to finish all the sections of writing before asking peer review for writing, or the peer reviewing student needed to finish peer review of all sections of the draft before asking for my review of writing, ...

To avoid these issues (or avoid keeping me up all night till last minute before the paper submission deadline), below is a new set of advice to my students:

Please write early, ask peer review early, and ask for my review of writing early!! Here are some specific things you should do:

(1). You don't need to wait till you have a complete draft (with all sections finished) before requesting peer review. You can submit your partial draft (e.g., one or more completed sections at a time) for peer review.

(2). You don't need to wait till your peer-reviewing student finishes reviewing all the sections of your draft before requesting my review of writing. You can submit your peer-reviewed partial draft (e.g., one or more peer-reviewed sections at a time) for my review of writing. Remind you that I have a policy of being able to review your draft for writing ONLY after a peer student has finished reviewing the writing of your draft and you have fixed the issues pointed out by the peer student. But you don't need to get your peer review done before I can review your draft for the ideas described in the draft to avoid the delay on giving you feedback on the ideas in your draft.

(3). Budget your timeline to allow to ask me to review your writing for multiple iterations (instead of just one time or even 0 time iteration on any portion your writing before the submission).

Of course, it is most important for you to write early following my advice posted earlier!

On writing weekly lab book entries

Posted on Sunday Oct 19, 2008

Every week before the one-on-one meeting (if no regular one-on-one meeting arranged, then on a weekly base), a student should submit a lab book entry in our group wiki http://sites.google.com/a/ncsu.edu/ncsu-ase/Home/Labbooks for
*. Planned activities
*. Actual outcomes

For the "Actual outcome" category, you basically copy the "Planned activities" from the preceding week and then annotate them with the actual outcomes.

For each category, you need to organize your items in the following subcategories:

*. Tool development
Description of your task items
Expected artifacts: (here you put only specific tool components, with the details of the location in CVS, e.g., the genAxim method of /toolsrc/jias/jias/axioms/

Axiom.java)

*. Empirical evaluation
Description of your task items
Expected artifacts:(here you put only the writing portions for describing the evaluation or its results, with the details of the location in CVS, e.g., the evaluation section of /papers/icsm08-soa/)

*. Paper writing
Description of your task items
Expected artifacts:(here you put only the writing portions, with the details of the location in CVS, e.g., the approach section of /papers/icsm08-soa/)

*. Misc
Other task items not falling into the three preceding categories

Your task item's description shall be detailed enough so that I can distinguish it from a previous item in previous weeks. For example, you shouldn't put the same item description like "Preparing a Journal Version of XXXX" in multiple weeks. That is, from your description, I can tell the semantic difference of your new task item from any of your previous items in previous weeks.

Note that only recognizable artifacts are tool source code and formal writing in LaTeX being put in CVS. The artifact description shall describe enough details for me to trace down to the artifacts without further asking you. If you cannot put an artifact for a task item in one of the first three categories, you shall move the task item to the "Misc" category. For example, "Explore various tools such as XXX to use in the tool development" shouldn't be put under "Tool development" since there is no artifact (tool source code) being produced by this task item. This task item shall be put under "Misc"; just like reading research papers, you should always explore various tools along the way of your actual tool development.

For the "Actual outcomes", you copy the "Planned activities" over and annotate each item with some description of the completed portion. You also need to list "Actual artifacts" after the "Expected artifacts".

If you don't produce any portion of an expected artifact, you need to put "None produced" and color that item with the red color. If you produce only partial portions of the expected artifact, color that item with the orange color.

On reviewing a student's paper drafts

Posted on Saturday Oct 18, 2008

We will have three reviewing phases of your paper drafts:

Phase 1: Submit your draft to me for me to review the ideas (in the phase, I won't fix your writing but focus on your ideas, just like the way a PC member reviews your draft). After you receive my feedback, you revise your draft based on my feedback. Then you repeat Phase 1 till I explicitly tell you to do a peer review (moving on to Phase 2). When you send me a new version via iterations, you need to highlight the changed parts that need my attention.

Phase 2: Submit your draft to a peer student for the student to review both the writing and idea. After you receive the feedback from the student, you revise your draft based on the feedback. Move on to Phase 3.

Phase 3: Submit your draft to me for me to review both the writing and ideas. Note that I will review your writing only after you have gone through Phase 2.

This policy is also applicable to the intermediate writing during the research development besides the final writing before the submission deadline.

I want to point out another thing on intermediate writing during research development. Many students sit too long on a writing phase before actual tool development or evaluation. I expect the turnaround time for submitting your writing to me in a cycle of one week or two weeks. But some students take multiple weeks to prepare the writing to submit to me. Doing so loses the point of the original intention: getting in-time and early feedback on your work.

Here are my more explicit guideline on intermediate writing during the life cycle of research development:

(1) Prepare the abstract, introduction, example, and related work sections and submit them to me.

--> Start tool development

(2) Prepare the approach section and submit it to me

--> Continue tool development

(3) Refine the approach section and prepare the evaluation design section and submit them to me

--> Finish tool development
--> Conduct evaluation

(4) Prepare the evaluation results section and submit them to me

--> Now you shall have a relatively complete draft after you add the conclusion section and discussion section if needed.

Additional points:

*. In each later phase, you should refine and improve sections written during early phases.

*. Before you write down your ideas (on your approach or evaluation design, etc.) in your paper as formal writing, you need to discuss with me to get my OK stamp on the things that you are going to write down. I had been through some cases where a student spent a lot of time in writing down what the student thought to be a reasonable idea before discussing with me on the idea. After I read the writing, I rejected the idea with good reasons. Then the student basically had to discard the old writing (with a lot of time previously being wasted). That is, don't rely on only the formal writing as the only media for communicating with me with your ideas or work. You need to fully discuss with me with what you plan to write down in your formal writing.

Again, remember that you shouldn't sit on the writing for one of the above phases for too long.

Why shall we (research advisors) avoid directly writing on students' research papers?

Posted on Sunday Oct 12, 2008

When students started their research in my research group, I told them that "Don't expect me to do tool implementation for your projects", "Don't expect me to do experiments for your projects", and "Don't expect me to write sections of your papers for you". Indeed, I didn't say that "Don't expect me to give you research ideas to work on" since it is very tough for students to come up with good research ideas to work on when they first start their research. At the same time, I have tried different ways of giving students space and opportunities to think about their own research ideas. Such ways are taking good effect and many students are improving themselves in coming up with good research ideas. That is a separate topic here and deserves another post of discussion.

Why shall I (as a research advisor) avoid directly writing on students' research papers?

Two main reasons. First, if I write for my students, they never get a chance to learn how to write themselves. If they don't know how to write papers, how can they write their own dissertation? The advisor cannot write the dissertation for them! How can they write their research papers after they graduate? The advisor cannot accompany the students for all their professional career.

Second, a bit selfish reason. I don't have much time to write papers for students: I have proposals to write (so far few of my students can write proposals for me but I have been trying to train them to have the capability when they get close to graduation), I have professional services to serve, I have a good number of students to work with, advise, and train, ...

Although I told students that I didn't write for them, I spent much effort in training them how to write and how to improve their writing. Below are several things that I do:

*. I mark my revisions and suggestions on hardcopy of their writing submitted for my review and revision; I don't directly write on the LaTeX source files or Word files (whatever format is used in paper writing). Doing so can "force" students to at least recognize and understand what I marked and commented.

*. I walk through with the students in person on some major issues being marked and tell them the reasons for my revisions. These reasons may not be written down on the hardcopy as comments. Basically, I want them to learn the corrections not only for the particular mistakes that the students made but also for future similar types of mistakes that the students shall avoid making. At the same time, I ask the students whether they have any doubts or questions on my marks and I then address their doubts and questions if any. Smart students will always take opportunities to ask me the reasons for revision places that the students don't understand.

*. Before requesting my review or revision of a student's paper, the student should ask a peer student to review and revise the student's paper. Doing so not only allows other students to be trained with review skills but also reduce my revision load on picking up those low-level, easy-to-spot writing issues. When I finish my revisions on the hardcopy, I expect that the peer student who reviewed the paper also look at my revisions and asked themselves "why didn't I catch these issues?" Another major motivation for reducing my revision load is that when a paper includes too many low-level writing issues, I would be busy in correcting those low-level writing issues and forget about the logical flows and high level technical content issues. That is, the student, the author of the paper, will lose much of the benefit given by me in improving his or her paper. Including too many low-level writing issues in a student's paper before submitting it for my revision would just do harm to the student, not being able to take full advantage of my advice on their high level, more important, writing issues and research issues.

*. I will pay much attention to the abstract and introduction sections, whose writing is very important and students often don't do a good job. These parts often require much more iterations of my revisions than other parts of the paper. Students often don't do a good job in organizing the logical flow and the organization or reasoning of the motivations.

*. I will ask students to turn their common mistakes into some regular expression patterns, and then use a tool called style-check developed by Neil Spring so that the students can guard against some low-level issues themselves (if they cannot guard against these issues manually easily). Frankly, I found that several of my students still make many low-level mistakes repeatedly and they don't use style-check as I suggested. I hope these students can take full advantage of tools to help address the low-level issues of their writing, and exploit and focus my guidance on the high-level issues of their writing.

Next I discuss what I do to help students to write early and write enough so that I can train their writing.

Many of you may wonder: when deadlines are coming and students are lagging behind in their writing, we would have to jump in to make these deadlines by writing the papers for the students since we really don't want to miss these important deadlines. Indeed, I had been through these situations.

One possible way to deal with that is to really impose strict policies to students: if they don't prepare their papers early enough for us (advisors) to review or revise, let's just bypass the deadlines. In principle, bypassing an important deadline will do more harm to students than the advisor: the advisor has multiple students to work with and have multiple papers to submit, but the students individually may just have a limited number of papers to submit yearly or in their whole graduate program. Each year, there exist only several top conference paper deadlines and a small number of good or right conferences for the work. Missing some important deadlines produces very negative impact on students' research record building (indeed as well as the advisor's), given that often the time papers will be rejected by top conferences.

But the above way often doesn't work (either for the advisor or the student). The advisor doesn't want to miss important deadlines either. The students also want to make the deadlines but their deadline-making or engineering skills (c.f. my talk slides on research skills) are often not good. Often the time, the students postpone their paper writing to the last minute; they often don't like paper writing.

Then how can I help students to write early? Here is what I do.

*. After a student (or I, or both) has some promising research idea, and both I and the student feel that the idea could have a high chance of leading to a good research project, before the student goes ahead to carry out the implementation of the approach, I will ask the student to write down the following sections (see my slides on writing papers for a paper's structure):
-- Abstract
-- Introduction
-- Example
-- Approach (the description of the approach could be high level)
(No implementation section or evaluation section is required at this stage)
-- Related work
-- Conclusion

If the student can produce some preliminary results (e.g., by doing some manual walk-through of the approach), the student can also put in a "preliminary results" section. The resulting draft would form a fine workshop submission draft (but note that it is not necessary for us to submit it as a workshop submission unless really needed in some cases to gather early feedback on the work from the community).

Basically, the student presents to me a "design" document (similar to one in software development) before the student starts implementation. Then I can revise and understand what the student is planning to implement, and brainstorm with the student on possible new ideas or improvement of the existing ideas based on the writing. Doing so can avoid some of the past issues on a student's approach or research project being a bit "black-box" to me and I found out issues when reading the student's draft when being very close to the deadline and then the student couldn't fix the issues that I identified.

Note that, at this stage, the student must identify a convincing motivating example and describe it in the example section before moving on to implement the approach. I had one past bad case where a student couldn't find a convincing motivating example for his approach and then moved on to the implementation of the approach, and ended up with an unsuccessful project.

*. After the student implements the approach, the student expands and refines the approach section and adds the implementation section. I will review and revise these sections. Note that in this step when the student does some hands-on stuffs, new ideas may come up and the originally written down approach can be revised and improved. I am very supportive on the student's hands-on experience and generating new ideas from hands-on experience. But that doesn't mean that the student should directly dive into tool implementations without thinking enough or finishing the first step of paper writing described above. In my belief, good new ideas can come from both creative thinking before tool implementation and during/after tool implementation.

*. Before the student carries out the evaluation, the student needs to write the experimental design subsection and other subsections in the evaluation section that can be written before the experimental results are available. Doing so can help me make sure that the student is doing the right things in the evaluation.

*. After the student finishes the evaluation, the student fills in the experimental results subsection. I will review the new subsections as well as the whole draft before submission.

Doing this style of phased writing, students benefit in several major ways. (1). They don't rely on me to write for them before the deadline (since I won't). (2). They allow me to have opportunities to iterate with them on their writing and improve their writing since in the early phases, we don't have the deadline pressure. (3). They can gather feedback from me early on to help them to avoid making wrong decisions or misinterpreting my suggestions in their research development. (4). Our remote research collaborators (if any) can read through the early formal writing to give us feedback and suggestions on the research development.

So far I have quite good experience in training and educating students to write research papers with the above ways. In the end, I indeed enforce what I told my students "Don't expect me to write sections of your papers for you". More importantly, most of the students (after working with me for a short period of time) in my group can independently write whole papers without my writing there (with my helps only in a way of my not-very-significant revising on hardcopies of their papers). I am very glad to see that happen while knowing that many of my fellow junior faculty members are still busy writing papers for their students.

Hope my experiences described above can further help my students to understand why I am doing so and further improve what they are doing while working with me, and help other advisors at least to think how to do better along this line of advising students in writing papers...

Research skills

Posted on Saturday Oct 04, 2008

Yesterday I gave a talk on research skills, whose slides are here. Feedback is welcome.