Intructions for Interviewers

Experiences of Learning to Code

Author

Affiliations

Joe Marsh Rossney

UK Centre for Ecology & Hydrology

University of Edinburgh

Abstract

This document contains step-by-step instructions for conducting 1-1 interviews with students over Microsoft Teams. We begin with the point of first contact with the selected student, and end with instructions on how to redact and format the Microsoft Teams transcript, ready for analysis.

1 Making contact

At this point we need to reach out to the selected students using the emails they provided in the ‘registration of interest’ survey.

It is important to realise that by using email we are significantly increasing the number of ways in which a ‘human error’ could result in the identity of a future participant being shared.

Please take your time and be extra vigilant!

Important

If you are sending an email to 2 or more students, always use the BCC¹ field for their email addresses. There should be no email addresses in the to or CC fields!

1.1 Students selected for interview

You will need to email these students to arrange a time for the interview.

This might sound easy but it’s a task that rapidly increases in complexity as the number of students increases.

Make sure you are clear that the interview is expected to last 45 minutes.

1.2 Students not selected

Don’t immediately tell them, since some of the originally selected students may drop out or fail to respond, in which case we may want to ask some of the originally deselected students.

Thank you very much for volunteering to participate in this study…

2 Before the interview

2.1 Scheduling the interview

We are using Microsoft Teams to conduct interviews, which is recommended by the University of Edinburgh Information Services for 1:1 interviews where security is important.

Open Microsoft Teams. This can be done in your browser without needing to download the app.

1. Open your calendar, navigate to the desired meeting time and click + New meeting.
2. Set the title to ‘Interview for research project: Experiences of learning to code’.
3. Set the start time to the time agreed with the student, and the end time to 45 minutes later.
4. Open the Show as: drop-down menu in the top left of the meeting dialogue box, and select Busy.
5. In the same drop-down menu, where should now read Show as: Busy, check the option that says Private.
6. Over on the top-right, click on the Options button, open the drop-down menu underneath Who can bypass the lobby? and select People who were invited. (If you don’t see these settings, check that the Online meeting toggle has been set.
7. In the lowerst box that says ‘Type details for this new meeting’, write: ‘Please ensure that you have completed the Informed Consent Form before the interview!’
8. Below the title, add the student as a required attendee.
9. Double check that everything is in order, and hit Save in the top-right.

The student will receive an invitation to the meeting via email. In many cases they will actually press ‘accept’ and you will get a notification about it, but do not panic if they don’t.

Important

It is very important that the meeting is private (step 5) since otherwise it may be visible to any member of the university.

Further guidance for setting up private, secure Teams meetings is provided by ITS here.

2.2 Once the interview is scheduled

For each interview that you schedule, make a note containing the following information:

1. Anonymous case ID (e.g. ‘Student F’, not their name or UUN!)
2. Date and time of interview
3. Any relevant information provided in their response to the sign-up survey
4. An indication of whether or not they have completed the Informed Consent Form yet.

You should check the JISC survey containing the Participant Information Sheet & Informed Consent Form regularly. Whenever you see that another student has completed the form, look up their case ID, and make a note that they have done so.

2.3 The day before the interview

Send students an email to remind them of the upcoming interview (don’t forget to use BCC if you are emailing multiple students!).

If they still haven’t completed the informed consent form, remind them that they will have to complete this before the interview can start.

3 During the interview

3.1 Starting the interview

1. Say hello, and thank the student for taking the time to participate.

2. Introduce yourself and your role in the project very briefly.

3. Tell them that before the interview starts you will just run through a few aspects of the interview so that you’re both on the same page.

   • Treat this more like a conversation than an interview.
   • Feel free to take the conversation in any direction you feel is important or interesting.
   • You can decline to answer any question.
   • It’s ok to mention people by their names, since these will be redacted from the transcription.
   • Most importantly: The purpose is not to judge or compare, but to simply listen to a wide range of experiences and perspectives.

4. If they have not already completed the Informed Consent Form, stop here! They will have to do this now, while you wait (though it shouldn’t take long).

Caution

If the student has not completed the Informed Consent Form you cannot proceed with the interview!

5. Ask them if they have any questions.

6. Ask them if they are happy to move onto the interview and for you to start the recording.

7. Start the recording by going to More > Record and transcribe > Start recording. This should start both the recording and transcription, but you should double check by opening the live transcription in the sidebar.

8. Ask the student to confirm, for the recording, that they have read the Participant Information Sheet and completed the Informed Consent Form.

3.2 Tips for during the interview

At this point you are ready to start the interview proper, for which you will want to refer to the Interview Guide.

Some general tips:

• If your screen is large enough, keep the guide next to the Teams window, or somewhere that you can read it without making it obvious that you are looking somewhere other than at the student.

• It can be very annoying when a student’s speech is split into multiple chunks in the transcript that are separated by the interviewer making noises (“yeah”, “ok”, “mmm”) that register as speech. You can help as an interviewer by not making noises while the interviewee is speaking. Consider using purely-visual indicators to show the student you are listening, such as nodding or frowning thoughtfully.

• Try to ask questions directly, clearly and succinctly. The style should be conversational and feel natural, but try to keep question preambles and ad-libbed speech fairly short. Partly this is because the more you say the greater the chance that you will accidentally say something you didn’t mean to.

• Keep it professional, non-judgemental and try to maintain a balanced and neutral stance on issues if possible.

• Do not bring your personal experiences or opinions into the conversation. Try to redict any questions about your personal opinions in a polite but firm way.

3.3 Ending the interview

1. Stop the recording.

2. Thank the student once more for their time.

3. Remind them that they will receive their voucher by email in the next few weeks.

4. Check with the student that they know how to get in touch with you if they have further questions about the project or things to add.

5. Say goodbye and end the meeting.

4 After the interview

4.1 Immediately after the interview

4.1.1 Record your immediate impressions

The first and most important thing to do is to note down your immediate impressions from the interview. Reflexive thoughts are equally if not more valuable at this time than analytical ones.

For example, you could make a note of:

• Any parts of the interview that felt particularly interesting or productive
• Anything that jumped out at you as interesting or unusual
• Any parts of the interview that did not go so well, and why
• Reflections on how you could have improved the interview, e.g. by following up on something or wording something differently

4.1.2 Post-interview cleanup

Once you are happy that you’ve gotten your thoughts down, go to the Chat section of Teams. The most recent chat should contain the recording and transcript of the interview (the transcript might take some time to appear).

Click on View recap, and then on Open in Stream in the top-right of the window (underneath Join). This should open Microsoft Stream in a new tab.

There are two things to do here:

1. On the top-right, click the red-pink drop-down menu next to Copy link, click on Manage access and then ‘stop sharing’ with the student.

2. In the top left, rename the video with the case ID (e.g. Student F).

4.2 Rewatching and editing the transcript

The meeting transcript generated by Teams is a reasonable starting point, for out purposes at least. However, it will inevitably contain errors, ommissions and bad grammar, and may also contain passages that could be used to identify an individual. Fortunately it is possible to edit the generated transcript in situ using the Stream app. To edit a block of transcript in Stream you just need to click on it, make the edits, then hit enter to save.

Warning

Be under no illusions: the way we are transcribing these interviews would not be considered ‘good practice’ by qualitative researchers. This is a legitimate criticism of our work, to which my response can be summarised as

1. We simply did not have time/funds to transcribe every interview by hand and in full detail.
2. We are not doing a ‘deep’ sociological or linguistic analysis — this is a bit of a straw man, but the missing detail in the transcripts is unlikely to make a differnece to the kinds of claims we will make based on this data.

4.2.1 Transcription errors

If you try to correct every grammatical error this will take an extremely long time. Focus your effort on making edits that correct serious transcriptions errors or omissions, or clarify passages that would be very confusing to parse without the accompanying audio/video.

Tip

The most essential transcription errors to correct are those that, if left alone, would pose a risk of mis-representing what the student said.

4.2.2 Redactions

You must redact information that could be used to identify an individual. Clearly this means redacting the names of individuals, but may also mean redacting the names of courses, places, organisations, details of projects, etc.

You can redact some text by replacing it with an informative placeholder within square brackets. To give some examples,

> My mate Dave did his project with Professor Thompson...
> Where I come from in Finland we study...
> Last summer I did an internship with Rockstar Games...

becomes

> My mate [name] did his project with [professor's name]...
> Where I come from in [European country] we study...
> Last summer I did an internship with [company]...

4.2.3 Transcribing other elements of speech

It might seem appropriate to include things such as false starts or abrupt cutoffs, pauses, non-verbal sounds and facial expressions in the transcript, since these can help to paint a more detailed, accurate representation of the interaction as it really happened.

When doing so, please follow the conventions described by the University of Toronto.

Note that ‘umms’ and ‘errs’ are not transcribed by convention, which is fortunate for us since Teams tends not to transcribe them.

4.2.4 Analysis

Rewatching the interview can and should serve dual purposes; as well as making corrections to the transcript, you should treat this as an opportunity for analysis. During this process you can make notes underneath those you made immediately after the interview.

4.3 Downloading and formatting the transcript

When you are happy with the corrections and redactions you have made on Stream, download the interview transcript to a secure location on your computer, choosing the .vtt format.

If you open this file in any text editor (e.g. notepad) you will see that it looks something like this:

WEBVTT

91b3f3c3-44c6-4a8b-8c0a-add105d816bd/32-0
00:00:10.087 --> 00:00:13.130
<v Name of Interviewer>Something you said, probably
weirdly split over two lines.</v>

91b3f3c3-44c6-4a8b-8c0a-add105d816bd/32-1
00:00:13.130 --> 00:00:16.270
<v Name of Student>Something the student said
in response.</v>

91b3f3c3-44c6-4a8b-8c0a-add105d816bd/32-2
00:00:16.270 --> 00:00:18.927
<v Name of Interviewer>Now it's you speaking again.</v>

...

4.3.1 Removing the student’s name

The most glaring problem is that the name of the student is still present — we could not change that on Teams/Stream.

Ideally you would run a search-and-replace on this file to replace all instances of Name of Student with their case ID, i.e. Student X.

Tip

If your computer runs on Linux or MacOS and you’re comfortable running a shell script, you can do this with sed.

sed -i 's/Name of Student/Student X/g' student_X.vtt

However, the tool we will use next to format the transcript can also do the search-and-replace, so you can skip this step as long as you do the formatting step straight away (so that the file containing the student’s name does not sit on your computer for very long).

4.3.2 Formatting the transcript

The formatting tool is a Python script which can be found at github.com/ExpLrnCode-2024/teams-transcript-formatter along with instructions for use.

I strongly recommend that you then create a virtual environment with Python 3.11 or higher, using either conda or venv, before installing the tool. I’m afraid I cannot provide reliable instructions for doing this on Windows (but we talked about it in person)!

python -m pip install git+https://github.com/ExpLrnCode-2024/teams-transcript-formatter

In your working directory (which may be the folder containing the transcript), you should create a file called .env which contains a singe line INTERVIEWER="Your Name" where Your Name is as it appears in the transcript. You can also do this by running the command

dotenv set INTERVIEWER "Your Name"

Now run the script,

format-transcripts student_X.vtt

and it should produce a file called student_X_formatted.txt. Open this file in your text editor, and check that it looks something like

Interviewer (00:10):
    Something you said, probably weirdly split over two lines.

Student (00:13):
    Something the student said in response.

Interviewer (00:16):
    Now it's you speaking again

...

If this has not worked, get in touch with me!

Important

You should now delete the original .vtt file from your computer, since it contains the name of the interviewee! (Unless you removed it yourself)

This file should not be ready for further analysis in QualCoder.

4.4 Deleting the interview recording and transcript

According to the University of Edinburgh’s Privacy Statement the recording and transcript from a Teams meeting will be deleted automatically after 700 days.²

That is, however, far longer than necessary for our purposes. Moreover, GDPR dictates that we have a responsibility to delete this data as soon as it is no longer needed for the original purpose stated in the Informed Consent Form. Hence, we must delete the interview recordings and transcripts as soon as we are reasonable confident we no longer need them.

I propose that we delete the recordings after our paper has passed peer review, since it is plausible that we will want to refer to the original recording in case e.g. part of a transcription is identified as confusing or ambiguous, or we need to double check that published quotes, including the grammar, are faithful to the original recording.

Microsoft provide guidance on how to delete a recording on Teams.

Footnotes

1. BCC stands for ‘Blind Carbon Copy’

2. The University are the ‘Data Controller’ under GDPR, and therefore hold responsibility for ensuring that recordings are held on a secure platform.

Copyright

2025, Joe Marsh Rossney