Semi-Structured Interview
& Observation
Overview
Qualitative Interview
Mini User Needs Assessment
Overarching Question
What features do users deem useful or frustrating about the Coursera video UI interface and related features on the webpage (ex. Note Taking) while taking a course? What features hinder or help their learning process? Can features be tweaked or added to supplement their learning experience?
Research Method
Semi-Structured Interview
& Observation
& Observation
Data Analysis
Affinity Wall
(Thematic Analysis
& Recursive Abstraction)
(Thematic Analysis
& Recursive Abstraction)
Interesting Findings
Finding
01
Users like and want more visual resources and examples that are easily accessible and obviously discoverable from the video webpage.
Finding
02
Though incredibly useful and helpful, users pause and rewind excessively for short videos.
Finding
03
Even though a particular user rewinds, pauses, and takes note frequently she still struggled to retain information.
Does the pausing feature cause users to take verbatim notes instead of generative notes?(generative notes are more conducive to conceptual understanding/learning)
Does the pausing feature cause users to take verbatim notes instead of generative notes?(generative notes are more conducive to conceptual understanding/learning)
Finding
04
Users could be using the "take note" button to take/save screenshots with less effort but they aren’t. Why?
Finding
05
Users have other preferred means of note taking and preferences that explain why they didn’t use the notes & highlight features.
Finding
06
Though a preference for other means of note taking exists, the notes feature on the webpage is still considered unintuitive and confusing.
Finding
07
Users did not realize they could download PowerPoint slides of videos and thought it was a useful feature.
Finding
08
Users want a streamlined way to access other pages and overview weekly, course and specialization information.
Introduction
Background Information
Description of Product
Coursera is an online learning platform that offers users the opportunity to explore a variety of different courses and even offers degrees or specializations. Users may use the platform for pure curiosity of learning as well as a means to advance their career.
Scope
The webpage that contains the actual Course Content Video and related features (note taking), for a course
Main Focus of Mini User Needs Assessment
Once users join a Coursera course and are in the process of learning, what features do users deem useful or frustrating about the Coursera video UI interface and related features on the webpage (ex. note taking)? Since we all “learn” differently or have our preferences when learning, the question aims to explore what features can be frustrating or useful during the learning experience (i.e watching the video) and what features users feel hinder or help their ability to learn.
Coursera video webpage
Overarching Question
What features do users deem useful or frustrating about the Coursera video UI interface and related features on the webpage (ex. Note Taking) while taking a course? What features hinder or help their learning process? Can features be tweaked or added to supplement their learning experience?
Methods
Participants
3
Interviewers
Myself
Design/Methodology
Semi-Structured Interview & Observation
Interview Protocol / Script
[Introduction]
Hi,I’m Lyn! How are you today?
Today I am here to interview you and better understand how you use Coursera when you are in the process of taking a Course. Our specific focus will be on the webpage that contains the course content (video, transcript, and related features). During this interview I will ask you a couple of questions and also observe you performing a task relating to the service. I’d like you to pretend as if I have no prior knowledge of the application and as such answer my questions with as much detail as possible. The interview will take about an hour.
Anything you say is confidential and data will be aggregated in a way such that your comments will be anonymous. Please let me know if there is anything you would like to keep off the records. This interview is entirely voluntary- if you want to stop at any time please let me know. We can end the interview at any point. For data integrity reasons we like to record our interviews. No one other than theresearch team will have access to this recording. Is it alright if we record this interview?
Great!
Do you have any questions?
Let’s begin!
[Warm Up Questions]
How often do you use Coursera?
How many courses are you taking?
When was the last time you used Coursera?
[Core Questions]
Q1.
Why did you decide to take these courses or join Coursera?
- Follow up questions:
- What do you hope to gain from the course?
- What is your goal for taking this course?
- Do you intend on completing the course? If so, after you have completed the course what do you want to walk away with at the end of the course?
- What do you intend on doing with the knowledge you have attained from the course?
Q2.
Can you tell me about the most recent time you used Coursera?
- Follow up questions:
- Why were you on Coursera? What were you doing?
- Where were you? Were you alone?
- What was your setup? Where you at a desk, on a couch, on a bed etc?
- How long where you on Coursera for?
- What course where you working on? Are you working on it with someone else?
- How many videos did you watch?
- Can you walk me through exactly what you did while on Coursera?
- Now I want you to think about the time before that, can you tell me about that time?
For the following questions I want you to think more specifically to when you are engaging with the webpage that contains the actual content (video, transcript) and its related features for any course you are taking. (Shows them the page)
Q3.
Can you think back to the last time you were on this page. Can you tell me a little bit about that?
- Follow up questions:
- Can you walk me through what you did?
- Did you watch the video? Were you doing other things while watching the video?
- Did you pause the video at any time? Why?
- Why did you think you did the things you did?
- Did you come across any problems or nuisances?
- Can you tell me about the time before that…?
Q4.
Can you think of a specific instance when you used any of the features on this webpage?
- Follow up questions:
- What were you trying to accomplish?
- Why do you think you decided to use that feature?
- What is your opinion on that feature?
- Did it cause you any frustration at any point in time? Can you tell me about that time?
- Why do you think is? (if the user never used any of the features?
Q5.
Can you think of a particular instance when something on this webpage caused you any frustration?
- Follow up questions:
- What were you doing?
- What were you trying to do?
- What do you think the reason for the frustration was?
- What was the source of the frustration?
- How did you try to address the problem?
- What is your opinion on the issue?
Q6.
Can you tell me about a time that you engaged with another feature while simultaneously watching the video?
- Follow up questions:
- Can you tell me a little bit more about that?
- What feature or features did you engage with?
- How would you describe you experience while engaging with both?
- Did you come across any issues?
Q7.
Can you tell me about a time that you went to this webpage and did not watch the video/had no intention on watching the video and instead wanted to perform another task or do something else?
- Follow up questions:
- What did you want to do/ what were your intentions on the webpage?
- Do you feel as if you were able to successfully do whatever it is that you wanted to do?
- Did you encounter any problems or nuisances?
- What feature/s did you use?
Q8.
Can you tell me about the last time you attempted to learn or memorize content from one of the videos? (for the quiz at the end of the module or for you own knowledge)
- Follow up questions:
- Tell me more about what you did to accomplish this goal?
- Did you open up the webpage with that particular content?
- Was this the first time you were on that particular content?
- Did you attempt to memorize the content the first time you were exposed to the video or did you come back to it at a later time?
- Can you tell me about the time before that?
Q9.
Everyone has their own style of learning or process of learning. Tell me a little bit about how you feel you learn best?
- Follow up questions:
- Tell me about the last time you had to learn something
- Can you tell me how you went about "learning it"?
- Did you feel like you had a good understanding of whatever it is you learned?
- Tell me about a time when you had to learn something and you thought you learned it but apparently you didn't learn it well enough? What do you think contributed to this?
- Are there any frustrating aspects about learning?
Q10.
How do you feel regarding your experience on this webpage?
- Follow up questions:
- How do you feel about the interface?
- Why do you think you feel that way?
- How do you feel about the features?
- How would you compare this experience with in-person instruction such as a lecture or experience being in a classroom?
[Explanation for the Core Questions]
Q1 – Q2: Getting general information about their intention and typical behavior. Intentions for the course may influence which features they decide to use (if any). This will help me understand how they tend to engage with the course content (example Person A just watches the video but Person B likes to highlight important aspects of the video while watching)
Q3-Q7: Create scenarios to help jog the user’s memory of instances when they could have used a feature.This can hopefully help them more easily recall any potential issues with those features. Also aides in identifying intention for use of those features.
Q8-10: These questions aim to gain knowledge in potential gaps in “needs” of the webpage. By comparing to how they normally learn maybe this can help create some ideas for feature enhancement of existing features or generation of ideas for new features.
[Abstract Generalized Questions for Fun]
Is there a feature or attribute you wish was included?
Is there something that could have been better done to help you learn or make the experience more enjoyable?
Is there anything else you would want to add any opinions or ideas of any sort?
[Potential Observation Questions]
Ask user to perform a task - Think Aloud Protocol
Ask user to perform a task - Think Aloud Protocol
I am now going to ask you to perform a couple of tasks while I look over your shoulder and watch you perform them. All the tasks are related to the page we have been discussing. While performing the tasks I ask that you let me know what you are thinking throughout the process. You can truly tell me anything at all.
Tasks that will vary greatly per user
1. Let’s say you are continuing the course you have been working on. I want you to pickup where you last left off and engage with the next course content video and webpage as you normally would at home.
2. Now I want you to do what you normally do if you had to memorize information on this particular content.
Specific Tasks
Now I want you to move on to the next video
3. While watching the video I want you to use the highlight feature to highlight things you deem important.
4. (When the person says they are done with the last task) Now I want you to check how many videos or content you have to do before your next assignment. (To see if they will use the content feature icon on the left right)
Follow up questions for tasks
What are you doing?
What are you thinking?
What do you think when you do that?
Why are you hesitating?
[Conclusion]
Thank you so much! That concludes our interview. I may reach out to you if I have any additional follow up questions. We will also report the results to you. If anything occurs to you that you forgot to mention please feel free to reach out! Here is my contact information.
If you have any questions or have anything to add, you are free to ask me now.
Thanks again!
Results
Highlighted Findings
Finding
01
Users like and want more visual resources and examples that are easily accessible and obviously discoverable from the video webpage.
Quotes
"For the course assignment 10 sketches it would have been cool to have some examples"
Participant 03
Recommendations
· Encourage professors to Include more examples (even previous student work)
·Left hand panel could function to hold supplemental material relating to that topic/ video
·Taking google’s ux course as an example, after a video they provide the same topics covered as an article and it typically includes links to other websites (with more examples)covering that topic
·Left hand panel could function to hold supplemental material relating to that topic/ video
·Taking google’s ux course as an example, after a video they provide the same topics covered as an article and it typically includes links to other websites (with more examples)covering that topic
Full Analysis
This higher level finding is supported by a combination of three other clusters. First users reported using, liking, and wanting more visual resources and examples. Second, streamlining issues were reported, such that users felt that course resources were not easily accessible from the webpage containing the course content video. Third, features did not attract attention and thus users were unaware of capabilities of features (discoverability aspect).
The summary cluster - users like and want more visual resources and examples to supplement their learning - is supported by two lower level clusters. The first lower level cluster supports the notion that users like using existing resources and want more visuals. In support of this cluster all 3 users interviewed considered themselves to be visual learners. Participant 03expressed wanting more visual content available like examples, they stated “For the course assignment 10 sketches it would have been cool to have some examples”. Participant 02 expressed how useful she considers the course documentation page “the documentation page has a table and link for the method…which is really useful for me to understand what the code is doing”. Thus, visual resources and examples are crucial in aiding users in their understanding of course material but there is a lack of it. The second lower level cluster provided evidence that users like to take screenshots to save as examples. Two users expressed taking screenshots to save for a later time. This supports a desire and like for visual resources and examples.
The second main cluster summarized that streamlining issues were reported, in particular one user expressed that the resource documentation page is confusing to get to. She expressed the sentiment “every time I try to get to it from Coursera I get lost”. She mentioned that the link was given to her in the first introduction video but for convenience she saves it in her notes. She further expressed the sentiment that the webpage containing the course content and video did not have a good place to document references. The user also expressed in general that she feels like she always has to do a bunch of clicks to get to where she wants. These examples support the need for supplemental material to be easily and directly accessible from the webpage containing the course content video.
The third main cluster- features did not attract attention thus users are un aware of availabilities of features - provides one direct support and general support for the need of supplemental material to be discoverable in the webpage containing the course content video. The main evidence is that participant 03had no idea he could download PowerPoints of the course. In general, it appeared that many features were left unexplored. Participant 02 expressed “I don’t know how to actually utilize the highlight feature”. Participant 03 also did not realize that the left hand panel could open, he said “I never thought about that icon… it didn’t catch my eye”. Participant 01 did not realize the highlight followed the speaker while talking. This provides support that supplemental material should be easily discoverable, if available, so that the user knows it exists.
The summary cluster - users like and want more visual resources and examples to supplement their learning - is supported by two lower level clusters. The first lower level cluster supports the notion that users like using existing resources and want more visuals. In support of this cluster all 3 users interviewed considered themselves to be visual learners. Participant 03expressed wanting more visual content available like examples, they stated “For the course assignment 10 sketches it would have been cool to have some examples”. Participant 02 expressed how useful she considers the course documentation page “the documentation page has a table and link for the method…which is really useful for me to understand what the code is doing”. Thus, visual resources and examples are crucial in aiding users in their understanding of course material but there is a lack of it. The second lower level cluster provided evidence that users like to take screenshots to save as examples. Two users expressed taking screenshots to save for a later time. This supports a desire and like for visual resources and examples.
The second main cluster summarized that streamlining issues were reported, in particular one user expressed that the resource documentation page is confusing to get to. She expressed the sentiment “every time I try to get to it from Coursera I get lost”. She mentioned that the link was given to her in the first introduction video but for convenience she saves it in her notes. She further expressed the sentiment that the webpage containing the course content and video did not have a good place to document references. The user also expressed in general that she feels like she always has to do a bunch of clicks to get to where she wants. These examples support the need for supplemental material to be easily and directly accessible from the webpage containing the course content video.
The third main cluster- features did not attract attention thus users are un aware of availabilities of features - provides one direct support and general support for the need of supplemental material to be discoverable in the webpage containing the course content video. The main evidence is that participant 03had no idea he could download PowerPoints of the course. In general, it appeared that many features were left unexplored. Participant 02 expressed “I don’t know how to actually utilize the highlight feature”. Participant 03 also did not realize that the left hand panel could open, he said “I never thought about that icon… it didn’t catch my eye”. Participant 01 did not realize the highlight followed the speaker while talking. This provides support that supplemental material should be easily discoverable, if available, so that the user knows it exists.
Finding
02
Though incredibly useful and helpful, users pause and rewind excessively for short videos and for a particular individual it did not seem to benefit her retention of the information.
Does the pausing feature cause users to take verbatim notes instead of generative notes?(generative notes are more conducive to conceptual understanding/learning)
Quotes
“But I retained very little information from writing it down… I failed the quiz…The information didn’t go through I was just writing it”
Participant 01
“It was a lot of going back then writing, looking… and then again rewinding, writing notes, and then looking…because I loose my train of thought”
Participant 01
"I never watch it all the way through in one sitting"
"I pause almost every time I am taking notes"
"I pause almost every time I am taking notes"
Participant 02
Recommendations
· Investigate frequency and extent of potential issue · Online learning platforms should promote more effective ways for information retention. (Could be done via more frequent quizzes, in video quizzes)
· Online learning platform should promote effective learning strategies & note taking.
· Online learning platform should promote effective learning strategies & note taking.
For Further Investigation
· Other users did not report issues with passing their quizzes despite reporting frequent pausing and rewinding, thus it's important to investigate the frequency of this issue. Does it only effect a small number of users, a certain type of user? Or was this merely an anomaly?
· It is still possible that the pausing/ rewinding feature enables users to take more verbatim notes instead of generative notes? (however this may not effect everyone as extremely)
· Situationmay insinuate that better means of note taking could and should be promoted
· Are users of onlinecourses in general retaining less information because of the freedom ofpausing/rewinding?
· Are users of online courses spending more time taking notes and not reaping actual benefits from this additional effort?
· It is still possible that the pausing/ rewinding feature enables users to take more verbatim notes instead of generative notes? (however this may not effect everyone as extremely)
· Situationmay insinuate that better means of note taking could and should be promoted
· Are users of onlinecourses in general retaining less information because of the freedom ofpausing/rewinding?
· Are users of online courses spending more time taking notes and not reaping actual benefits from this additional effort?
Full Analysis
This finding is supported by two general clusters. The first cluster summarizes the behavior of users pausing and rewinding as being “a lot” for generally short videos, typically for purposes of note taking. The second cluster focuses on a particular individual who, despite pausing and rewinding excessively to take notes, still failed her quiz. All users reported pausing and rewinding a lot. Users paused in particular for note taking purposes. Users reported pausing for many reasons the top reason being for note taking. Other reasons were included, for example participant 02mentioned pausing in order to respond to work emails. Participant 02 expressed“I pause almost every time I am taking notes”. All participants expressed their pausing and rewinding habit as a lot. Participant 01 expressed her rewinding as almost excessive by stating “I rewinded a couple times… at one point I lost track”. Participant 02 expressed “I do go back pretty often especially when it’s a dense concept… I go back like 10 seconds there is no button I just estimate it…”. Despite not expressing any frustration for rewinding maybe a 10second rewind should be included. To summarize all users reported pausing and rewinding a lot especially for note taking purposes and it brings up the question if there is such a thing as excessive rewinding and pausing, especially for generally short videos?
The second cluster focuses on one particular individual. This participant reported pausing and rewinding a lot for note taking purposes but still did not retain the information well enough to pass her quiz. She described her sentiment as“But I retained very little information from writing it down” and her previous encounter with a video as “It was a lot of going back and then writing, looking at the video… because I lose my train of thought”. The participant mentioned that rewinding is one of her favorite features because she does not have to feel guilty about asking someone to repeat themselves. Her failing to retain information is worrisome because it brings into question whether all her effort was in vain.
This is an interesting case because it brings into question if a well-liked feature like pausing and rewinding can cause users to engage in habits that are not effective for learning and retention. In particular is the pausing feature allowing users to take more verbatim notes than generative notes (that summarize and paraphrase)? In a study conducted by Mueller andOppenheimer they found that students who took longhand notes tended to take generative notes and as such performed better in conceptual questions than those who took laptop notes that took verbatim notes. You can’t pause in real life to take verbatim notes so you are forced to summarize, thus I wonder if the pausing feature enables people to take verbatim notes and as such loose the learning that occurs when taking generative notes in person.
Are some users, like this particular participant, engaging in habits, like excessive pausing and rewinding, that are time consuming but ineffective learning strategies? Should the system provide recommendations for how to best retain information? Should the system identify some troublesome habits? How do you balance user freedom while providing assistance to better learning?
The second cluster focuses on one particular individual. This participant reported pausing and rewinding a lot for note taking purposes but still did not retain the information well enough to pass her quiz. She described her sentiment as“But I retained very little information from writing it down” and her previous encounter with a video as “It was a lot of going back and then writing, looking at the video… because I lose my train of thought”. The participant mentioned that rewinding is one of her favorite features because she does not have to feel guilty about asking someone to repeat themselves. Her failing to retain information is worrisome because it brings into question whether all her effort was in vain.
This is an interesting case because it brings into question if a well-liked feature like pausing and rewinding can cause users to engage in habits that are not effective for learning and retention. In particular is the pausing feature allowing users to take more verbatim notes than generative notes (that summarize and paraphrase)? In a study conducted by Mueller andOppenheimer they found that students who took longhand notes tended to take generative notes and as such performed better in conceptual questions than those who took laptop notes that took verbatim notes. You can’t pause in real life to take verbatim notes so you are forced to summarize, thus I wonder if the pausing feature enables people to take verbatim notes and as such loose the learning that occurs when taking generative notes in person.
Are some users, like this particular participant, engaging in habits, like excessive pausing and rewinding, that are time consuming but ineffective learning strategies? Should the system provide recommendations for how to best retain information? Should the system identify some troublesome habits? How do you balance user freedom while providing assistance to better learning?
Finding
03
Users could be using the “Take Note” button to take/save screenshots with less effort but they aren’t. Why?
Quotes
"I don't actually know how to use the notes feature"
Participant 02
"Immediately after trying it I was like nope"
"I didn't figure it out so I decided to take handwritten notes"
"I didn't figure it out so I decided to take handwritten notes"
Participant 01 in reference to trying the notes feature
Recommendations
· Separating the functionality of the notes feature (screenshotting being a separate function)or simply replacing the notes features with a screenshot button
· Changing the icon& language of the note features to speak to the users need (screenshotting)
· Improving how the note currently appears on the left hand panel
·Not trying to replace peoples existing means of taking notes, but instead supplement it
· Changing the icon& language of the note features to speak to the users need (screenshotting)
· Improving how the note currently appears on the left hand panel
·Not trying to replace peoples existing means of taking notes, but instead supplement it
For Further Investigation
· User statistics/click rates would be beneficial to shed more light on the notes button and frequency of use
· Why people are quickly abandoning the notes features when they do try it (more specifics)
· Why people are quickly abandoning the notes features when they do try it (more specifics)
Full Analysis
What we know
1. All users reported liking and taking manual screenshot (CMD + opt + 4) to save as examples
2. Currently the “Take Note” button saves a screenshot to the right hand notes panel along with other data points (portion of transcript in the form of a quote, and a time stamp)
3. None of the users reported using the “Take Note” button We can see that taking screenshots is a user need and a functionality that is supported by the “Take Note” button but it isn’t being used.
Why didn’t they use the feature, if it supports a functionality they are doing (screenshoting)?
1. User’s didn’t explore the notes feature and are thus unaware of its functionality
Hypothesis:
Icon & language of “take notes” is ambiguous and doesn’t speak to the user’s need
The user who didn’t even try the take notes feature also thought it would be useful to have a screenshot feature, he actually described the exact functionality of the take notes button “I would like to be able to take a screenshot of this moment and take notes to specifically that point”. This hints at the thought that maybe if the “Take Notes”button was a screenshot button that it would prompt more users to at least try the functionality, because it would speak more directly to the user’s need.
Didn’t explore because of existing preferences for other means of note takingUsers reported taking either hand written notes or notes in google docs. The other user who did not explore the notes feature reported taking manual screenshots to save into her google documents.She even reported returning to a video and searching for the screen to take a snapshot. Thus we can clearly see the need for screenshotting. Because users already have existing means of note taking, maybe users did not bother to explore the “take note button” because they weren’t looking to replace their existing means of taking notes. Here we see again an issue of language. This thus can support the idea that maybe if the icon or language of take notes was geared more towards taking a screenshot, the feature would be more likely to be explored.
2. User’s quickly lost interest in the notes feature expressed dislike of it and that it was not intuitive
Could insinuate low probability of adoptability based on the users reported dislike of it and quick abandonment
After getting over the first hurdle of having the icon attract sufficient attention to prompt the user to try the feature the second hurdle is actually improving the notes features itself (presentation of the data on the left handpanel) so that users actually use the feature to make screenshotting easier.
1. All users reported liking and taking manual screenshot (CMD + opt + 4) to save as examples
2. Currently the “Take Note” button saves a screenshot to the right hand notes panel along with other data points (portion of transcript in the form of a quote, and a time stamp)
3. None of the users reported using the “Take Note” button We can see that taking screenshots is a user need and a functionality that is supported by the “Take Note” button but it isn’t being used.
Why didn’t they use the feature, if it supports a functionality they are doing (screenshoting)?
1. User’s didn’t explore the notes feature and are thus unaware of its functionality
Hypothesis:
Icon & language of “take notes” is ambiguous and doesn’t speak to the user’s need
The user who didn’t even try the take notes feature also thought it would be useful to have a screenshot feature, he actually described the exact functionality of the take notes button “I would like to be able to take a screenshot of this moment and take notes to specifically that point”. This hints at the thought that maybe if the “Take Notes”button was a screenshot button that it would prompt more users to at least try the functionality, because it would speak more directly to the user’s need.
Didn’t explore because of existing preferences for other means of note takingUsers reported taking either hand written notes or notes in google docs. The other user who did not explore the notes feature reported taking manual screenshots to save into her google documents.She even reported returning to a video and searching for the screen to take a snapshot. Thus we can clearly see the need for screenshotting. Because users already have existing means of note taking, maybe users did not bother to explore the “take note button” because they weren’t looking to replace their existing means of taking notes. Here we see again an issue of language. This thus can support the idea that maybe if the icon or language of take notes was geared more towards taking a screenshot, the feature would be more likely to be explored.
2. User’s quickly lost interest in the notes feature expressed dislike of it and that it was not intuitive
Could insinuate low probability of adoptability based on the users reported dislike of it and quick abandonment
After getting over the first hurdle of having the icon attract sufficient attention to prompt the user to try the feature the second hurdle is actually improving the notes features itself (presentation of the data on the left handpanel) so that users actually use the feature to make screenshotting easier.
Discussion
Threats to Validity
- Like all qualitative interviews, it is subjective
- Low number of participants, I only had three but at least 7 is more common
- Very minimal instances of leading the witness but in a few instances it may have occurred
- The users were not all typical members, 2 of the users were less frequent users of the platform, 1 was more of a daily user
- Scope was too broad, because the notes features is composed of many components (like highlighting, notes button, right hand panel, left hand panel)
- Due to 2 remote interviews (2 of the observation questions where changed) thus results were mostly dependent on the interview
- Low number of participants, I only had three but at least 7 is more common
- Very minimal instances of leading the witness but in a few instances it may have occurred
- The users were not all typical members, 2 of the users were less frequent users of the platform, 1 was more of a daily user
- Scope was too broad, because the notes features is composed of many components (like highlighting, notes button, right hand panel, left hand panel)
- Due to 2 remote interviews (2 of the observation questions where changed) thus results were mostly dependent on the interview