Data Collection Tools Swap Shop

SIP lecture session 2.

TASK:

⟶ In your pairs, half an hour to: ⟶Describe / share your data collection method with you partner. ⟶What “pitfalls” can you spot in each other’s work? What would improve it? Think about order of questions, syntax, clarity, scaffolding…

OUTPUT:

When piloting the questionnaire, I didn’t have a full questionnaire to use, although I had some thoughts around the types of questions I’d like to ask and a provisional layout/order for those, which is shared in the session:

Provisional questionnaire questions

Positive:

FeedbackAction
So far the questionnaire looks neat and questions seem thought through with particular relevance to research topic. If this is maintained in the final data collection tool, this will help build a clear picture/story in relation to research question  Maintain neat visual aesthetic and clarity with questions order when making refinements

Constructive:

FeedbackAction
Could broaden the scope of answers on likert/scale questions  Broaden scope of answers from 3 to 5 different types of answers
Obscurity with use of double questions, e.g. where I have grouped questions about ‘feedback’ and ‘teaching’ into one question – split them into two questions.  Adjust questions to avoid double referencing and separate into singular questions for ‘teaching’ and ‘feedback’ related questions.
Language – the term ‘imposter syndrome’ can seem quite intimidating to readers          Take extra care with wording, omit use of terms ‘imposter syndrome’ and ‘imposter phenomenon’ on data collection tools (Clance 2021, Kaplan 2009). Instead focus on and wording questions around the thoughts and feelings that can be associated with IP without directly referencing the term.  
Specify if you’re referring to online/in-person teaching/feedback given current COVID climate  Clarify the teaching/feedback context in question e.g. online vs in person, written vs verbal etc.

References

  • Bell, J. (2010) Doing your research project: A guide for first-time researchers in education, health and social science. Maidenhead: Open University Press.
  • Clance, P. (2021) Email to Stephanie Martindale, 6th March 2021.
  • Clance, P. (2013) Imposter Phenomenon. Available at: https://paulineroseclance.com/impostor_phenomenon.html
  • Kaplan, K. (2009) Unmasking the impostor. Nature, 459(21): 468-469. doi: 10.1038/nj7245-468a.

Research Methodology

Using Bell’s (2010) guidance on choosing an appropriate data collection method, I started with the question ‘what do I need to know and why?’ followed by ‘what is the best way to collect this information? Answering this in relation to my research question and associated questions that I want to explore:

How pedagogical applications in feedback delivery influence the provocation of imposter phenomenon in higher education students.

  1. Do the pedagogic applications of HE lecturers in feedback deliveryhave influence on the provocation of imposter phenomenon (IP) in HE?
  2. What practices can teachers adopt in their delivery of teaching and feedback to help mediate the development of imposter phenomenon?

I mapped the various possibilities for data collection and the options (including pros/cons) as a starting point:

A mind map of research methods

We discussed this further in our lecturer, breaking out into small groups to identify the pro’s and con’s of each method type, before collating them in a padlet document, which I’ve referred back to throughout the journey of this project:

https://artslondon.padlet.org/ncurrant/methods

In order to begin refining my mind-mapped thought and designing my data collection tool, I took a simplified approach and broke down my main research topic into smaller, more digestible questions:

Question: What exactly do I need to know in relation to my research question?

Answers:

  • Do students experience IP?
  • To what extent do student experience IP?
  • What influence does the teaching delivery have on IP?
  • What influence does feedback delivery have on IP?
  • If a relationship is established between these variables (teaching/feedback) and IP, how do students think teachers can influence or change in the learning environment to help mediate the effects of IP?
  • What can be changed or implemented in the learning space to do this?

When thinking about determining an appropriate methodology for my project, I utilised questions encouraged by O’Leary (2020) and used this as a framework to help further develop my approach to data collection. My answers are highlighted in red:

☑ Who

  • Who do you want to be able to speak about? IP – considered a sensitive topic, the learning environment and the relationship between the two aforementioned topics
  • Who do you plan to speak to/observe? Students at UAL, students who experience IP to some degree

☑ Where

  • What is the physical domain of your sample? Sample will be participating via digital/online mediums

☑ When

  • How do your methods fit into your time frame? Short time frame, small scale study, requires non time-consuming data collection method
  • Is timing relevant to the credibility of your methods? No

☑ How

  • How will I implement my methods? Online/digital implementation required due to COVID 19 restrictions on social distancing

☑ What

  • What will you look for/what will you ask? I will ask about student experiences of IP, and whether the learning environment has had any influence on these

With a clearer understand of the type of information I needed to retrieve to investigate the research questions, I started thinking about additional considerations and constraints within the scope of the project:

Constraints

  • Time – 3 month project timeframe – restrictive
  • Funds – no budget available to support use of costly resources to support project
  • Resource – limited capacity to broaden scope and depth of project with only 1 researcher

Chosen Methodology

Giving thought to the advantages and disadvantages of each method, the constraints of the project, and the information I require for my research, I chose to proceed with the questionnaire method of data collection.  This method of data collection will allow to me collect data in a short timeframe, with no financial investment required and with only myself and researcher.

Considerations

  • Qualitative data – depth, richness of data

Vs.

  • Quantitative data – credibility through larger samples and statistical analysis
  • Questionnaire has both qualitative and quantitative features for mixed-method approach
  • Sampling – convenience sampling method used due to aforementioned constraints
  • Negotiating access to participants

Designing my Questionnaire – Characteristics

Given that one of the first things I’d like to find out from my data collection is whether students experience Imposter Phenomenon (IP), I explored ways to measure and identify this in my sample. It was important for me to obtain a sample from participants with IP, as their experiences and shared data would be key in helping me understand the extent of IP to then help me analyse and formulate conclusions.

During my reading and research further into IP, I came across Dr. Pauline Clance, a clinical psychologist with an extensive body of work on the topic of IP (2021, 2013, 1978). Clance developed the Clance Imposter Phenomenon Scale (CIPS), which is a questionnaire tool to help identify markers for IP and IP traits. I reached out to Clance via email and sought permission to use the questionnaire as part of my study, for which permission was granted.

Given that first part of my research required me to identify participants experiencing IP with the CIPS questionnaire, I knew I needed to expand beyond this with an additional questionnaire to identify more of the specific detail required to help me explore my research question. I identified the CIPS as questionnaire A. I then developed my own research questionnaire which, throughout this project and on participant facing communication, I identify as questionnaire B. This questionnaire focusses more closely on IP specifically in the learning environment in relation to teaching and feedback experience:

  • Designed as a questionnaire to be disseminated digitally for improved accessibility and mitigation of cost implications
  • Varied question style – open /closed/likert questions
  • To be used in conjunction with Questionnaire A
  • The complete data collection tool is formatted in two parts (Questionnaire A/B):
    • Use of Dr. Clance’s IP scale questionnaire to understand prevalence and extent of IP in students (questionnaire A)

in conjunction with:

  • Specific questionnaire designed to answer the research question (questionnaire B)