1. Evaluation Strategy
Although the intervention has not yet been enacted, this post outlines how it will be evaluated once implemented. In keeping with an Action Research approach, my evaluation strategy prioritises situated observation and participant experience, recognising that meaningful insights may emerge through active participation rather than abstract measurement alone (McNiff, 2013). I’ve therefore designed my data collection and analysis methods to be lean, ethical and feasible within a Fine Art studio context.
2. Accessibility and Evaluation
The project will draw upon a combination of observational notes, optional anonymous participant feedback and documentation of how the system I’ve designed is to be configured and accessed during use. My evaluation will consider technical performance such as latency, stability and ease of use, as well as any pedagogical and accessibility dimensions. Key questions include; who is able to use the system comfortably; how it affects studio flexibility; and whether it enables participation that would otherwise be difficult.
3. Three Tasks
There are three tasks contained in the intervention that are designed to introduce participants to the use of the technology, whilst observing their reactions, problem solving and identifying areas of friction with the system. The hope is that by interacting with the system physically whilst narrating this experience, participants unlock deeper observations that in-turn improve their responses in the post-task feedback form. These methods align with Action Research’s emphasis on situated, reflective data and match the ethical commitments laid out in my Ethical Action Plan.
4. Three Provocations
The intervention as a whole is structured around three short, task-based activities, each designed to introduce participants to a specific aspect of the proposed hybrid studio system. These tasks are not intended as a means to test user competence but as provocations that highlight moments of ease, hesitation, adaption or perhaps sheer frustration(!). Whilst taking part in each task, participants will be encouraged to self-narrate their actions and responses using a think-aloud approach. Think-aloud, alongside other verbalization methods are well established in qualitative and usability research as a means of accessing tacit knowledge and decision making processes as they occur, rather than relying exclusively on retrospective accounts (Ericsson and Simon, 1993; Nielsen, 1993).
5. Transcripts and Ethical Ai Usage
With participant consent, audio recordings of these narrated task performances will be created and then transcribed using AI-assisted transcription tools. The use of AI in this case is pragmatic rather than epistemic: automated transcription supports efficiency and accessibility and particularly under the time constraints of this project. I have no intention to use the tools here as a replacement for human interpretation however: transcripts will be reviewed, corrected and annotated by hand to ensure accuracy and full contextual understanding. I do also intend to use AI tools to assist with the initial organisation and extraction of reoccurring themes, such as points of friction, expressions of confidence or uncertainty and references to accessibility. However ultimately all analytic decision making will remain the responsibility of the researcher.
6. Post-Task Questionnaire
The narrated task data will be embellished by a short post-activity questionnaire (created in Microsoft Forms) that invites participants to reflect on usability, accessibility, confidence and perceived impact on studio practices. Ultimately I intend to utilise a combination of real-time narration, reflective feedback and observational notes, to triangulate my chosen methodologies and strengthen the credibility of my findings. The use of AI tools in this process is overseen by the commitments I have made in my Ethical Action Plan and aligns with guidance on transparency, data protection and researcher responsibility in educational research (BERA, 2024).
7. Form Links
Below are links to the Info Pack, Consent Form and Post-Task Feedback Questionnaire – all of which are to be shared with participants of the pilot. All are Microsoft OneDrive Links.
8. References
BERA (2024) Ethical Guidelines for Educational Research. 5th edn. London: British Educational Research Association.
Ericsson, K.A. and Simon, H.A. (1993) Protocol Analysis: Verbal Reports as Data. Rev. edn. Cambridge, MA: MIT Press.
McNiff, J. (2013) Action Research: Principles and Practice. 3rd edn. London: Routledge.
Nielsen, J. (1993) Usability Engineering. San Diego: Academic Press.
Leave a Reply