Background Vectors by Vecteezy
As I have moved forward with this project, it has become clear that the Voice Learning Assistant (VLA) is more complex than I first imagined. Though it can be planned and mapped to a certain extent, there are many areas that are difficult to judge in terms of time allocation. There is also a more involved set of iterations with the testing phase to ensure the final system meets with the initial objectives. In this post I will look at the more difficult aspects of planning this project, where the uncertainties lie, how I plan to measure success and where secondary data will be useful.
Planning Difficulties
The initial phases of research, design and build flow in a fairly linear way and time allocation here are more straightforward. However, once the prototype is ready, an iterative cycle of testing, feedback and refinement begins for alpha and beta testing; and to some extent pilot testing. This stage will likely expose issues and ideas that are not within the original design and so will need careful management to ensure the project remains on track.
Furthermore, the collection of both quantitative and qualitative data concurrently means that I will need to arrange interviews and observations alongside the system testing. In principle this is straightforward, however, as I consider the ethical approvals required, timetabling for students and teachers and ensuring system is stable for use during the sessions it becomes clear that there are many factors to keep organised at once.
The interplay between the disciplines will also introduce a potential for broad changes. It is foreseeable that limitations in the technical deployment will affect the pedagogy, or that feedback from users warrants a change in the prompting. The interplay between the disciplines requires my plan to remain flexible enough to deal with these revisions whilst remaining structured enough to ensure the timely completion of the project (Wollen, 2025).
What does Success Look Like?
Wingate's approach to success requires looking across multiple dimensions in this context to ensure the objectives have been met, rather than simply asking "does it work?". In the context of the VLA, technical success will certainly be important as a non-functional system cannot be tested by anyone, but there are further objectives that must also be assessed.
An central objective for me is whether the system actually improves the learning in terms of knowledge recall, understanding and application. Assessing whether the system actually identifies and resolves misconceptions, helps them to verbally more scientifically accurate response, and improves their confidence will by just as important as the technical success.
Linked to the above, the user experience will form another objective for measuring success. All to often in education the "best method" identified by cognitive psychology for students to memorise new information leads to tasks that are repetitive and don't engage learners. Since the system will require students to voluntarily engage for maximum efficacy, their experience and enjoyment are also of paramount importance.
There is also a measure of success linked to the project itself. Have I planned the project well? Was it executed as expected? When unexpected results arose, were they handled? Was the project delivered on time?
Finally, there is a measure of success that will come from new insights. The system may be perfect and there may be further work to consider, but if I can reveal something new about the human-computer interaction at the heart of this project then that would also be a success, and so warrants being identified as an objective (Wingate, 2025, Chapter 6).
Secondary Data
Secondary data is implicit within this project as the planning phase is built upon the work of others. I have already used findings from research into artificial intelligence, prompt engineering, and efficient system design along with instructional methods, mastery framework design, cognitive load theory and student motivation to name a few. I plan to use many more and will need to be certain of the credibility of the studies I draw from the construct the final design of the system. I plan to use peer-reviewed papers and academic literature for this purpose, and use practitioner findings to add context rather than drive decision making.
Usability is also important with secondary data being relevant to this context. There are many studies on the teaching, learning and tutoring of various subjects, each of which will add further context and depth to this project but will be unlikely to fit with the VLA use-case specifically. With this in mind, I will use secondary data sources for guidance on design choices and the anticipate common misconceptions and pedagogical mistakes (St-andrews.ac.uk, 2018).
Final Thought
I have realised that this project will exist in the space between rigid structure and complete flexibility. I will plan the project in detail as discussed in previous posts, but ensure there is room for responsive iteration and changes whilst using milestones to ensure the project remains on track for the final completion date. My success criteria will be objective driven to ensure the system meets the needs of the students and supports their learning rather than simply "working". I also recognice the importance of collecting data for analysis as well as using secondary data from the wider research community to use as guidance, reference, and to support decision making.
Thanks for reading and please check back for the next post.