AustraliaEducation2024 – ongoing

Macquarie University

Virtual Peer student chatbot

Macquarie University chatbot pilot: students who used it scored 9.45% higher on the exam

WorkingVendor's case studyVendor ↗

Problem

Students wanted help with coursework and administrative questions outside office hours. The university needed answers that stay tied to approved course material.

Approach

The university built Virtual Peer with Microsoft and the partner Celebal Technologies on Azure OpenAI Service. The bot answers only from a curated base of lecture slides, transcripts and policies, and links each answer to its source. A two-week pilot ran with 1,400 second-year psychology students before exams.

Results

9.45%higher average exam score for students who used the bot in the two weeks before the exam, after controlling for past results (university pilot)Vendor's case study[1]

84%share of administrative answers tutors rated equal to or better than human answersVendor's case study[1]

20,800 messagesmessages sent in the two-week pilot, 80% of them outside university operating hoursVendor's case study[1]

Takeaway

Tutors rated 84% of the chatbot's administrative answers equal to or better than human answers. Gaps in the university's own documentation caused most of the weak answers.

Sources

Macquarie University students' exam scores up by nearly 10 per cent thanks to new AI-powered chatbot
Microsoft Source Asia24 Mar 2025Vendor publication

Source summaries

[1]Microsoft Source Asia24 Mar 2025

Microsoft describes how Macquarie University built a chatbot that answers from approved course material. It reports the pilot's exam-score gain, usage and tutor ratings.

Full details are in the original sources.

Added 25 Oct 2026 · Updated 25 Oct 2026