Chat with us, powered by LiveChat

Measuring and Promoting Improvement 2019-12-02T11:26:19+00:00

Measuring and Promoting Improvement

The bksb Assessment Engine uses complex algorithms based on probability and “best fit” calculations to produce highly accurate measurements.

These can be extremely precise and identify not only a level but also where a student sits in terms of the distance to the next level.  For example, a student could be regarded as E3.1 (a low Entry 3) through to E3.5 (midway between Entry 3 and Level 1) and on to E3.9 (almost Level 1).

The bksb Functional Skills measurements run from Pre-Entry through to a point above Level 2, with the table below showing scores ranging from a low Entry 3 to a high Level 1.

Progress vs Improvement

Most computer-based learning systems for Functional Skills (and other courses) have tended to track “progress” by giving students tasks to undertake and then monitoring whether they have been completed.  Unfortunately, the term “completed” is subjective as merely accessing a resource can at times render it “complete”.   Another problem with “progress” tracking is that it is merely a checklist and tells us nothing about whether the tasks that have been prescribed have improved the student’s ability.  Most knowledge testing in a resource is incomplete, hence the data thereby provided is limited in use.  As a result, a better approach would be to track “improvement” rather than “progress”, as that is the real metric that we are interested in.

To create a reliable system for tracking improvement, there first needs to be accurate and precise measurement which allows for the detection of even small advances.  For example, a student could be measured as being E3.2 (fairly low Entry 3), and then as E3.4 (almost halfway), and then again as E3.7 (well on the way to Level 1).  Whilst the student has stayed at Entry 3, it is nonetheless clear that he or she is improving.  The data also suggests that it will not be long before the student is Level 1, and there is a good chance that they will achieve this upon the next measurement.  Without this level of precision, however, the measurement data would simply show results of Entry 3, Entry 3 and Entry 3, and it would be impossible to draw any conclusions from this.

Thankfully, the new bksb course structures make full use of the ability to measure accurately and detect incremental advances.  Using these measurements, the system will take an improvement in a module such as “Calculations” and apply it to both its parent element “Number” and its parent subject “Maths”.  As students then work through resources and take assessments, the platform will use these changes to create estimates as to what their overall ability for a subject might be.  For example, a “Subject” such as Maths can be broken down into “Elements” such as “Number”, “Measure, Shape and Space” and “Handling Data”:

Elements are subsequently broken down into “Modules” such as “Calculations” or “Measure”.  Using the “improvement tracking”, it is then possible to take a detected improvement in a module and estimate a new ability level for the related element.  Likewise, this improvement to an element can be applied to the overall subject.  Therefore, as a student completes resources and takes progress check assessments, the system is constantly applying these results across the entire subject to produce estimates of overall ability.

Promoting Improvement

As the bksb system no longer places students on a single level, they are allowed to access content at all levels, with the data obtained from the Assessment Engine ensuring that this is done in a sensible and effective way. For example, where a student is identified as working at L1.1 (a low Level 1) they will be given access to both Level 1 and Level 2 content for that module, but the platform will recommend that they study at Level 1. When the same student improves to L1.5 (midway between Level 1 and Level 2), however, the recommendation will change to Level 2.  The system recommends content at the “closest” level, thereby introducing higher content when the student is ready for it. As the journey from one level to another is managed, students are “stretched and challenged” in a sensible and realistic way.

With the new course structure allowing for a more dynamic and realistic profile (students are not restricted to a single level and are likely to have different ability scores for different modules), it is likely that students will be recommended content at different levels for different modules.  This allows for weaknesses to be addressed properly with a realistic path towards the desired level, whilst also ensuring that strengths are improved and that students are given the opportunity to be challenged.

The end result is a system which is far better at developing students and managing their learning, whilst also providing more useful data for tutors in terms of a likely destination and the progress being made.

You can find out more about bksb systems and products on our product page below.

bksb products