Learning Analytics Roadmap: An ROI On Dropout Detective™ [LAR Series #3]

The “Fosway Analytics Maturity Model” frames our reviews of Moodle analytics plugins. The official Moodle discussion about the dimensions of analytics uses this model as well. The layout of this model is a 6-step ladder towards “Maturity”, along 3 stages. The features of a plugin places it at one of the steps because it offers all the features of the previous ones. Fosway suggests progress towards Maturity implies a deeper involvement of (Bigger) Data.

**** Follow our series on The Learning Analytics Roadmap here ****

Operational Analytics. These are tools to keep track of things.

1. Passive reporting: They involve basic ratios and averages. They provide a dashboard with all relevant metrics. Individual student profile information is available. Moodle offers passive reporting out of the box.

2. Proactive reporting: Adds tracking, calculates benchmarks and KPIs (Key Performance Indicators) to offer comparisons.

3. Silo’d analytics: They provide domain-specific insight by going beyond performance ratios. They illustrate the ways specific knowledge acquisition (say chemistry, or cryptography) consolidates.

Strategic Analytics. Tools to influence decision making.

4. Integrated Analytics: They expand beyond the learning operation of an organization. It tracks and correlates metrics across its supporting and strategic business units. Think learning-centered Balanced Scorecard.

5. Predictive Analytics: They add forecasting algorithms, such as regression analysis. They can offer dynamic future scenarios that update with the most recent data available.

Transformational Analytics. Tools that automate the generation of intelligence, foster innovation and create new knowledge-based markets.

6. Machine Intelligence Automation: They adapt processes autonomously. An ongoing, massive algorithmic effort that perfects learning methodologies on individual basis.

The more optimistic among us see the Fosway framework as a starway to heaven. But on the opposite side, a voice of reason seems to be asking: How much will the future cost for my organization? And will it be worth it?  The Fosway framework does not seem to have these answers for us.

AspirEDU’s Dropout Detective, cost and benefit.

Dropout Detective produces a Risk Index Score for students. It examines not only grades on submissions but also fulfillment on deadlines. As an institutional product, it can take a look at a student’s history and their current courses. It weights grades by its per-course amount. Developers Kim Munzo and Juan Leyva claim the algorithm is self-correcting, “to a degree”. They liken it to a credit score.

By its definition, Dropout Detective goes in the Predictive Analytics step. It caters to institutional administrators more than teachers, as it deals with school retention. It includes dashboards for administrators, advisors and instructors, that update on a daily basis.

Dropout Detective has two versions available for download. One for Moodle 3.0, one for Moodle 2.4 to 2.9. Check out GitHub for installation instructions.

Use of Dropout Detective requires an AspirEDU account. Information on pricing is available for Dropout Detective by request only. Fee structures of similar offerings usually include software licensing, professional parametrization and post-sale service.

Dropout Detective illuminates a critical meeting point between an institution philosophy and its finances. According to AspirEDU, dropout rates correlate with institutional image and tuition. When dropout rates threaten financial viability, Dropout Detective should be only the first of many remedies.

On Learning ROIs

There is something you should not expect Dropout Detective to offer, and that is how to prevent dropouts in the first place.

Once you have a Risk Index Score for your students and an amount of students at high enough risk, it is time for new policies. No plugin will give you the straightforward answers you need. Not on Moodle, or any LMS.

It does not mean there are no supporting tools to make decisions. Not only about dropouts. Not only for present, but for all future menaces to your organizational strategy. At the latest MoodleMoot US, the Chief Learning Officer of Lynda.com, Britt Andreatta, mentioned one. It is the ROI on Learning Pyramid.

Developed by the ROI institute, it is an interesting view of learning interventions towards a final effect on performance. First, it exhibits the current level of “analytics literacy” of an organization. Second, it illustrates the commitment of a management team on learning as an institutional practice.

Andreatta raises a waking voice about a widespread practice that is noxious for all learning initiatives looking to improve performance. That is, to measure the success of an intervention only from subjective responses. Opinion and satisfaction polls are often hold as the best and only measuring sticks. In reality, these are important metrics, but from the ROI on learning framework, they are literally the bottom of the food chain.

After all, learning implies effort, and we usually don’t enjoy making an effort. We might enjoy their outcomes, which are seldom immediate. The lesson from the ROI on learning is that targeting performance takes a long term view, because quantitative evidence takes time to produce meaning. Efforts to raise awareness about the value of analytics, organization-wide, are equally demanding in patience.

Conclusions

Dropout Detective is an ambitious solution that applies smart algorithms to the specific problem of dropout rates. It is rather an outlier in the portfolio of learning analytics plugins for Moodle.

It is understandable that most of the learning analytics solutions in the market today belong to the “Operational” stage, with an increasing share in the “Strategic” stage. We might have not yet reached a point of convergence between technology and our understanding of learning to provide more sophisticated solutions. But we also do not seem to be far from knowledge-based products that give us smart, personalized learning experiences.

Analytics is a language. Creating an “analytics-literate” culture takes long term efforts. The ROI on Learning pyramid allows management to visualize how far they have come, and how far there is left to go for learning interventions to affect performance and innovation.

We’d love for you to share your thoughts in the comments below!

Corrections made on September 3rd, 2016.