A comprehensive rubric for instructional design in e-learning

Literature review: slightly different paths for e-learning

By the year 2002, the fusion between in-class instruction and online instruction was already being recognised as a major new trend; old practices were being transferred online, and different institutions had rather different understandings of the concept of e-learning (Frydenberg, 2002; Graham et al., 2013; Young, 2002). As late as 2011, Guri-Rosenblit and Gros (2011) concluded there were “noticeable gaps” in e-learning research and definition of terminology. One of the reasons for this lack of coherence is:

The technological environment within which modern education operates is becoming increasingly complex, offering new possibilities but also giving rise to challenges. We have seen a continual evolution of technologies and how they are used since the introduction of the Internet

(Conole and Alevizou, 2010, p. 9).

De Freitas and Conole (2010) observed the trend of more global, more networked and more mobile technology infrastructure and these are emerging in online pedagogy. Indeed, McLoughlin and Lee (2010) argued that this new landscape:

Calls for the active involvement of students in defining their learning goals and choosing both ICT tools and strategies for learning; it also requires recognition that user and learner generated content has a central place in a curriculum that fosters self-regulated learning

(p. 38).

Learning in the digital age thus requires a re-thinking of teaching and learning and not just replicating existing practices with new technology (Beetham and Sharpe, 2013).

While in the early years of public internet the emphasis was on the technical foundations of e-learning, the pedagogical implications started shifting direction from managing the logistics of e-learning to managing the content (Govindasamy, 2002). An “interactive and constructive potential of e-learning” was being recognised, one which contrasts with the traditional sage on the stage approach of information transfer. However, it had to prove it was more than just a more convenient way to access content (Garrison and Anderson, 2011, p. 54).

The focus of course design and pedagogy has shifted from teaching to learning, with teachers also becoming learners in the process of professional development and engagement with their students. Rather than transferring the passive teacher-centred form of learning to the online domain, a constructivist approach would make the experience learner centred, where active learning and engagement takes place (Rovai, 2004). The emphasis in higher education has shifted from the delivery of instruction to the production of learning (Barr and Tagg, 1995).

Far from simply providing a more comfortable channel to access instructional content, Nagel and Kotzé (2010) found, “When students engage in online activities and take responsibility for the quality of interaction, they can have a superior learning experience” (p. 218). ICT can indeed be successfully applied to enhance the educational process, especially in making learners active participants (Tomte and Sutherland Olsen, 2014). On the other hand, the digital literacy of academic staff remains a challenge (Johnson et al., 2015), with many feeling the pressure of 24/7 online connectivity in the age of social media (Grove, 2017).

E-learning faces issues of suspicion (Casey, 2008) and quality (Jung and Latchem, 2012) when compared to traditional instruction. However, a meta-analysis by Means et al. (2013) on behalf of the US Department of Education reveals a significant increase in performance for blended learning but not for pure online learning. This confirmed earlier conclusions by Pankin et al. (2012).

In their literature review on the use of Web 2.0 technologies in HE, Conole and Alevizou (2010) observed a number of key challenges proposed when technology meets education, including the tensions around the nature of openness, and changes in the role of educators and students. HE in Europe seems to be responding positively to these rapid developments in e-learning, though the adoption of e-learning and MOOCs is gradual (Gaebel, 2015).

Empirical evidence is increasing in respect to online learning and new developments such as open educational resources (OER), mobile learning, software agents for online evaluation (Daradoumis et al., 2013), bring your own device, the flipped classroom, wearable technologies, adaptive learning technologies and the Internet of Things (Johnson et al., 2015). These have attracted the attention of researchers to an ever-higher degree in recent years. However, more research on OER and MOOCs is required (Bozkurt et al., 2015).

The NMC Horizon Report 2017 (HE Edition) refers to a medium-term trend of using digital tools to measure knowledge and skills acquisition in online learning environments, including collaboration and creativity (Adams Becker et al., 2017).

According to the Times Higher Education Teaching Survey 2017, half of academics and 68 per cent of administrators agree that students benefit from digitised content, but they evidence less enthusiasm for recording lectures and putting them online. The use of social media for instructor-student contact is still not widespread as many academics feel the pressure of constantly being in demand by students (Grove, 2017).

Crossbreeding between social network sites and e-learning is increasingly observable. Though Facebook is not a replacement for VLEs in education, the latter “could certainly learn something from Facebook and the nature of its user created groups and networks, instant communications, alerts and like/sharing features” (Hogg, 2013, p. 35). These VLEs, such as MOODLE and Blackboard, had to introduce features similar to the increasingly popular social network sites (Ronan, 2015), while some social network sites such as LinkedIn are assuming educational roles (Hoffman, 2015). This development and the proliferation of MOOCs are some of the most interesting but controversial trends in higher education right now (Johnson et al., 2014). Early research shows that MOOCs are indeed contributing to online learning, though more empirical research is needed (Gamage et al., 2015; Glance et al., 2013).

An overview of four different rubrics

A synthesis of the rubrics to formulate the new comprehensive rubric

The comprehensive rubric

The synthesis of the four rubrics just referenced has produced the following comprehensive rubric that covers all the aspects mentioned by the four institutions. This comprehensive rubric is not a collation of the four rubrics but a synthesis of the separate approaches that – in the author’s view – reflects the context of e-learning as explained at the beginning of this paper. This rubric has not been tested in lab setting or a real-life scenario.

There are ten main standards, each containing specific standards.

Ten main standards and specific standards are as follows:

1. Instructional design – an analysis of the learning needs and the use of appropriate strategies and methods to meet them:

   • Structure of learning.

   • Learning aims and objectives – what the instructor needs to achieve with the learning process.

   • Learning outcomes – what learners need to achieve to have successfully completed the learning process.

   • Instructional strategies and methods.

2. Course opening – welcoming learners:

   • Accessibility – the instructor gives clear instructions on how to access all elements of the online learning environment.

   • Role – the instructor gives clear information about his professional role in the learning environment.

   • Description – a course description with pre-requisites (if any), clear learning outcomes and what is expected of the learners is provided.

   • Behaviour – the learners are made aware of regulations, policies and ethics than govern the course.

   • Integrity – the instructor is aware of the academic integrity needed to facilitate learning.

   • Technical competences – the learners are made aware of the technical competences needed to successfully reach the learning outcomes.

   • Ownership – the instructor gives learners the opportunity to share their own learning goals.

3. Assessment of learning – determining what the learner has learnt and subsequent accreditation:

   • Goals and objectives – the learners are aware of what is expected of them when they are assessed.

   • Strategies – clear, well-defined and measurable assessment of learning outcomes suited to the level of the learners.

   • Grading – grades are given in a fair and transparent manner through appropriate assessment instruments sanctioned by the institution.

   • Feedback – both instructor and learners are given the opportunity to provide feedback related to grading.

   • Management – learners have access to their grades and feedback at all times so that they can track their learning progress.

4. Interaction and community – the exchanges between instructor and learners that build a community that supports teaching and learning:

   • Fostering – the instructor welcomes learners and gives them the opportunity to communicate and create an online environment that fosters peer learning and engagement.

   • Management – community building is supported by clear instructions, rules and regulations. While the instructor facilitates engagement, learners are invested with the ownership of community building.

   • Peer learning – group work and other activities that foster peer learning are encouraged and structured not only to fulfil the learning outcomes, but also to present learners with an opportunity to learn skills and competences that go beyond such outcomes, e.g., digital literacy.

5. Instructional resources for teaching and learning:

   • Provision – learning materials are either provided by the instructor or the learners are given enough time to procure such resources. The difference between compulsory and optional resources is to be made clear.

   • Application – the instructor clearly explains how the resources are going to be applied and utilised.

   • Entitlement – the instructor makes sure that the resources indicated to fulfil the learning outcomes are open and accessible by all the learners without unwarranted technical, financial or administrative barriers. The use of OER should be encouraged.

   • Variety – learning resources are varied in terms of the multimedia content and multi-modal delivery channels to cater for the different learning preferences of learners.

   • Openness – the instructor should give learners the opportunity to suggest their own resources for adoption in the course.

   • Academic integrity – the instructor promotes best practice in the use of third party resources, including anti-plagiarism practices and sound academic research/writing practices. The use and/or adherence to the creative commons licensing framework is encouraged.

6. Learner support – learners enabled to achieve their maximum potential:

   • Instructional support – the instructor explains his/her role in the process.

   • Academic support – learners know how to obtain such services as mentoring, advice and other skills that support them in achieving the learning outcomes.

   • Technical support – learners know how to obtain technical support to overcome potential issues in accessing the learning area and achieving the learning outcomes.

   • Administrative support – learners know how to obtain administrative support to overcome potential issues in accessing the learning area and achieving the learning outcomes.

7. Technology design – technology is at the service of teaching and learning:

   • Support – all the utilised technologies and resources support the achievement of the aims and objectives of the instructor and the learning outcomes for learners.

   • Centricity – all technologies and resources used support a learner-centric rather than an instructor-centric educational approach. The learners must be in control and technology must assist them in achieving the learning outcomes.

   • Openness – the technical infrastructure used to deliver the teaching and learning is procured and implemented according to open standards and formats that maximise the value for money and the range of options to fulfil the learning outcomes and the academic needs of faculty and learners.

   • Authentication – authentication at different levels (device, software, virtual learning environment, specific course/learning area) should provide access to a safe and secure teaching and learning environment with the minimum number of steps possible to access the learning areas.

   • Access – the virtual learning environment/learning area is device/platform agnostic as much as possible, thus accessible over different software platforms, browsers and computing devices. The instructor provides alternative resources if any of these are not easily accessible for technical reasons related to special needs of learners.

   • Interface – the user interface and navigation in the learning area is simple enough to be conducive to teaching and learning without the need to possess advanced ICT skills and competences.

   • Investment – the technical requirements of the instructional resources and the virtual learning environment/learning space do not require learners to make any significant new investment in hardware, software and online services to be able to access and use these resources to fulfil the learning outcomes.

   • Management – learners are aware of the rules, regulations and policies at institutional and at learning community level that govern the use of the technological infrastructure supporting e-learning.

8. Course evaluation – feedback to improve teaching and learning:

   • Entitlement – instructors should give learners the opportunity to provide feedback on the whole learning experience. On the other hand, instructors should also be able to provide their feedback within their organisation.

9. Course closing:

   • Assessment – learners should have access to their grades and the course material after the closure of the course (depending on the institution’s access policies). The final grades should be provided within a reasonable timeframe after the closure of the course.

   • Resolution – all pending issues between the instructor and the learners are resolved.

   • Archiving – the instructor makes sure the course/learning area resources, texts, communication, etc., are backed-up or archived (in line with the institution’s access policies) in a safe and secure way.

10. Instructional design cycle:

    • Academic review – the instructor and the organisation review the course description, the experience gathered, and the evaluation given.

    • Technical review – the instructor, with the relevant technical unit in the organisation, reviews the performance of the technical infrastructure used to deliver teaching and learning.

    • Administrative review – the instructor, with the relevant administrative unit/s in the organisation, reviews the administrative processes supporting the delivery of teaching and learning.

This comprehensive rubric is being published under the following Creative Commons Licence: Attribution ShareAlike 4.0 International (CC BY-SA 4.0). This grant of the licence does not claim to cover the four rubrics individually.

Recommended actions in support of instructional design for e-learning in higher education

The following recommendations are compiled from the literature review and the development of the comprehensive rubric:

1. Teaching and learning should dictate the technological implementation of supporting tools and facilities, not the other way round.

2. An educational institution wishing to provide e-learning opportunities needs a clear vision and a strategy sustaining e-learning from an academic, technological and administrative point of view.

3. The use of e-learning pedagogies should be promoted as an official part of the professional work of academics, with tangible incentives and rewards that can take different forms, including professional recognition, financial and material rewards.

4. An educational institution wishing to implement e-learning needs to invest in academic training and support, in technical support and a sound technical infrastructure, and in administrative support.

5. An educational institution should take time in getting to know the views and needs of its faculty and students.

6. A VLE should not be utilised as a simple online repository of content or for document management.

7. E-learning should attempt to benefit from the affordances provided by technology as guided by the digital pedagogies and experience with e-learning, especially providing students with an element of control over the pace, time and path of study. Otherwise, it simply serves as an extension of traditional teaching practices.

8. Student-instructor and student-student interaction, collaboration and work through the VLE (community building).

9. A VLE alone does not provide a complete range of tools to facilitate online learning; therefore, complementary tools such as social media should be sought.

10. E-learning, with MOOCs for example, is an opportunity to explore micro-credentialing and accreditation of online learning.

Discussion

The analysis of the rubrics has confirmed the myriad of possibilities provided by technology when applied to teaching and learning. Indeed, the resulting comprehensive rubric is rather long, and its components may indeed contribute to the welcome conclusion of the long-standing debate on whether e-learning is as rigorous and effective as traditional face-to-face environments (Casey, 2008; Jung and Latchem, 2012). Such a rubric will surely support faculty in the ever-increasing implementation of e-learning (Gaebel, 2015).

The active involvement of students in the learning process, rendered possible by technology, is well catered for in the comprehensive rubric. The early emphasis of technology in e-learning has given way to more credence in the pedagogical benefits (Garrison and Anderson, 2011; McLoughlin and Lee, 2010; Rovai, 2004).

The lack of common definitions of e-learning and its constituents is notable in the chosen rubrics, but there is nevertheless a common approach: empowering the educator to empower the student in an online environment that promotes learning.

Digital literacy of faculty and time pressures remains a challenge and even though technology is available, the application of the elements listed in the comprehensive rubric requires a level of digital competence from instructors (Conole and Alevizou, 2010; Grove, 2017; Johnson et al., 2015).

MOOCS, OER, BOYD, social elements, artificial intelligence, augmented and virtual reality are acquiring more space and attention in education, and the comprehensive rubric must take into consideration these new instructional approaches and updated digital pedagogies (De Freitas and Conole, 2010).

The tensions created by the implementation of technology in HE, especially the changing role of educators and students brought by more social, more ubiquitous and more open learning spaces, will surely bring to light any gaps between the planned and the actual implementation of e-learning. It is up to the educator, as a professional, to mind these gaps and bridge them.

Limits of scope

It is not within the scope of this paper to review quality assurance processes and administrative components, but to propose a rubric for course design and self-review of faculty and HE institutions for a better alignment with what is regarded as current standard best practice.

Suggestions for further research

The proposed comprehensive rubric does not provide a scale for assigning points when applied, thus giving a weighting to the elements of the rubric perceived as more important than others. There are other rubrics on e-learning by reputable HE institutions that could be included in the analysis. The litmus test for the rubric is its application in real-life situations. This is an excellent opportunity for follow-up research analysing the outcomes of its application.