Report HEIs

ByGiuliano Alviani
image_pdfimage_print

BizMOOC Report 1.3b


Identification of needs and gaps for the application of MOOCs by Higher Education Institutions

BizMOOC – BizMOOC – Knowledge Alliance to enable a European-wide exploitation of the potential of MOOCs for the world of business
Programme: Erasmus+ | Key Action 2 | Knowledge Alliances
Reference Number: 562286-EPP- 1-2015- 1-AT- EPPKA2-KA
Grant agreement number: 2015-2929 / 001-001
Project Duration: 36 months, 1/1/2016 – 31/12/2018

Identification of needs and gaps for the application of MOOCs by Higher Education Institutions

Authors: Oana M. Driha & Maryna Makeienko (University of Alicante), Christian Friedl (FH JOANNEUM), Darco Jansen (EADTU)

Report based on interviews carried out by all BizMOOC partners: FH JOANNEUM Graz (AT), Open University (UK), University de Alicante (ES), Burgas Free University (BG), University of Economics Krakow (PL), AVL List GmbH (AT), iversity GmbH (DE), Hasso-Plattner-Institut (HPI) für Softwaresystemtechnik GmbH (DE), DIDA srl (IT), Košice IT Valley (SK), The National Unions of Students in Europe (BE), EADTU (NL)

Abbreviations

BizMOOC – Knowledge Alliance to enable a European-wide exploitation of the potential of MOOCs for the world of business
CPDContinuous professional development
HE – Higher Education
HEI(s) – Higher Education Institution(s)
ICT – Information and Communication Technology
IT – Information Technology
MOOC(s) – Massive Open Online Course(s)
NGOs – Non-governmental Organisations
SPOC – Small Private Online Courses

Foreword

Since the beginning of the 20th century, the world has evolved at a much faster pace and in a more complex way as a consequence of globalisation. Rapid economic growth, increasing competitiveness and innovation were influenced by evolving information and communication technologies (hereinafter ICT). This impacted on all sectors, including higher education (hereinafter HE).

HE has undergone continuing change over the past two decades. This is due to six main factors:

  1. Internationalisation of HE and student mobility;
  2. An ever-growing demand for quality higher education and lifelong learning;
  3. Changing student demographics;
  4. The increasing use and demand of online and blended learning,
  5. Cross-border higher education;
  6. Recognition and quality assurance of qualifications in a digital world without borders.

(Patru & Balaji, 2016).

Given that around half of the global population (3.5 billion people) are regular users of the Internet, largely due to the expansion of mobile networks and decreasing prices, online content and interaction with a global audience have experienced a remarkable consolidation (ITU, 2016). The advances of Internet use have considerably contributed in changing the living standards of many people, their communication channels and their working habits and tools (Patru & Balaji, 2016). As a consequence, Internet use is also transforming how we govern and work.

Since the extensive media coverage of Massive Open Online Courses (MOOCs) from 2012 onwards, and the development and use of MOOCs worldwide, it is of note that Europe was slower than North America to use and implement this method of online learning (Gaebel, 2013). In the fall of 2013, the European Union (EU) launched the action plan Opening up Education (European Commission, 2013). This action plan is a joint concerted effort and integrated approach of DG Connect and DG EAC and focuses on innovative teaching and learning for all through ICT, contributing to the modernising EU education through OER, digital competencies, infrastructures, interoperability, equity, quality, visibility, licensing, certification, etc.

MOOCs in this respect are considered an instrument for opening up education, and as such many MOOC projects are funded by the EU over the last few years (for example HOME, ECO, EMMA, BizMOOC, TraMOOC). Regardless of the delay in Europe in using MOOCs for training / teaching / learning, Jansen & Schuwer (2015) highlighted that Higher Education Institutions (hereinafter HEIs) in the EU are quickly making sizeable gains in adapting MOOCs. Whereas in the United States of America (hereinafter USA) the number of HEIs, which are not interested in launching MOOCs, is increasing (33 per cent in 2013 and more than 58 per cent in 2015), an increasing number of HEIs have MOOCs or plan to develop or use them in the near future in the EU (Rutkauskiene et al., 2015).

Likewise, there are clear differences between Eastern and Western Europe. On one hand, Western European countries are the ones most engaged in EU MOOC initiatives (Open Education Europa, 2014), especially France (Muñoz et al., 2016), the United Kingdom, Spain (Open education Europa, 2014), Italy and Portugal (Jansen & Goes, 2016). On the other hand, the majority of Eastern European countries are new players as they have just joined this development (Dillenbourg, 2013). However, an increasing uptake and interest in MOOCs have been highlighted in countries such as the Czech Republic (Rohíková et al., 2015) or Lithuania (Rutkauskiene et al., 2015) recently.

Mulder & Jansen (2015) explored whether MOOCs can be instrumental in opening access to education. They concluded that MOOCs and their providers would not or probably cannot remove some barriers easily, but that MOOCs still can be seen as instrumental to widen access to higher education for millions of people and as such can contribute significantly to open education. Even if all over the world the open education movement is becoming a reality, further engagement and growth is needed. Developing countries have been historically a prime target of this movement, but active participation by the Global South is required in order for the open education movement to be relevant to developing country needs (UNESCO, 2002).

Access to the Internet alone is not sufficient anymore. It is necessary to find ways of making quality online learning materials more accessible and usable, particularly for less developed countries (Nti, 2015). In other words, policy-makers must fight against socio-economic inequalities whilst promoting the Internet as providing the opportunity to acquire the skills, knowledge and competences required by today’s changing world of work (ITU, 2016).

Different strategies and guidelines were proposed for higher education stakeholders (UNESCO & Commonwealth of Learning, 2015) to promote the use and production of OER facilitating the way towards open education as a key part of the Universal Declaration of Human Rights (Nti, 2015).

The Guidelines for Open Educational Resources (OER) in Higher Education recommend different strategies and approaches for the five higher education stakeholders: government(s), HEIs, academic staff, student bodies and quality assurance/accreditation bodies and academic recognition bodies. In less developed economies, some of these guidelines are being adapted and implemented, thus efforts are being made to encourage the active participation in the production and delivery of OER content (Diallo, Thuo, & Wright, 2013; Donkor, 2011).

Additionally, it is worth mentioning that among these recommendations for supporting OER contents, higher education stakeholders might consider them for developing similar strategies for MOOC contents.

1. Executive summary

Higher Education Institutions (hereinafter HEIs) play a central role in the educational system, especially in formal education, but also, particularly recently, enlarging its role in non-formal / informal education. With the explosive entry of MOOCs into the educational arena, HEIs are embracing MOOCs as an important format in both formal and informal education, the latter to their mission of sharing recent knowledge and to continuous profession development.

The BizMOOC project has considered the potential core role of HEIs in the MOOC area and conducted a study into HEI’s perspectives of their role in MOOC production and use. The BizMOOC consortium interviewed 50 different HEIs with low or no MOOC experience in Europe as part of this study.

The main findings are as follows:

  • The majority of HEIs interviewed are familiar with the MOOC concept and agree with the definition facilitated during the interviews, but only 20 per cent of them are involved in offering/producing non-traditional learning offers (similar to MOOCs).
  • Among the benefits of MOOCs for students and non-traditional, eLearning and lifelong learners are the possibility of refreshing and deepening one’s understanding of a topic via expert tuition in addition to the freedom and flexibility of self-paced learning;
  • Despite high levels of eLearning development among the HEIs interviewed, less than half are developing online training specifically tailored for the business community;
  • Recognition procedures, a low level of language skills and quality assurance were the main perceived barriers to using MOOCs;
  • The main challenge to creating MOOCs was perceived to be production costs, but that could be mitigated via HEIs collaborating with businesses and other organisations;
  • An involvement of local, national and European policy-makers is needed especially due to the high potential of MOOCs to contribute to improving the education system;
  • The need to develop more work-orientated training could be addressed through MOOCs in order to meet the needs of industry regarding skills, knowledge and competences.

2. Introduction

Despite the fact that the MOOC movement is still a relatively recent phenomenon, emerging in 2008 and beginning its rapid expansion around 2011, most MOOCs are developed by the HE sector. In Europe, a large percentage (over 40 per cent) of HEIs seems to have developed a MOOC or is planning to develop a MOOC (Jansen & Goes, 2016). There is no doubt that European HEIs joined the MOOC movement later, but contrary to the recent developments in the USA, Western European HEIs are now engaging more with MOOC development and production (Jansen & Schuwer, 2015; Jansen & Goes, 2016). This is also occurring in some Eastern European countries (Rutkauskiene et al., 2015; Rohíková et al., 2015).

Frequently mentioned motives for developing MOOCs, as given by HEIs, are: boosting student recruitment; creating flexible learning opportunities (for new students); increasing institutional visibility and reputation; using MOOCs as areas of innovation (e.g., to improve the quality of on-campus offerings, contribute to the transition to more flexible and online education, improve teaching); responding to the demands of learners and societies (Jansen, Schuwer, Teixeira & Aydin, 2015). In addition, MOOCs are perceived as a tool enabling students to obtain their educational credentials more rapidly in addition to being a more flexible educational provision and a way of lowering the cost of education (Walton, Coningham & Horn, 2015). MOOCs are therefore an important global trend for both producers and students.

However, this does not automatically imply a better access to the higher education system. Without any formal credits for MOOC completion, MOOCs are just sharing knowledge. Moreover, access to higher education system requires in addition that those credits do count as part of a formal degree. This is hindered by the lack of adequate legislation for regulating open and online education, both by HEI and by national accreditation organisations (ongoing EADTU study 2017). In addition, the novelty of the methodology and low or non-existent experiences with MOOC business models have led to HEIs being hesitant to incorporate MOOCs as part of formal degrees. Gradually we see a shift from producing MOOCs based on regular courses to a mode where open education offering is becoming a part of formal degrees (for example with Micromasters & Nanodegrees).

ICT, and in particular web based technologies, has an increasing impact on our personal and professional lives over the last decades (Dumitrescu, 2015). MOOCs, as part of ICT, were able to become a hot topic in education through the combination of ICT, faculty presentations, subjects and the funding of initial courses (Billington & Fronmueller, 2013). An increased democratization and efficiency of instruction, as well as the transformation of teaching and learning processes are some of the consequences of using MOOCs within an educational context (Dumitrescu, 2015). Consequently, MOOCs are considered by some to be potentially the disruptive innovation of higher education (Hardesty, 2013). Even if some researchers stipulate that MOOCs are a challenge for some HEIs (Smutz, 2013) – as not every HEI can cope with the development of MOOCs, as there are no sustainable business models, quality assurance issues etc. –, others argue that MOOCs are just another ed-tech bubble that will not transform education (Crispin, 2012). However, even if the technologies and tools might change, the impact of MOOCs on higher education will last.

The Internet is employed for a vast amount of activities, from communication, entertainment, e-commerce and e-services, to learning and information research, for example. According to the latest Information Society report by ITU (2016), a significantly higher proportion of developing countries use the Internet for “education and other learning activities” (see Table 1). Thus, developing economies seem to be a large but under-explored market for open education and MOOC production, development and use, particularly as Sustainable Development Goals focus on education (see also Patru & Balaji, 2016).


Table 1. Proportion of developed and developing countries in which a particular activity is the top Internet activity or among top 3 or top 5 activities (latest data 2010-2015). Source: Source: ITU (2016).

The BizMOOC project is trying to identify reasons why some of the target groups involved (businesses/organisations, universities and society/labour force) are not engaging with MOOC production, design or use.

Despite the high level of interest in MOOCs in the Global North, they remain a relatively underused and understood tool for many organisations, education institutions and parts of society. Even if ICT plays a great role in our lives, particularly in relation to knowledge acquisition, the low or (in some cases) zero penetration of HEIs from Eastern European countries in the MOOC movement is so far surprising. In our study, Eastern Europe includes all European countries (developed or developing economies according to the International Monetary Fund) which entered the European Union (hereinafter EU) since the enlargement of 2004 onwards as well as countries which are not yet a member of the EU.

The aim of this BizMOOC study is therefore to identify user and potential user of MOOC perceptions of this type of online learning and subsequently identify gaps, needs, challenges and opportunities for collaboration between HEIs and businesses. The following report presents our findings, which are based on in-depth interviews with representatives of HEIs with low or no experience of MOOCs.

3. Methodology and sample

According to Marshall & Rossman (1989), qualitative research “entails immersion in the everyday life of the setting chosen for study, that values participants´ perspectives on their worlds and seeks to discover those perspectives, that views inquiry as an interactive process between the researcher and the participants, and that is primarily descriptive and relies on people´s words as the primary data” (p. 11). In line with Bryman (1989) and Wai-chung Yeung (1995), qualitative methodology implies the following aspects when it is used in organisational research:

  1. Adopting an insider´s stance to the organisation;
  2. A strong sense of contextualization;
  3. An emphasis on process;
  4. Unstructured approach with little prior theoretical orientation and no presupposed hypotheses;
  5. A variety of data sources such as field notes, interview transcripts and documents;
  6. Ability to obtain and retain close proximity to the phenomenon under study.

Moreover, as can be seen in Table 2, despite the prevalence of quantitative research in business, qualitative methodologies play a complementary role.


Table 2. Comparison of quantitative and qualitative. Source: Wai-chung Yeung (1995) based on Bryman (1988).

Qualitative interviews are a potentially powerful means of exploring the intricacies of different sectors and subsectors (Broom, 2005). This method offers a different means of exploring the subjective and complex experience of business communities and organisations in general as well as the specific decision making processes associated with both strategic planning and everyday activities.

In order to determine the reasons why the HE community is (geographically) only partially involved in the MOOC movement, an in-depth semi-structured set of interview questions was developed. This semi-structured format was chosen due to its flexibility, which enables an open dialogue to take place.

Following existing literature in the field of qualitative business research, and in line with previous MOOC research (see for example Jansen & Goes, 2016; Mulder & Jansen, 2015) a four-part interview guideline was developed. This comprised of five lead questions and 17 follow-up questions focused on the following aspects:

  1. A short introduction to the concept of Massive Open Online Courses (MOOCs) and participant perceptions:
    • Familiarity with the concept based on the definition facilitated to the interviewees;
    • Any added value associated to MOOCs;
    • Expectations;
    • Benefits;
  1. Organisation involvement and interest in online courses and MOOCs.
    • Identification of the main activities of the organisation connected to online training & MOOCs;
    • Identification of the availability, interest and motivation to use MOOCs.
  1. Determining factors for (and/or levels of interest in) collaborating with other organisations to produce and design MOOCs
    • Identification of the main reasons the organisation would/would not collaborate with others on MOOCs;
    • Identification of the topics and services around which the organisation is willing to collaborate with others.
  1. Existing regional or national infrastructure focused on open education
    • Identification of the existence or opportunities for a regional or national educational infrastructure on MOOCs, or open education in general as MOOCs have become an instrument for educational policy.

50 interviews with HEI representatives were conducted by eleven different BizMOOC project partner organisations. Interviewees were selected from HEIs with hardly no previous experiences of using or developing MOOC and according to the following criteria:

  • The HEI has not offered a MOOC to date prior of interview (however, the analysis revealed that 20 per cent were already involved in MOOC production);
  • Eastern1 and Western Europe are both represented through the selection of HEIs. However, the study also recognised the need to capture information regarding Eastern European experiences and therefore put an emphasis on interviewing Eastern European HEIs (60 per cent).
  • The HEI is not specialised and covers a diverse range of disciplines, including the applied sciences, medicine, technical studies and the humanities.
  • The HEI representative to be interviewed is senior and has some connection with eLearning.

It turned out, the persons nominated as representatives of their HEI for the interviews were mainly occupying an executive/management position whether vice-chancellors, deans, vice-deans, representatives of the eLearning or distance learning department or similar.

As can be seen in Table 3, representatives from 23 European countries were interviewed: 60 per cent of these were based in Eastern Europe and 40 per cent in Western Europe.


Table 3. Summary of Research Design (Eastern Europe includes all European countries (developed or developing economies according to the International Monetary Fund) which entered the European Union since the enlargement of 2004 onwards as well as countries which are not yet a member of the union). Source: BizMOOC.

For the purpose of statistical analysis five different categories were devised from the interview data and questions:

  1. Awareness and perception of MOOCs;
  2. Involvement in online and MOOC activities;
  3. Expectations of MOOCs, reasons for engaging with (or otherwise), benefits and barriers;
  4. Opportunities for, and interest in, collaborating with other institutions to create MOOCs;
  5. Existing infrastructures for promoting MOOCs at a regional and national level.

4. Results

4.1. Awareness of Massive Open Online Courses (MOOCs)

The first step in this research was determining interviewee familiarity with the concept of MOOCs. Table 4 gives an overview of the region where interviewees are based and their familiarity with the concept of MOOCs.


Table 4. Interviewee familiarity with MOOCs. BizMOOC based on in-depth interviews.

90 per cent of the interviewees were already familiar with MOOCs. The ones unfamiliar with the concept were provided with additional explanations. A total of 96.67 per cent of the HEIs located in Eastern European countries and Cyprus are familiar with the concept of MOOCs. Western European HEIs show a similar trend. However, even in instances where HEIs are familiar with the concept of MOOCs, only 20 per cent of these offer any online learning (similar to MOOCs). Around 70 per cent of interviewees in both East and Western Europe reported that they were not offering or producing any type of online learning of this type.

4.1.1. Definition of MOOCs

Based on the definition developed by different European projects (OpenupEd, 2014) and validated through a variety of surveys by Jansen & Schuwer (2015), the BizMOOC project offered the following definition to interviewees: “MOOCs are courses designed for large numbers of participants, that can be accessed by anyone anywhere as long as they have an internet connection; they are open to everyone without entry qualifications, and offer a full/complete course experience online for free”.

90 per cent of HEI representatives agreed with the definition facilitated (see Table 5).


Table 5. Do you agree with the definition of MOOCs offered during the interview? Source: BizMOOC based on in-depth interviews.

Although 90 per cent of interviewees agreed with the definition offered, participants offered the following observations and potential amendments:

  • MOOC are not necessarily for “free” anymore, particularly as additional services such as certificates of completion are increasingly charged for;
  • The characteristic “open to anyone regardless of previous experience or academic qualifications” does not seem applicable to the majority of existing MOOCs. This, as one interviewee described it, is because “some MOOCs would make it difficult for learners with a non-traditional background to engage or even feel welcome.” MOOC are therefore open enrolment, but the way in which they are structured or presented may not be as “open” as other formats. Moreover, introductory courses or even testing the level of prior knowledge of potential participants may help people find suitable MOOC more effectively. This issue was of particular concern to interviewees.
  • There were doubts from interviewees that a MOOC could offer “a full/complete course experience” as this depends on a number of factors, including completion rates. According to the interviewee, prior knowledge could play a determinant role as it could conduct to the successful completion of the course, if the learners enrolled in MOOCs also have the necessary motivation in this sense. Nevertheless, collecting valuable eLearning experience is not necessarily connected to the full completion of a course (see also the discussion following below).
  • The “massive” aspect of MOOCs was viewed as unsuitable for accommodating diverse and specific learner needs. This has led to the development of individualised courses known as Small Private Online Courses (SPOC).
  • Intellectual Property Rights and personal data protection are hot topics in this kind of courses due to its “online for free” characteristic (outside formal educational system) while offered by third parties (MOOC platform and other service providers like proctoring, tutoring etc.)

In addition, some interviewees noted that the definition offered did not take into account that:

  • MOOCs are facilitated over a specific time period;
  • There is no support for learners;
  • Some MOOCs are certified or participation recognised within certain contexts.

For the authors of this report, the above comments made by interviews are related to

  1. misconception of the differences between a face-to-face, a blended or online course and a MOOC;
  2. those offering online courses stating that they are offering a MOOC and/or that their online course is open as well (so contributing to these misconceptions).

For example

  • Note that the cited definition also gives criteria to when the offer is a complete course. Recognition and certificates are indeed part of the criteria of a full/complete course experience (OpenupEd, 2015, p. 2). Most MOOC providers still offer a complete course for free including a certificate of completion. However, some additional services need to be paid for (including credits to be recognised as part of degree program) as part of freemium business models.
  • Completion rates are not the same as offering a complete course. The authors of this report suggest to approach this the other way round: Because anyone can follow a MOOC (without entry requirements), many will fail, because they lack the prior knowledge, motivation or other objectives before enrolling. Therefore, we do not agree that completion rates could determine a full course experience. And indeed, completion rates of MOOCs are most of the time low, because of its open enrolment (but also not incorporating target group characteristics in the design, not communicating the level of prior knowledge needed etc.)
  • There is a difference between entry requirements (certain age, completed secondary school and/or have a certain language/math skill) and prior knowledge. Most MOOC providers do not pose any entry requirements, so anyone can start the MOOC. However, to be successful, some prior knowledge is recommended.
  • The criteria as part of the MOOC definition (OpenupEd, 2014) includes the possibility to have start and end dates whenever the participant wants. OpenupEd was the first one that already included these self-paced MOOCs. It took until 2015 for the main MOOC providers to widely starting to offer them as well (although most of them are offered within a specific time-frame) (Class Central, 2015).
  • MOOCs do offer support to learners, but not on a personal level as in (some) on-campus HEIs. Support is part of the criteria of a complete course offering. Guidance for some parts is integrated in the design of the course by integrated feedback on questions and quizzes, but also by other means (like open universities did in the pre-internet era). Next, fora exist for support by peers including monitoring by academics. But the tutoring is indeed limited by the definition of MOOCs by its (potential) massive audience: “The (pedagogical model of the) course is such that the efforts of all services (including of academic staff on tutoring, tests etc.) does not increase significantly as the number of participants increases” (OpenupEd, 2014). However, recently, the big MOOC providers are tailoring their MOOC offering to business (employees) – introducing other abbreviations like SPOC (Small Private Online Courses).

4.1.2. Expectations

This research question examined interviewee expectation of different types of courses (face-to-face, non-MOOC online courses and MOOCs) and if MOOCs in particular were perceived as adding value to learning and teaching. The first issues tackled were general expectations from online courses including MOOCs. Responses have been summarised in Table 6.


Table 6. Expectations from online courses and MOOCs. Source: BizMOOC based on in-depth interviews.

One interviewee noted: “A MOOC should drive traffic to your own institution’s site; it should act as a marketing tool to drive traffic towards you. It should be able to increase your recruitment around particular subject areas; it should raise the profile of your university, and really it should be showcasing the best of what your institution can offer”.

Moreover, interviewees representing the HEI community included in our sample noted that MOOCs could:

  • Act as a low-cost version of high-quality courses. Learners cannot always afford to attend courses/training due to their cost and/or other associated costs (such as travel to/from the class). A MOOC may offer the opportunity to participate at a lower cost than attending face-to-face, even when taking into account the cost of course completion certificates.
  • MOOC material could be used to both inform and accredit work outputs such as reports. For example, if a MOOC was used to acquire up-to-date information on a specific topic, which was then incorporated into a report, this report could be graded and accredited.
  • Enable learners to work on a topic with which they are unfamiliar or on a new topic without course entry barriers.
  • Be useful for non-accredited personal and professional development, lifelong learning, adult learning, learning new languages etc. One of the main potential strengths of MOOCs is their applicability for continuous professional development (CPD) or professional networking purposes (i.e. for colleagues to keep abreast of new developments in their field, or to look at topics that they are interested in, but haven’t had time to investigate in depth).
  • Provide easier access to new knowledge and new research findings on an international scale.
  • Provide a taste of new subject areas for those wishing to explore a change in career.
  • Be suitable for disabled persons, for people located in rural and inner city areas, and for those who have a very busy schedule, mainly because of the autonomy and scalability of this type of courses.
  • Be suitable for employees and specialists to update their knowledge and keep up to date with changes to everyday working practices;
  • Open up knowledge and teaching. MOOCs were viewed by some interviewees as a great way of improving knowledge and gaining new skills on specific topics.
  • Be a good way to acquire knowledge and certification from a range of HEIs.
  • Be a great tool for not only improving specific skills also developing soft skills, digital literacy and other competencies. All of these skills are important to employers.

From an institutional perspective, the following additional features were expected from MOOCs:

  • Could serve as a research tool in order to both gather research data and understand effective MOOC development and delivery;
  • Could serve as a marketing tool if the HEI is trying to enlarge its reach or target new potential learners. MOOC are often created by or involve well-known professors, but they can also be used to showcase other academic and staff expertise.
  • Due to globalisation, education is increasingly international and corporate. Sensitivity to trends, new developments and being seen to innovate are important within this context.
  • Applicable at any educational level.
  • In some cases, MOOCs could compliment the formal education system. Sometimes it is considered that it is vital for students to have basic knowledge in a subject before commencing formal study. MOOCs could be used for this purpose.
  • There is a clear need to complement vocational training with HE; MOOCs could work perfectly in this context.
  • Could facilitate blended learning when taking the role of additional tutoring, as a kind of lexicon if something is unclear or just for acquiring general additional information (if the learner would like to deepen the content).
  • Could serve for updating existing curricula or for being added as vocational classes to a formal degree.

4.1.3. Added value

MOOCs are considered to add value not just for learners, but also for the HEIs involved. Therefore, the next questions asked participants to elaborate on the added values they expect. The responses were categorized and are presented in Figure 1.

One of the most attractive aspects of MOOCs is their internationalism, both in terms of content and geography (learners and course facilitators can come from all over the world). MOOCs enable anyone, regardless of educational attainment, to participate. In addition, MOOCs are a good marketing tool: providing an experience for learners that may influence their decision to enrol at a particular institution. Unsurprisingly interviewees viewed the quality of MOOCs material as important. The “open” aspect of MOOCs was also highly valued by those interviewed; although in that respect, it is not clear, because – although MOOC are open enrolment – they are not always open content-wise as material used is often copyrighted.


Figure 1. Added values of MOOCs in comparison to traditional online courses. Source: BizMOOC based on in-depth interviews.

To summarise, the openness of MOOCs is rated as the most relevant added value by 26 per cent of the interviewees, followed by “internationalisation”, “marketing tool”, “high quality” and “for free” which were also noted as relevant by more than 20 per cent of the interviewees.

Our research shows that MOOCs are associated with a large number of perceived and actual advantages by HEIs. In addition, MOOCs were also associated with some disadvantages, as Figure 2 shows. As can be seen, the main disadvantage highlighted by interviewees (36 per cent) was the perceived very low interaction level with the instructor/teacher/provider of the MOOC. Other reasons varied and saw 10 per cent or less of interviewees refer to them.


Figure 2. Limitations to and potential disadvantages of MOOCs . Source: BizMOOC based on in-depth interviews (Bad (student) time management refers to student time management. According to the interviews, the responsibility of the instructors/ teachers/producer of MOOCs seems to be rather high (as “high expectations from instructors/teachers” in the figure 2)).

4.1.4. Benefits of MOOCs

Five HEIs that participated in this study were not familiar with MOOCs. These HEIs did therefore not respond to the question regarding actual, rather than perceived, benefits of MOOCs. All remaining 16 Western European HEIs and 29 Eastern European HEIs considered the use of MOOCs to have clear benefits for students (90 per cent). In comparison, 74 per cent of this sub-sample is convinced that MOOCs are also of benefit to non-traditional, online and lifelong learning students (see Table 7).


Table 7. Benefits of using MOOCs for students, non-traditional students, eLearning and lifelong learning. Source: BizMOOC based on in-depth interviews.

Among the benefits of using MOOCs for traditional students (A traditional (undergraduate) student is a student “who enrols in college immediately after graduation from high school, pursues college studies on a continuous full-time basis at least during the fall and spring semesters, and completes a bachelor’s degree program in four or five years at the young age of 22 or 23. Traditional students are also typically financially dependent on others, do not have children, consider their college career to be their primary responsibility, and are employed only on a part-time basis if at all during the academic year.” (Centre of Institutional Effectiveness, 2004)) (in our understanding a full-time, enrolled student with secondary/high school diploma and studying with a clear objective, e.g. to get a job), the interviewees mentioned that MOOCs:

  • Could help to deepen and catch up with contents of on-campus courses;
  • Are perceived different from the didactical view and facilitate other “learning channels”; in this respect, MOOCs could also be an addition to on-campus courses – they are taught face-to-face which could be less appropriate to some learning types (e.g. geographical and time inflexibility) and MOOCs could hence constitute an additional support;
  • Offer flexible learning which can fit around a learner´s other responsibilities;
  • Bring together a range of topics and information;
  • Have the potential to enable virtual collaboration and the sharing of diverse perspectives and experiences between learners;
  • Give students the opportunity to update their skills and access additional information to complement any formal studies;
  • Offer an opportunity to develop ICT skills, soft skills, critical thinking, collaborative working etc.
  • Contemporary way of learning;
  • Gives access to high quality HE content.

4.1.5. Barriers to the use of MOOCs

16 out of 50 HEIs interviewed considered that formal recognition of participation and/or learning was needed for online courses such as MOOCs. The authors of this report assume that the existence of online badges and other forms of MOOC recognition were not known to those interviewees. Still, as MOOCs are not recognised by the formal accreditation system and education is not standardised across Europe, it is unclear as to how, and whether MOOC could fit with qualifications such as Bachelor or Master´s degrees or high school diplomas/GCSEs, etc. Most interviewees thought that some kind of separate accreditation was adequate and it should be appealing to a broad range of learners. In addition to certification, online badges (such as those developed by Mozilla) were also considered important by some interviewees. Language was considered a potential barrier to the use of MOOCs, particularly as many are English language only.

As expected, perceived production costs are one of the largest barriers for HEIs engaging with MOOC. In addition, the time needed to develop MOOCs content potentially places further demands on HEIs and the workload of their staff.

4.2. Skills alignment

Online training for the business community is not often part of European HEI activity. 56 per cent of the interviewed HEIs told us they were not currently involved in this kind of activity (see Table 8). Nevertheless, 92 per cent of our sample reported eLearning activity at their institution.


Table 8. Online training for the business community and eLearning involvement of HEIs. Source: BizMOOC based on in-depth interviews.

Among the group of institutions who do offer online training for businesses, the allocation Western/Eastern Europe within our sample is almost equal. However, within the Eastern European subsample, the majority of institutions (60 per cent) are not involved in online training for the business community.

Although 46 HEIs are involved in some way in eLearning (e.g. providing courses), this varies considerably from 5 per cent to 100 per cent. Of particular note is that eLearning is used by a higher percentage of staff in Eastern European HEIs than their counterparts in Western Europe.

Among the advantages of online learning, interviewees noted that:

  • It allows you to fit your learning around your work and home life;
  • You can usually set your own pace of study;
  • It is your decision as to when and where you study;
  • It doesn’t matter where you live – you can gain a degree from anywhere in the world;
  • As with a traditional (face-to-face) degrees, students may find that they gain useful, transferable skills, such as planning and research;
  • It often costs less than a full-time degree.

Given all these advantages of online courses, it was interesting to note the current level of online course use by the HEIs interviewed as part of our research:

  • To compliment face-to-face learning (a blended learning model);
  • Produced in conjunction with businesses and focused on the following areas: management skills, ICT skills, educational entrepreneurship, e-commerce, banking and economics skills, other business skills related to CPD, Social Media, Legislation and Freedom of Expression, languages (for non-native speakers) etc.;
  • As an alternative to in-house training;
  • A way of collaborating with businesses as part of Master and/or PhD programmes.

Furthermore, HEIs are involved in offering/producing training especially in areas of engineering, technology and IT, banking, management consulting, e-commerce, educational entrepreneurship, business skills, project management, Business English, business communication, negotiation skills, qualification courses etc. (see Figure 3).


Figure 3. Subject areas where interviewed HEIs are offering/producing online training (% of interviewed HEIs)

4.3. Reasons not to be involved in the use/production of MOOCs

Despite the fact that a large number of HEIs around the world are offering/producing online training, numerous challenges remain. According to the BizMOOC interviewees, the main challenges for HEIs are the perceived high cost of producing online courses and the costs related to hosting or joining a MOOC platform (see Table 9). HEIs need to invest money and support staff and as MOOCs are perceived as “free”, no direct income is expected by HEI to cover these investments.


Table 9. Main challenges faced: online training. Source: BizMOOC based on in-depth interviews.

It is worth noting that in some countries (such as Hungary), education is free (at the point of use) and therefore charging for MOOCs and online courses is prohibited. Different funding models would therefore need to be adopted, depending on geographical location and applicable legislation.

4.4. Opportunities for collaboration

Prior to discussing potential collaboration between business communities and HEIs, we asked interviewees about the improvements the online learning might need in their opinion. One interviewee suggested that following each iteration of an online course, material should be refreshed and updated. Another interviewee suggested that more activities should be included in online material. Quality could be improved by including more insights into business practices. Live streaming sessions and activities, live quality assurance and recordings to watch on demand were all viewed as essential by interviewees. Some interviewees thought there was a need for OER repositories whilst others noted the need to ensure that learner feedback was gathered and used to inform future iterations of a course.

In addition, the following observations were made by interviewees in relation to collaborating with the business community on MOOC development:

  • The possibility of creating MOOCs for business appears a very good opportunity for demonstrating and promoting university-business collaboration, although differences in working cultures and practice would need to be negotiated/addressed;
  • It is likely that HEIs would need to support and advise businesses on pedagogy;
  • The interest in using, providing and developing MOOCs in collaboration was mentioned by more than 30 per cent of the sample. The design (forms and templates), the marketing methods of providing the courses and identifying the target groups, the acquisition of MOOC content and experience exchange in legal rights when offering and/or compensating would be also interesting fields of future collaboration.
  • By producing MOOCs, a better picture to future students coming from abroad could be facilitated according to one of the interviewees.
  • The University of The Third Age might be also interested in using MOOCs. This way, initiation in basic and not so basic IT skills beforehand could be entailed, for example.
  • MOOCs were perceived as suitable for use in the following areas: in-service training, special needs, remedial training, experimental learning, continuous engineering education, open online education opportunities for educators, re-training academic staff, bookkeepers, faculty administrators and whoever is involved in day-to-day university business.
  • MOOCs were considered to be of use to those contemplating HE level study;
  • Local language MOOCs were considered to be important by interviewees;
  • 78 per cent of HEIs in our study were interested in collaborating with the business community to develop MOOCs. Half of these HEIs are based in Eastern Europe. A range of company types were suggested: IT companies, multinationals, call centres support services, medical companies, cultural heritage presentation, preservation and advertising, banks, training and consultancy companies, employment offices, NGOs, sport clubs/organisations, or other HEIs.

The perceived costs of producing MOOCs were noted as the main reason for collaboration.

It was deemed essential to develop course material relevant to the needs and experiences of the labour market and interviewees considered partnership with business to be a plus point in this respect. Working in partnership with business was also considered to be important in strengthening ties between the HEI and business communities.

4.5. MOOC infrastructure and tools for promotion at the regional and national level

Policy makers should consider the role and potential of MOOCs in the education system, even if some researchers are questioning the impact of MOOCs on the future of education (Majhanovich, 2015) as some others like Iqbal, Zang, Zhu, Chen & Zhao (2014) are predicting that effects of MOOCs on education could be visible by 2022 (at least in engineering education). The design of educational policy is of special relevance in general, but it seems to be even more important for MOOCs (Patru & Balaji, 2016), because of its explosive evolution in less than a decade and its potential in affecting the evolution of educational system.

MOOCs are one way in which quality education could be made available to large numbers of learners, particularly in developing countries, in addition to potentially reducing the cost of formal HE.

As stated in a report by the Norwegian government (NOU, 2014), the main reasons for involving governments in the design and development of MOOCs are to:

  • Promote the expansion of innovative education;
  • Boost and motivate the use of new technologies and new ways of learning.
  • Further lifelong learning through expansive, open, free and accessible learning opportunities;
  • Encourage blended education by combining online and campus/face-to-face learning;
  • Broaden access to (high-quality) educational content;
  • Contribute to marketing and branding of HE offer in addition to increasing the potential for diversifying HE revenue sources;
  • Promoting the improvement of digital skills;
  • Facilitating networks enlargement;
  • Support recycling knowledge;
  • Use MOOCs as an admission tool;
  • Facilitate the possibility of (potential/future students) testing beforehand the HE products/services offered, when considering a university study.

The potential of MOOCs to facilitate job-orientated training in order to meet the needs of industry (skills, knowledge and competences) is increasing. However, national strategies to promote the role of MOOCs in offering a potential solution to unemployment and/or refining professional development are absent (Garrido et al., 2016). Equality of access to education should therefore be at the centre of any policy related to reducing unemployment (Patru & Balaji, 2016).

Furthermore, the design and delivery of the right skills for the labour market rests on the shoulders of the businesses, educational providers and the learners´ community. Thus, to offer the required 21st century skills, representatives from industry, the educational sector and government must work together.

At the time of writing, five countries in Europe have some kind of policy related to MOOCs. In the Czech Republic, there is a Digital Education National Strategy, which references MOOCs. In Poland, there are some grants to develop MOOCs offered by the National Centre for Research and Development and the Erasmus program. In Greece, a bottom up approach has been initiated where citizens request specific courses at a University and the Ministry of Education discusses these requests. In the Netherlands, the Ministry of Education Culture and Science has published an agenda “HE2025, the value(s) of knowing” with two aims: that by 2025 all HE educators will openly license and share their teaching materials and that all HE institutions will recognise each other’s MOOCs (Ministerie van Onderwijs, Cultuur en Wetenschap, 2015). An investment program running from 2015-2019 makes one million euro available each year for projects furthering open online education in HE (Ministerie van Onderwijs, Cultuur en Wetenschap, 2014). At the time of writing, the program is operational for two years. Every year, HEIs can apply for funding for their open/online activities and there is a 5 years research project analysing all these funded projects called “Sooner” (Results availble on the project website: http://sooner.nu/about-the-project/ ) with some results already available.

In France, the national MOOC platform is supported by the French government in close cooperation with HEIs. In 2016, the legal structure has been changed from the governmental lead to a more collaborative association supported by the government. Other non-MOOC examples related to open course regulation can be found in Finland, for example. There, the universities have an open university department which is providing open courses for a low fee regulated by the Finish government. OpeneningUpSlovenia is another governmental policy initiative implemented in Slovenia.

While in the USA the legislation regarding MOOCs was designed and seemed to have a good feedback (like in the case of open licensing), the majority of European countries do not however have a clear policy/strategy for supporting MOOCs despite the involvement of an increasing number of higher education stakeholders and even companies and organisations. Given the evolution experienced in the last years, this should be considered and tackled.

One approach could be the uptake and adaption of good practices where policy initiatives have worked, such as the before-mentioned US open licensing model. The final version of the MOOC BOOK (www.mooc-book.eu; Version 2.0 to be published in October 2018.) developed in the course of the BizMOOC project is going to feature selected good practice examples.

5. Conclusion

European HEIs have responded to MOOC in a variety of ways and MOOC uptake and development is not consistent across Europe, particularly when use/development between East and West is considered. However, both Eastern and Western Europe increasingly consider MOOCs as a good potential way of training staff and teaching students (both formal and informal).

The majority of HEIs in the BizMOOC sample are familiar with the MOOC concept and agree with the definition facilitated during the interviews, but only 20 per cent of the sample are involved in offering/producing non-traditional learning offerings (such as non-MOOC online courses).

Clear benefits for students and non-traditional, eLearning and lifelong learners were also noted by interviewees. In particular, the opportunity to access up-to-date information, broaden networks and study at a time suited to learner´s own commitments were highlighted.

Over 90 per cent of the HEIs interviewed are involved in eLearning, with less than half currently developing online training tailored for the business community. Current training largely focuses on business, management and economics, consultancy, engineering, technology and IT, and areas where the legislation obliges companies to carry out training sessions for their employees.

Interviewees highlighted a number of barriers and challenges including low or no recognition, low levels of English language skills and quality assurance. However, MOOC production costs seem to be the main perceived challenge for the majority of interviewees. Collaboration between HEIs and business was viewed as one possible solution.

The importance of the role of the government in responding effectively to economic challenges and workforce changes was also noted. Skills must be kept up-to-date and personnel must be willing to engage with new technologies and innovations as part of their everyday working lives. MOOC have a lot to offer in this respect and their strategic increased use/development has the potential to contribute considerably to furthering education policies. Similarly, MOOCs can also be used to meet the needs of industry. As suggested previously, collaboration between various sectors is necessary for this joined up approach to be effective.

Note

    Acknowledgments

    This research is conducted as part of the European Union-funded project BizMOOC – Knowledge Alliance to enable a European-wide exploitation of the potential of MOOCs for the world of business (Ref. 562286-EPP-1-2015-1-AT-EPPKA2-KA). We would like to thank all partners in BizMOOC for their contributions, especially Beck Pitt from the Open University UK for proof-reading and critically reviewing. However, sole responsibility for this article lies with the authors, and the Commission and BizMOOC partners are not responsible for any use that may be made of the information contained therein.

    References

    Billington, P. J., Fronmueller, M. P. (2013). MOOCs and the future of higher education. Journal of Higher Education Theory and Practice, 13(3), 36-42. Retrieved from http://www.na-businesspress.com/JHETP/BillingtonPJ_Web13_3__4_.pdf

    Broom, A. (2005). Using qualitative interviews in CAM research: A guide to study design, data collection and data analysis. Complementary Therapies in Medicine, 13: 65-73.

    Bryman, A. (1988). Quantity and Quality in Social Research. Unwin Hyman, London.

    Bryman, A. (1989). Research Methods and Organization Studies. Unwin Hyman, London.

    Centre of Institutional Effectiveness (2004). A Fresh Look at Traditional and Non-traditional Undergraduates at KSU. Retrieved from http://eimirvic.kennesaw.edu/AnalyticStudies/Students_2004_Traditional_Nontraditional_Undergraduates.pdf

    Christensen, G., Steinmetz, A., Alcorn, B., Bennett, A., Wood, D., Emanuel, E.J. (2013). The MOOC phenomenon: Who takes massive open online courses and why? Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2350964

    Class Central (2015). MOOC Trends in 2015: Rise of Self Paced Courses. How Have MOOC Platforms and MOOC Takers Alike Reacted To Self Paced? Retrieved from https://www.class-central.com/report/mooc-trends-2015-rise-self-paced-courses/

    Crispin, W. (2012) MOOCs and other ed-tech bubbles. Edtechnow.net. Retrieved from http://edtechnow.net/2012/12/29/moocs-and-other-ed-tech-bubbles/

    Diallo, B., Thuo, C.W., Wright, C.R. (2013). Approaches to the production and use of OER: The african virtual university experience. In McGreal, R., Kinuthia, W., Marshall, S. (Eds.). Open educational resources: Innovation, research and practice (pp. 91-103). Vancouver, BC: Commonwealth of Learning and Athabasca University.

    Donkor, P. (2011). Developing and using open educational resources at KNUST. Educational Technology Debate. Retrieved from http://edutechdebate.org/digitallearning-resources/

    Dumitrescu, V.M. (2015). The Present-Day Trend Towards Massification In Higher Education: The Mooc Model. The 11th International Scientific Conference eLearning and software for Education Bucharest, April 23-24, 10.12753/2066-026X-15-110.

    European Commission (2013a). Opening up education: Innovative teaching and learning for all through new technologies and open educational resources. Brussels, Belgium. Retrieved from http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52013DC0654&from=EN

    Garrido, M., Koepke, L., Andersen, S., Mena, A., Macapagal, M., Dalvit, L. (2016). An examination of MOOC usage for professional workforce development outcomes in Colombia, the Philippines, & South Africa. Seattle: Technology & Social Change Group, University of Washington Information School.

    Grossman, R. (2013). Are Massive Online Courses in Your Future? Organizational and Employee Development Special Report, SHRM, August 1.

    Hardesty, L. (2013). Higher-ed leaders meet to discuss future of online education. MIT News Office. Retrieved from http://web.mit.edu/newsoffice/2013/edx-summit-0306.html

    Hill, P. (2013, June 4). MOOCs beyond professional development: Coursera´s big announcement in context (Blog post). Retrieved from http://mfeldstein.com/moocs-beyond-professional-development-courses-big-announcement-in-context/

    Iqbal, S., Zang, X., Zhu, Y., Chen, Y.Y., Zhao, J. (2014). On the impact of moocs on engineering education, in MOOC, Innovation and Technology in Education (MITE), 2014 IEEE International Conference on. IEEE, pp. 101–104. Retrieved from https://www.researchgate.net/profile/Sajid_Iqbal5/publication/269854019_On_the_Impact_of_MOOCs_on_Engineering_Education/links/5497f13e0cf20f487d319788.pdf

    ITU (2016). Measuring the Information Society Report 2016. International Telecommunication Union. Place des Nations, Geneva.

    Jansen, D., Schuwer, R. (2015). Institutional MOOC strategies in Europe. Status report based on a mapping survey conducted in October – December 2014. EADTU. Retrieved from http://www.eadtu.eu/documents/Publications/OEenM/Institutional_MOOC_strategies_in_Europe.pdf

    Jansen, D., Schuwer, R., Teixeira, A., Aydin, H. (2015). Comparing MOOC adoption strategies in Europe: Results from the HOME project survey. International Review of Research in Open and Distributed Learning, 16(6). doi:10.19173/irrodl.v16i6.2154

    Jansen, D., Goes-Daniels, M. (2016) Comparing Institutional MOOC strategies. Status report based on a mapping survey conducted in October – December 2015. EADTU – HOME project. Retrieved from http://eadtu.eu/images/publicaties/Comparing_Institutional_MOOC_strategies.pdf

    OpenupEd (2015). Definition Massive Open Online Courses (MOOCs). Retrieved from http://www.openuped.eu/images/docs/Definition_Massive_Open_Online_Courses.pdf

    Marshall, C., Rossman, G.B. (1989). Designing qualitative research. Newburry Park, CA: Sage.

    Majhanovich, S. (2015). Neo-liberalism takes hold: Educational reform in the brave new faculty of education. Educational Practice and Theory, 37(2): 5–23.

    Ministerie van Onderwijs, Cultuur en Wetenschap (2014). De voorzitter van de Tweede Kamer der Staten-Generaal Postbus 20018 2500 EA..DEN HAAG. Retrieved from https://www.rijksoverheid.nl/documenten/kamerstukken/2014/01/08/kamerbrief-over-digitalisering-van-het-hoger-onderwijs

    Ministerie van Onderwijs, Cultuur en Wetenschap (2015). Strategische Agenda Hoger Onderwijs en Onderzoek 2015-2025. Retrieved from https://www.rijksoverheid.nl/documenten/beleidsnota-s/2015/07/07/de-waarde-n-van-weten-strategische-agenda-hoger-onderwijs-en-onderzoek-2015-2025

    Mulder, F., Jansen. D. (2015). MOOCs for opening up education and the OpenupEd initiative. In C. J. Bonk, M. M. Lee, T. C. Reeves, T. H. Reynolds (Eds.), MOOCs and open education around the world. New York, NY: Routledge. Retrieved from http://www.eadtu.eu/documents/Publications/OEenM/OpenupEd_-_MOOCs_for_opening_up_education.pdf

    NOU (Official Norwegian Reports) (2014). MOOCS for Norway: New digital learning methods in higher education. Retrieved from https://www.regjeringen.no/contentassets/ff86edace9874505a3381b5daf6848e6/en-gb/pdfs/nou201420140005000en_pdfs.pdf

    Nti, K. (2015). Supporting Access to Open Online Courses for Learners of Developing Countries, International Review of Research in Open and Distributed Learning, 16(4): 156-171.

    Patru, M., Balaji, V. (2016). Making Sense of MOOCs. A guide for Policy-Makers. United Nations Educational, Scientific and Cultural Organization (UNESCO), Fontenoy, 75352 Paris 07 SP, France. Retrieved from http://unesdoc.unesco.org/images/0024/002451/245122E.pdf

    Radford, Walton, A., Coningham, B., Horn, L., (2015). MOOCs: Not just for college students – How organizations can use MOOCs for Professional Development.” Employment Relations Today 41(4): 1-15.

    Rohlíková, L., Rohlík, O., Jansen, D., Goes-Daniels, M. (2016). Comparing Institutional MOOC strategies – 2015 Czech Republic report. EADTU. Retrieved from http://home.eadtu.eu/news/112-country-reports-on-uptake-of-moocs-by-heis

    Rutkauskiene, D., Gudoniene, D. Jansen, D., Goes-Daniels, M. (2016). Comparing Institutional MOOC strategies – 2015 Country report Lithuania. EADTU. Retrieved from http://home.eadtu.eu/news/112-country-reports-on-uptake-of-moocs-by-heis

    Smutz, W. (2013). MOOCs Are No Education Panacea, but Here’s What can Make Them Work. Forbes. Retrieved from http://www.forbes.com/sites/forbesleadershipforum/2013/04/08/moocs-are-no-educationpanacea-but-heres-what-can-make-them-work/?cid=dlvr.it

    Sreeleakha P., Manikandan, N. (2015). The Way Forward with MOOCs – Professional Development Perspectives. American International Journal of Research in Humanities, Arts and Social Sciences, 15-514: 29-32.

    UNESCO (2002). Forum on the impact of open courseware for higher education in developing countries (CI-2002/CONF.803/CLD.1). Paris: UNESCO. Retrieved from http://unesdoc.unesco.org/images/0012/001285/128515e.pdf

    Wai-chung Yeung, H. (1995). Qualitative Personal Interviews in International Business Research: Some Lessons from a Study of Hong Kong Transnational Corporations, International Business Review, 4(3): 313-339

    Yuan, L., Powell, S., Oliver, B. (2014). Beyond MOOCs: Sustainable Online Learning in Institutions. Bolton: CETIS. Retrived from http://publications.cetis.ac.uk/2014/898

    Leave a Reply