Advanced
search

Prelims

John N. Moye

A Machine Learning, Artificial Intelligence Approach to Institutional Effectiveness in Higher Education

ISBN: 978-1-78973-900-8, eISBN: 978-1-78973-899-5

Publication date: 29 July 2019

pdf (3.2 MB) ePub (559 KB) Article view Figure view Cited (2) cite article

Citation

Moye, J.N. (2019), "Prelims", A Machine Learning, Artificial Intelligence Approach to Institutional Effectiveness in Higher Education, Emerald Publishing Limited, Leeds, pp. i-xiii. https://doi.org/10.1108/978-1-78973-899-520191009

Publisher

:

Emerald Publishing Limited

Copyright © 2019, John N. Moye.

Half Title Page

A Machine Learning, Artificial Intelligence Approach to Institutional Effectiveness in Higher Education

Title Page

A Machine Learning, Artificial Intelligence Approach to Institutional Effectiveness in Higher Education

By

John N. Moye

United Kingdom – North America – Japan India – Malaysia – China

Copyright Page

Emerald Publishing Limited

Howard House, Wagon Lane, Bingley BD16 1WA, UK

First edition 2019

© John N. Moye. Published under exclusive licence by Emerald Publishing Limited.

Reprints and permissions service

Contact: permissions@emeraldinsight.com

No part of this book may be reproduced, stored in a retrieval system, transmitted in any form or by any means electronic, mechanical, photocopying, recording or otherwise without either the prior written permission of the publisher or a licence permitting restricted copying issued in the UK by The Copyright Licensing Agency and in the USA by The Copyright Clearance Center. Any opinions expressed in the chapters are those of the authors. Whilst Emerald makes every effort to ensure the quality and accuracy of its content, Emerald makes no representation implied or otherwise, as to the chapters’ suitability and application and disclaims any warranties, express or implied, to their use.

British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library

ISBN: 978-1-78973-900-8 (Print)

ISBN: 978-1-78973-899-5 (Online)

ISBN: 978-1-78973-901-5 (Epub)

Contents

List of Tables vii
Author Biography xi
Foreword xiii
Chapter 1 Defining, Measuring, and Assessing Effectiveness 1
Chapter 2 Creating Shared Mission, Vision, and Values 15
Chapter 3 Measuring and Assessing Program Structure: Intended Performance 31
Chapter 4 Measuring and Assessing Instruction: Intended Performance 77
Chapter 5 Measuring and Assessing Support Services: Intended Performance 99
Chapter 6 Functional Data Modeling: Identifying the Drivers and Constraints of Actual Performance 133
Chapter 7 Institutional Data Modeling: Looking Beyond the Data 145
Chapter 8 Continuous Quality Improvement 173
Afterword 195
References 197
Index 215

List of Tables

Table 1. Power Analysis of Functional Performance Indicators. 14
Table 2. MVVs Statements for a School of Pharmacy. 19
Table 3. Environmental Measurement to Support Institutional Definition. 22
Table 4. Institutional Environmental Data Summary. 23
Table 5. Environmental Data Aggregation. 27
Table 6. SWOT Analysis. 29
Table 7. Curriculum Measurement Tool. 34
Table 8. Course Structure Measurement. 37
Table 9. Curriculum Measurement Program Data Summary. 39
Table 10. Curriculum Assessment Institutional Assessment. 40
Table 11. Program Course Structure Data Summary. 41
Table 12. Institutional Course Structure Assessment. 42
Table 13. Program Learning Outcomes Measurement Rubric. 44
Table 14. Program Learning Objectives Measurement. 46
Table 15. Rubric Model. 53
Table 16. Program Learning Outcomes Measurement Rubric Model. 54
Table 17. Program Learning Outcomes Data Summary – Program Level. 56
Table 18. Program Learning Outcomes Assessment – Institutional Level. 56
Table 19. Program Learning Objectives Data Summary Tool. 57
Table 20. Institutional Learning Objectives Assessment. 63
Table 21. Institutional Mission Alignment Measurement. 65
Table 22. Institutional Values Alignment Measurement. 68
Table 23. Competitor Measurement. 71
Table 24. Target Populations Measurement. 72
Table 25. Market and Career Measurement. 72
Table 26. Infrastructure Measurement. 74
Table 27. Program Feasibility Data Summary. 74
Table 28. Program Feasibility Assessment. 75
Table 29. Learning Engagement Observation Form. 79
Table 30. Learning Experience Questionnaire. 81
Table 31. Student Learning Outcomes Portfolio. 84
Table 32. Learning Engagement Data Report. 88
Table 33. Distribution of Engagement Strategies for an Individual Instructor for Multiple Courses. 89
Table 34. Learning Engagement Data Summary. 91
Table 35. Learning Engagement Data Aggregation. 92
Table 36. Learning Experience Data Report. 93
Table 37. Learning Experience Data Summary. 96
Table 38. Learning Experience Data Aggregation. 97
Table 39. Mission, Vision, and Values for a Student Services Function. 100
Table 40. Student Life Measurement. 103
Table 41. Measurement of Internship/Externship Experiences. 104
Table 42. Academic Success Center Measurement. 106
Table 43. Library Questionnaire. 107
Table 44. Disabilities Services Measurement. 109
Table 45. Counseling Services Measurement. 110
Table 46. Career Services Measurement. 112
Table 47. Graduate Student Support Services Measurement. 114
Table 48. Student Life Service Assessment. 116
Table 49. Student Life Component Assessment. 117
Table 50. Internship/Externship Experiences Service Assessment. 117
Table 51. Internship/Externship Experiences Component Assessment. 119
Table 52. Academic Success Center Service Assessment. 120
Table 53. Academic Success Center Component Assessment. 121
Table 54. Library Service Assessment. 122
Table 55. Library Component Assessment. 123
Table 56. Disabilities Services Service Assessment. 123
Table 57. Disabilities Services Component Assessment. 124
Table 58. Counseling Services Service Assessment. 125
Table 59. Counseling Services Component Assessment. 126
Table 60. Career Services Service Assessment. 127
Table 61. Career Services Component Assessment. 128
Table 62. Assessment of Student Support Services Components. 129
Table 63. Graduate Assessment of Student Support Services Functional Assessment. 130
Table 64. Graduate Student Support Services Institutional Assessment. 131
Table 65. Correlation Matrix (Spearman Rho). 136
Table 66. KPI Clusters Based on the Strength of Correlation. 137
Table 67. Institutional Dashboard Summarizing Functional Assessments. 141
Table 68. Goal-Path Approach to Sensemaking. 147
Table 69. A Trending Analysis Using the Goal-Path Approach at Institutional Level. 149
Table 70. Identification and Articulation of the KPIs for the Institution. 151
Table 71. Trending Report of Sensemaking Data for the Institution. 157
Table 72. Percentage of Students Demonstrating Program Learning Objectives by Programs. 158
Table 73. Percentage of Students Demonstrating Program Learning Outcomes for All Programs. 158
Table 74. Strategic Constituency Analysis of Student Support Services KPIs. 159
Table 75. Institutional Student Support Service Responsibilities. 159
Table 76. Internal Feasibility KPIs Summary and Aggregation for All Programs. 160
Table 77. Competing Values Environmental Analysis. 162
Table 78. Institutional Effectiveness Executive Report. 165
Table 79. Action Planning Inputs. 171
Table 80. Action Planning Process. 172
Table 81. Institutional Effectiveness Report – Four Years. 175
Table 82. Definitions of Indicators in Institutional Effectiveness Report. 177
Table 83. Power Analysis of Functional Performance Indicators. 178
Table 84. Correlation Matrix for Performance Indicators. 182
Table 85. KPI Clusters of Strongest Correlations. 184
Table 86. Action Items. 189
Table 87. Action Planning. 191

Author Biography

John Moye is a native of Jacksonville, FL, where he attended Jacksonville University. During his undergraduate and master’s degree experience, he studied with a series of forward-looking thought leaders in education from which he developed an interest and belief in the science of learning and the power and importance of education for all learners. These interests have accompanied him throughout his career and led to a focus on the performance, effectiveness, and responsibilities of higher education.

Dr Moye continued his pursuit of the science of learning through his Ph.D. studies at Florida State University, where he focused his research in the field of psychophysics. Heavily impacted by the burgeoning field of neuroscience, he examined the response of the human perceptual systems to sensory stimuli as a model for understanding learning as a psychophysical process in individuals and organizations. The conceptual frameworks contained in this text are based on the evidence of the psychophysics of learning that are still emerging in the academic learning literature.

Dr. Moye has held effectiveness positions with numerous institutions of higher learning in the United States, including Saint Mary’s University of Minnesota, Capella University, and De Paul University, Chicago, IL, in which he has researched, developed, and applied these approaches to the development of relevant, innovative, and effective learning environments. In addition, he has contributed to a wide array of other institutions of higher learning as a consultant, which has provided a comprehensive perspective on the science of measurement and assessment in complex organizations.

Dr Moye believes the leverage point in the system of institutional improvement to be the availability of authentic, credible, and trustworthy information to make sense of institutional performance for effectiveness improvement efforts. The research and development of systematic assessments to measure the effectiveness of unique institutions is a focus of his ongoing professional efforts.

Foreword

This work seeks to catalyze discussion and thinking about the information required to measure, assess, and make sense of institutional performance with credible and trustworthy data. To those who believe it is possible to improve the performance of our institutions this work offers a method to improve service to students and society through data-informed problem-solving and decision-making.

To achieve this outcome requires data that objectively describe the “actual” performance of the institution, which the faculty and staff use to understand current performance and improve the future performance of their programs and institutions (Tadesse, Manathynga, & Gillies, 2018). The result is a system in which the principles of machine learning define the data processing functions and create a credible and trustworthy artificial intelligence for institutional effectiveness (Yousef, Allmer, Baştanlar, Özuysal, & Walker, 2013). The purpose of this work is to offer a fully aligned system of authentic assessments, which provide faculty and staff with credible and trustworthy information to monitor, demonstrate, and enhance institutional performance (Swaggerty & Broemmel, 2017).

The processes and procedures in this work adapt recent and current strategies of performance measurement, assessment, and sensemaking in the discipline of organizational effectiveness into a science-based approach to the assessment and sensemaking of institutional effectiveness in higher education (Cameron & Whetten, 2013). The principles of organizational assessment and the sciences of educational and psychological measurement and assessment define the content and structure of the information collected in this system (Knight, McLaughlin, & Howard, 2012). As such, this approach is a “best science” approach to institutional assessment and effectiveness. In this work, the goal is to present a fully-aligned system of assessments for institutional effectiveness, which are disciplined by appropriate technologies.

The methods and instruments employed in this assessment system emerged from research, design, development, and testing the results of their use as institutional effectiveness assessments for a cross-section of higher education institutions. These instruments have consistently yielded stable, statistically powerful, credible, and trustworthy data about the performance of the institution. These data inform authentic assessments, data modeling, and sensemaking functions to evaluate the effectiveness of the institution (Swaggerty & Broemmel, 2017).

The principles of machine learning and artificial intelligence frame the data modeling and sensemaking strategies to visualize actual institutional performance from multiple perspectives. The output of this approach is a system that provides credible, trustworthy, and meaningful data for the evaluation of effectiveness, which the human intelligence in the institution evaluates.

Manage cookies

We’re listening — tell us what you think

Join us on our journey