|
|
|
| Construction of an Academic Evaluation Theory from the Perspective of Complexity Theory—Exploratory Research Based on Grounded Theory |
| Lu Hongru1, Ruan Xuanmin1, Cheng Ying1,2, Chen Ya1 |
1.School of Information Management of Nanjing University, Nanjing 210046
2.School of Chinese Language and Literature of Shandong Normal University, Jinan 250014 |
|
|
|
|
Abstract Evaluation in academia is an issue of wide concern in society. The policies and regulations promulgated in recent years have shown that China attaches great importance to this issue. This paper employs grounded theory to analyze research on evaluation theory and practice and summarizes 18 primary evaluational factors through open coding and eight key evaluational factors through axial coding. Finally, selective coding determines the logical relationship between key evaluational factors, guided by the basic principles and pivotal elements of complexity theory, to construct a theory of evaluation in academia. The analysis shows that based on the evaluation context, the evaluator forms clear evaluation thoughts (evaluation purpose, evaluation classification, evaluation dimension, and theoretical basis) through interaction with the evaluation object and stakeholders, and further determines evaluation conditions (evaluation criteria, evaluation system, and quality control) to restrict and standardize the implementation of evaluation. Real-time evaluation output (evaluation data) and final evaluation output (evaluation result) are generated through the overall evaluation process.
|
|
Received: 25 July 2019
|
|
|
|
1 刘志明. 以发展着的理论指导新的实践[EB/OL]. (2017-11-13) [2018-10-23]. http://theory.people.com.cn/n1/2017/1113/c40531-29642265.html.
2 实践没有止境理论创新也没有止境[EB/OL]. (2018-03-07) [2019-02-20]. http://theory.people.com.cn/n1/2018/0307/c40531-29852958.html.
3 叶继元. 人文社会科学评价体系探讨[J]. 南京大学学报(哲学·人文科学·社会科学), 2010, 47(1): 97-110, 160.
4 辛涛, 李雪燕. 教育评价理论与实践的新进展[J]. 清华大学教育研究, 2005, 26(6): 38-43.
5 Dark M, Mirkovic J. Evaluation theory and practice applied to cybersecurity education[J]. IEEE Security & Privacy, 2015, 13(2): 75-80.
6 文庭孝. 科学评价理论体系的构建研究[J]. 重庆大学学报(社会科学版), 2008, 14(3): 63-69.
7 吕红. 高校信息公开政策评价理论体系的构建研究[J]. 图书情报工作, 2012, 56(18): 25-28, 92.
8 Dewey J. Valuation and experimental knowledge[J]. The Philosophical Review, 1922, 31(4): 325-351.
9 Shadish W R. Evaluation theory is who we are[J]. The American Journal of Evaluation, 1998, 19(1): 1-19.
10 Donaldson S I, Lipsey M W. Roles for theory in contemporary evaluation practice: Developing practical knowledge[M]// The SAGE Handbook of Evaluation. Thousand Oaks: SAGE Publications, 2006: 57-75.
11 Hansen M, Alkin M C, Wallace T L. Depicting the logic of three evaluation theories[J]. Evaluation and Program Planning, 2013, 38: 34-43.
12 Scriven M. Evaluation theory and metatheory[M]// International Handbook of Educational Evaluation. Dordrecht: Springer, 2003, 9: 15-30.
13 Alkin M C, Christie C A. An evaluation theory tree[M]// Evaluation Roots. Thousand Oaks: SAGE Publications, 2004: 13-65.
14 张志昂. 研究生教育收益评价理论初探[J]. 教育与经济, 2003(2): 45-48.
15 张敏, 唐任伍. 我国高等教育资源利用效率评价理论研究[J]. 清华大学教育研究, 2006, 27(5): 24-28, 41.
16 王俭. 当前我国教育评价理论研究存在的问题与实践误区的价值取向分析[J]. 教师教育研究, 2008, 20(6): 49-55.
17 李凌艳, 李勉. 从西方教育评价理论发展的视角看我国学校评估研究[J]. 教育理论与实践, 2010, 30(2): 25-29.
18 Bergsmann E, Schultes M T, Winter P, et al. Evaluation of competence-based teaching in higher education: From theory to practice[J]. Evaluation and Program Planning, 2015, 52: 1-9.
19 王媛, 马小燕. 基于模糊理论与神经网络的人才评价方法[J]. 佳木斯大学学报(自然科学版), 2006, 24(3): 408-410.
20 赵伟, 林芬芬, 彭洁, 等. 创新型科技人才评价理论模型的构建[J]. 科技管理研究, 2012, 32(24): 131-135.
21 许海云, 李长玲. 基于未确知理论与信息熵方法的技术类知识产品的价值评价体系[J]. 图书情报工作, 2009, 53(16): 65-68.
22 乔欢. 数字图书馆评价理论框架构建者萨拉赛维奇[J]. 国家图书馆学刊, 2006, 15(1): 74-76.
23 Millstone E, van Zwanenberg P, Marshall F. Monitoring and evaluating agricultural science and technology projects: Theories, practices and problems[J]. IDS Bulletin, 2010, 41(6): 75-87.
24 U.S. News. How U.S. news calculated the best global universities rankings[EB/OL]. [2018-11-02]. https://www.usnews.com/educa tion/best-global-universities/articles/methodology.
25 Times Higher Education. World university rankings 2018 methodology[EB/OL]. [2018-08-21]. https://www.timeshighereducation.com/world-university-rankings/methodology-world-university-rankings-2018.
26 Quacquarelli Symonds. The QS world university rankings[EB/OL]. [2019-03-12]. https://www.topuniversities.com/qs-world-university-rankings/methodology.
27 上海软科教育信息咨询有限公司. Academic ranking of world universities 2018[EB/OL]. [2018-10-22]. http://www.shanghai ranking.com/ARWU-Methodology-2018.html.
28 High education funding council for England. Evaluation of REF 2014[EB/OL]. [2018-08-22]. https://www.ref.ac.uk/2014/media/ref/content/pub/assessmentframeworkandguidanceonsubmissions/GOS%20including%20addendum.pdf.
29 南京大学中国社会科学研究评价中心. 中文社会科学引文索引来源期刊/集刊遴选实施方案(试行)[EB/OL]. [2018-10-23]. http://cssrac.nju.edu.cn/a/zlxz/20171221/2857.html.
30 上海社会科学院智库研究中心. 2017中国智库报告[EB/OL]. (2018-02-07) [2018-09-23]. http://www.199it.com/archives/690079.html.
31 Walton M. Applying complexity theory: A review to inform evaluation design[J]. Evaluation and Program Planning, 2014, 45: 119-126.
32 Jorm C, Roberts C. Using complexity theory to guide medical school evaluations[J]. Academic Medicine, 2017, 93(3): 399-405.
33 Eoyang G, Oakden J. Adaptive evaluation: A synergy between complexity theory and evaluation practice[J]. Emergence-Complexity & Organization, 2016, 18: 3-4.
34 Walton M. Expert views on applying complexity theory in evaluation: Opportunities and barriers[J]. Evaluation, 2016, 22(4): 410-423.
35 Smith N L. Improving evaluation theory through the empirical study of evaluation practice[J]. Evaluation Practice, 1993, 14(3): 237-242.
36 靳代平, 王新新, 姚鹏. 品牌粉丝因何而狂热?——基于内部人视角的扎根研究[J]. 管理世界, 2016(9): 102-119.
37 Corbin J, Strauss A. Basics of qualitative research: Techniques and procedures for developing grounded theory[M]. Thousand Oaks: SAGE Publications, 2008: 159-160, 198-199.
38 许庆瑞, 吴志岩, 陈力田. 转型经济中企业自主创新能力演化路径及驱动因素分析——海尔集团1984~2013年的纵向案例研究[J]. 管理世界, 2013(4): 121-134, 188.
39 Carr G, Loucks D P, Bl?schl G. Gaining insight into interdisciplinary research and education programmes: A framework for evaluation[J]. Research Policy, 2018, 47(1): 35-48.
40 Walther A, van den Bosch J J F. FOSE: A framework for open science evaluation[J]. Frontiers in Computational Neuroscience, 2012, 6: 32.
41 谭和平. 我国公益类科研院所运行评价体系——基于管理熵理论的研究[J]. 社会科学研究, 2008(5): 192-194.
42 王飒. “985”高校图书馆科研评价服务实践调研与分析[J]. 图书情报工作, 2016, 60(1): 26-31.
43 夏云霞, 徐涛, 翟康, 等. 研究所科研团队绩效评价的探索与实践[J]. 科研管理, 2017, 38(S1): 510-514.
44 Granit-Dgani D, Kaplan A, Flum H. Theory-based assessment in environmental education: A tool for formative evaluation[J]. Environmental Education Research, 2017, 23(2): 269-299.
45 Fjortoft N F, Schwartz A H. Evaluation of a pharmacy continuing education program: Long-term learning outcomes and changes in practice behaviors[J]. The American Journal of Pharmaceutical Education, 2003, 67(2): 35.
46 Anderson E, Smith R, Hammick M. Evaluating an interprofessional education curriculum: A theory-informed approach[J]. Medical Teacher, 2016, 38(4): 385-394.
47 林梦泉, 陈燕, 任超, 等. 约束条件下的学科建设绩效评价理论体系探究[J]. 中国高教研究, 2018(7): 17-21.
48 吉汉强, 李丽舒, 黄超云, 等. 文献资源建设绩效评价指标体系构建的实践研究[J]. 图书馆建设, 2011(4): 26-30.
49 de Lima R G, Lins H N, Pfitscher E D, et al. A sustainability evaluation framework for Science and Technology Institutes: An international comparative analysis[J]. Journal of Cleaner Production, 2016, 125: 145-158.
50 Yu L P, Chen Y Q, Pan Y T, et al. Research on the evaluation of academic journals based on structural equation modeling[J]. Journal of Informetrics, 2009, 3(4): 304-311.
51 Klein J T. Evaluation of interdisciplinary and transdisciplinary research: A literature review[J]. American Journal of Preventive Medicine, 2008, 35(Suppl 2): S116-S123.
52 赵伟, 包献华, 屈宝强, 等. 创新型科技人才分类评价指标体系构建[J]. 科技进步与对策, 2013, 30(16): 113-117.
53 Klautzer L, Hanney S, Nason E, et al. Assessing policy and practice impacts of social science research: The application of the Payback Framework to assess the Future of Work programme[J]. Research Evaluation, 2011, 20(3): 201-209.
54 Cousins J B, Earl L M. The case for participatory evaluation[J]. Educational Evaluation and Policy Analysis, 1992, 14(4): 397-418.
55 郑烨, 胡春萍. 基金委是透明的吗?——基于循证评估的科学基金信息公开实践评价[J]. 情报杂志, 2017, 36(9): 78-85.
56 张士强, 马光成. 毕业设计质量评价的研究与实践[J]. 中国高教研究, 2001(10): 80-81.
57 辛志勇, 杜晓鹏, 许晓晖. 澳大利亚学校价值观教育实效性评价实践[J]. 比较教育研究, 2016, 38(9): 7-13.
58 邱均平, 任全娥. 国内外人文社会科学科研成果评价比较研究[J]. 国外社会科学, 2007(3): 58-66.
59 Simpson K M, Porter K, McConnell E S, et al. Tool for evaluating research implementation challenges: A sense-making protocol for addressing implementation challenges in complex research settings[J]. Implementation Science, 2013, 8: 2.
60 Stufflebeam D L. The relevance of the CIPP evaluation model for educational accountability[J]. Journal of Research and Development in Education, 1971, 5(1): 19-25.
61 肖远军. CIPP教育评价模式探析[J]. 教育科学, 2003, 19(3): 42-45.
62 王建明, 王俊豪. 公众低碳消费模式的影响因素模型与政府管制政策——基于扎根理论的一个探索性研究[J]. 管理世界, 2011(4): 58-68.
63 吴今培, 李雪岩, 赵云. 复杂性之美[M]. 北京: 北京交通大学出版社, 2017: 27, 53, 104.
64 Trenholm S, Ferlie E. Using complexity theory to analyse the organisational response to resurgent tuberculosis across London[J]. Social Science & Medicine, 2013, 93: 229-237.
65 苗东升. 复杂性科学研究[M]. 北京: 中国书籍出版社, 2013: 124.
66 Gare A. Systems theory and complexity: Introduction[J]. Democracy & Nature, 2000, 6(3): 327-339.
67 魏巍, 刘仲林. 国外跨学科评价理论新进展[J]. 科学学与科学技术管理, 2011, 32(4): 20-25.
68 陈一壮. 试论复杂性理论的精髓[J]. 哲学研究, 2005(6): 108-114.
69 Dillman L M. Comparing evaluation activities across multiple theories of practice[J]. Evaluation and Program Planning, 2013, 38: 53-60.
70 臧莉娟. 期刊评价结果权威性影响因素分析——以人文社会科学期刊评价为中心[J]. 中国出版, 2015(16): 11-15.
71 Ladyshewsky R K, Taplin R. Evaluation of curriculum and student learning needs using 360 degree assessment[J]. Assessment & Evaluation in Higher Education, 2015, 40(5): 698-711.
72 Mark M M. Building a better evidence base for evaluation theory[M]// Fundamental Issues in Evaluation. New York: The Guilford Press, 2008: 111-134.
73 刘晓君, 李险峰, 张建刚. 一个新自治混沌系统的混沌同步控制[J]. 复杂系统与复杂性科学, 2007, 4(4): 51-57.
74 金吾伦, 郭元林. 复杂性科学及其演变[J]. 复杂系统与复杂性科学, 2004, 1(1): 1-5.
75 Marchal B, van Belle S, de Brouwere V, et al. Studying complex interventions: Reflections from the FEMHealth project on evaluating fee exemption policies in West Africa and Morocco[J]. BMC Health Services Research, 2013, 13: 469.
76 吴彤. 复杂的实在[J]. 自然辩证法研究, 2005, 21(6): 1-4, 10.
77 叶继元. 图书馆学期刊质量“全评价”探讨及启示[J]. 中国图书馆学报, 2013, 39(4): 83-92.
78 Hartmeyer R, Stevenson M P, Bentsen P. Evaluating design-based formative assessment practices in outdoor science teaching[J]. Educational Research, 2016, 58(4): 420-441.
79 严明清. 地方社科院成果评价体系及科研管理创新与思考[J]. 社会科学管理与评论, 2009(4): 74-79, 112.
80 胡琳, 刘倩, 舒予. 图书馆员国际交流项目评价体系研究[J]. 大学图书馆学报, 2017, 35(3): 18-24.
81 Bonaccorsi A, Piccaluga A. A theoretical framework for the evaluation of university-industry relationships[J]. R & D Management, 1994, 24(3): 229-247.
82 Alkin M C. Evaluation theory and practice: Insights and new directions[J]. New Directions for Evaluation, 2003, 2003(97): 81-90.
83 Rallis S F, Rossman G B. Dialogue for learning: Evaluator as critical friend[J]. New Directions for Evaluation, 2000, 2000(86): 81-92.
84 于永达, 贺贵才. 自主知识产权产品评价: 理论与实例研究[J]. 科学学研究, 2008, 26(6): 1188-1192.
85 季靖. 高校教师教学质量评价实践及启示[J]. 中国流通经济, 2001, 15(S1): 32-35.
86 The Evaluation Center at Western Michigan University. Evaluation checklists[EB/OL]. [2018-09-08]. https://www.wmich.edu/evaluation/checklists,2018-03-02/.
87 Stufflebeam D L. Evaluation plans and operations checklist[EB/OL]. [2020-07-24]. https://wmich.edu/sites/default/files/attachments/u350/2018/eval-plans-operations-stufflebeam.pdf.
88 李月琳, 梁娜, 齐雪. 从交互维度到交互功能: 构建数字图书馆交互评估理论模型[J]. 中国图书馆学报, 2016, 42(1): 66-82.
89 Phillips-Wren G, Mora M, Forgionne G A, et al. An integrative evaluation framework for intelligent decision support systems[J]. European Journal of Operational Research, 2009, 195(3): 642-652.
90 Dahler-Larsen P. Theory-based evaluation meets ambiguity: The role of Janus variables[J]. American Journal of Evaluation, 2018, 39(1): 6-23.
91 Tackett S, Grant J, Mmari K. Designing an evaluation framework for WFME basic standards for medical education[J]. Medical Teacher, 2016, 38(3): 291-296.
92 吕潭华. 可持续发展评价指标体系研究[D]. 厦门: 厦门大学, 2002.
93 郎永杰, 杨青, 郭芳芳. 产学研合作绩效评价: 内涵、缘起与实践[J]. 教育理论与实践, 2017, 37(24): 6-8.
94 Xu H, Xing L N, Huang L. Evaluation of scientific publications with hesitant fuzzy uncertain linguistic and semantic information[J]. Journal of Intelligent and Fuzzy Systems, 2015, 29(6): 2737-2742.
95 陶军. 权利保障维度的国家科研资助评价体系构建——以国家社科基金为例[J]. 宏观质量研究, 2017, 5(2): 82-97.
96 金福, 陈伟. 遗传算法之父——霍兰及其科学工作[J]. 自然辩证法通讯, 2007, 29(2): 86-93, 112.
97 Gangopadhyay P. Making evaluation meaningful to all education stakeholders[R]. The Evaluation Center, Western Michigan University, 2002: 1-3.
98 陈向明. 社会科学中的定性研究方法[J]. 中国社会科学, 1996(6): 93-102. |
|
|
|
