Abstract A technology convergence prediction method based on the semantic representation of patent classification sequence and text is proposed to enrich the network and text semantic representation of patent classification, realize their more effective semantic fusion, and improve the effect of technology convergence prediction. First, the semantic representation of the patent classification sequence is directly carried out, and a technology convergence prediction method based on the semantic representation of the patent classification sequence is proposed, considering the location and context of patent classification. Second, the patent classification text allocation method is designed according to the ranking importance of patent classification in the sequence while the technology convergence prediction method is formed based on the semantic representation of patent classification text. Then, a multi-feature fusion method and a technology convergence prediction method combining patent classification sequence structure and the semantic representation of text content are proposed. Finally, based on the theory and method of link prediction, the proposed multi-technology convergence prediction methods are quantitatively evaluated. Experiments in the unmanned aerial vehicle field confirm that the effect of the patent classification sequence semantic representation model is better than other network representation learning methods. The text assignment method of patent classification by importance is better than the average text distribution method, which can better predict technology convergence. In the semantic fusion model, “Support Vector Machine + Hadamard Product” has the best performance, which is better than the single patent classification sequence and the patent classification text method. The method used in this study can better predict the possible technology convergence and provide better reference for technology layout and technology research and development.
Zhang Jinzhu,Li Yifeng. Technology Convergence Prediction by the Semantic Representation of Patent Classification Sequence and Text[J]. 情报学报, 2022, 41(6): 609-624.
1 Hacklin F, Raurich V, Marxt C. How incremental innovation becomes disruptive: the case of technology convergence[C]// Proceedings of the 2004 IEEE International Engineering Management Conference. IEEE, 2004: 32-36. 2 翟东升, 刘鹤, 张杰, 等. 一种基于链路预测的技术机会挖掘方法[J]. 情报学报, 2016, 35(10): 1090-1100. 3 Katz M L. Remarks on the economic implications of convergence[J]. Industrial and Corporate Change, 1996, 5(4): 1079-1095. 4 Greenstein S, Khanna T. What does industry convergence mean?[C]// Competing in the Age of Digital Convergence. Boston: Harvard Business School Press, 1997: 201-226. 5 Stieglitz N. Digital dynamics and types of industry convergence: the evolution of the handheld computers market[M]// The New Industrial Dynamics of the New Digital Economy. Social Science Electronic Publishing, 2007: 179-208. 6 Karvonen M, Kassi T, Kapoor R. Technological innovation strategies in converging industries[J]. International Journal of Business Innovation and Research, 2010, 4(5): 391-410. 7 Kim T S, Sohn S Y. Machine-learning-based deep semantic analysis approach for forecasting new technology convergence[J]. Technological Forecasting and Social Change, 2020, 157: 120095. 8 Roco M C, Bainbridge W S. Converging technologies for improving human performance: integrating from the nanoscale[J]. Journal of Nanoparticle Research, 2002, 4: 281-295. 9 Lind J. Convergence: history of term usage and lessons for firm strategies[EB/OL]. [2013-03-05]. http://www.itseurope.org/ITS%20CONF/berlin04/Papers/1_LIND.doc. 10 娄岩, 杨培培, 黄鲁成. 基于专利的技术融合测度方法及实证研究[J]. 科研管理, 2019, 40(11): 134-145. 11 李姝影, 方曙. 测度技术融合与趋势的数据分析方法研究进展[J]. 数据分析与知识发现, 2017, 1(7): 2-12. 12 陈悦, 王康, 宋超, 等. 一种用于技术融合与演化路径探测的新方法: 技术群相似度时序分析法[J]. 情报学报, 2021, 40(6): 565-574. 13 Batagelj V. Efficient algorithms for citation network analysis[OL]. (2003-09-14) [2013-03-05]. http://arxiv.org/pdf/cs/0309023v1. 14 Verspagen B. Mapping technological trajectories as patent citation networks: a study on the history of fuel cell research[J]. Advances in Complex Systems, 2007, 10(1): 93-115. 15 Martinelli A. An emerging paradigm or just another trajectory? Understanding the nature of technological changes using engineering heuristics in the telecommunications switching industry[J]. Research Policy, 2012, 41(2): 414-429. 16 Kim J, Lee S. Forecasting and identifying multi-technology convergence based on patent data: the case of IT and BT industries in 2020[J]. Scientometrics, 2017, 111(1): 47-65. 17 Park I, Yoon B. Technological opportunity discovery for technological convergence based on the prediction of technology knowledge flow in a citation network[J]. Journal of Informetrics, 2018, 12(4): 1199-1222. 18 Rodriguez A, Kim B, Turkoz M, et al. New multi-stage similarity measure for calculation of pairwise patent similarity in a patent citation network[J]. Scientometrics, 2015, 103(2): 565-581. 19 No H J, Park Y. Trajectory patterns of technology fusion: trend analysis and taxonomical grouping in nanobiotechnology[J]. Technological Forecasting and Social Change, 2010, 77(1): 63-75. 20 Ko N, Yoon J, Seo W. Analyzing interdisciplinarity of technology fusion using knowledge flows of patents[J]. Expert Systems with Applications, 2014, 41(4): 1955-1963. 21 Han E J, Sohn S Y. Technological convergence in standards for information and communication technologies[J]. Technological Forecasting and Social Change, 2016, 106: 1-10. 22 Nesta L, Saviotti P P. Coherence of the knowledge base and the firm's innovative performance: evidence from the U.S. pharmaceutical industry[J]. The Journal of Industrial Economics, 2005, 53(1): 123-142. 23 Pennings J M, Puranam P. Market convergence & firm strategy: towards a systematic analysis[EB/OL]. [2013-03-05]. http://userpage.fu-berlin.de/~jmueller/its/conf/berlin04/Papers/1_LIND.doc. 24 陈悦, 王康, 宋超, 等. 基于技术融合视角下的人工智能技术嵌入态势研究[J]. 科学学研究, 2021, 39(8): 1448-1458. 25 Caviggioli F. Technology fusion: identification and analysis of the drivers of technology convergence using patent data[J]. Technovation, 2016, 55/56: 22-32. 26 Lee W S, Han E J, Sohn S Y. Predicting the pattern of technology convergence using big-data technology on large-scale triadic patents[J]. Technological Forecasting and Social Change, 2015, 100: 317-329. 27 李丫丫, 赵玉林. 基于专利的技术融合分析方法及其应用[J]. 科学学研究, 2016, 34(2): 203-211. 28 吴晓燕, 胡雅敏, 陈方. 基于专利共类的技术融合分析框架研究——以合成生物学领域为例[J]. 情报理论与实践, 2021, 44(10): 179-184. 29 王宏起, 夏凡, 王珊珊. 新兴产业技术融合方向预测: 方法及实证[J]. 科学学研究, 2020, 38(6): 1009-1017, 1075. 30 Feng S D, An H Z, Li H J, et al. The technology convergence of electric vehicles: exploring promising and potential technology convergence relationships and topics[J]. Journal of Cleaner Production, 2020, 260: 120992. 31 Preschitschek N, Niemann H, Leker J, et al. Anticipating industry convergence: semantic analyses vs IPC co-classification analyses of patents[J]. Foresight, 2013, 15(6): 446-464. 32 Eilers K, Frischkorn J, Eppinger E, et al. Patent-based semantic measurement of one-way and two-way technology convergence: the case of ultraviolet light emitting diodes (UV-LEDs)[J]. Technological Forecasting and Social Change, 2019, 140: 341-353. 33 王格格, 刘树林. 国际专利分类号间的知识流动与技术间知识溢出测度——基于中国发明授权专利数据[J]. 情报学报, 2020, 39(11): 1162-1170. 34 王鑫, 赵蕴华, 高芳. 基于分类号和引文的专利相似度测量方法研究[J]. 数字图书馆论坛, 2015(1): 57-62. 35 刘知远, 孙茂松, 林衍凯, 等. 知识表示学习研究进展[J]. 计算机研究与发展, 2016, 53(2): 247-261. 36 Ietswaart R, Gyori B M, Bachman J A, et al. GeneWalk identifies relevant gene functions for a biological context using network representation learning[J]. Genome Biology, 2021, 22(1): 55. 37 李枫林, 柯佳. 基于深度学习的文本表示方法[J]. 情报科学, 2019, 37(1): 156-164. 38 Mikolov T, Sutskever I, Kai C, et al. Distributed representations of words and phrases and their compositionality[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates, 2013: 3111-3119. 39 Le Q V, Mikolov T. Distributed representations of sentences and documents[C]// Proceedings of the 31st International Conference on International Conference on Machine Learning. JMLR.org, 2014: II-1188-II-1196. 40 Tang D Y, Qin B, Liu T. Document modeling with gated recurrent neural network for sentiment classification[C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2015: 1422-1432. 41 Yang Z C, Yang D Y, Dyer C, et al. Hierarchical attention networks for document classification[C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2016: 1480-1489. 42 Mahata D, Kuriakose J, Shah R R, et al. Key2Vec: automatic ranked keyphrase extraction from scientific articles using phrase embeddings[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2018, 2: 634-639. 43 Pagliardini M, Gupta P, Jaggi M. Unsupervised learning of sentence embeddings using compositional n-gram features[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2018, 1: 528-540. 44 Saha T K, Joty S, Al Hasan M. Con-S2V: a generic framework for incorporating extra-sentential context into Sen2Vec[C]// Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Cham: Springer, 2017: 753-769. 45 Tian H, Zhuo H H. Paper2vec: citation-context based document distributed representation for scholar recommendation[OL]. (2017-03-20). https://arxiv.org/pdf/1703.06587.pdf. 46 Jain S, Howe B, Yan J Q, et al. Query2Vec: an evaluation of NLP techniques for generalized workload analytics[OL]. (2018-02-02). https://arxiv.org/pdf/1801.05613.pdf. 47 Han J L, Song Y, Zhao W X, et al. hyperdoc2vec: distributed representations of hypertext documents[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2018, 1: 2384-2394. 48 张金柱, 于文倩, 刘菁婕, 等. 基于网络表示学习的科研合作预测研究[J]. 情报学报, 2018, 37(2): 132-139. 49 Perozzi B, Al-Rfou R, Skiena S. DeepWalk: online learning of social representations[C]// Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM Press, 2014: 701-710. 50 Grover A, Leskovec J. node2vec: scalable feature learning for networks[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM Press, 2016: 855-864. 51 Tang J, Qu M, Wang M Z, et al. LINE: large-scale information network embeddin[C]// Proceedings of the 24th International Conference on World Wide Web. Republic and Canton of Geneva: International World Wide Web Conferences Steering Committee, 2015: 1067-1077. 52 Wang D X, Cui P, Zhu W W. Structural deep network embedding[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM Press, 2016: 1225-1234. 53 Hamilton W L, Ying R, Leskovec J. Inductive representation learning on large graphs[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates, 2017: 1025-1035. 54 丁钰, 魏浩, 潘志松, 等. 网络表示学习算法综述[J]. 计算机科学, 2020, 47(9): 52-59. 55 Tang J, Qu M, Mei Q Z. PTE: predictive text embedding through large-scale heterogeneous text networks[C]// Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM Press, 2015: 1165-1174. 56 Dong Y X, Chawla N V, Swami A. metapath2vec: scalable representation learning for heterogeneous networks[C]// Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM Press, 2017: 135-144. 57 Shi C, Hu B B, Zhao W X, et al. Heterogeneous information network embedding for recommendation[J]. IEEE Transactions on Knowledge and Data Engineering, 2019, 31(2): 357-370. 58 Yang C, Liu Z Y, Zhao D L, et al. Network representation learning with rich text information[C]// Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2015: 2111-2117. 59 Zhang D K, Yin J, Zhu X Q, et al. Homophily, structure, and content augmented network representation learning[C]// Proceedings of the 2016 IEEE 16th International Conference on Data Mining. IEEE, 2016: 609-618. 60 Sun X F, Guo J, Ding X, et al. A general framework for content-enhanced network representation learning[OL]. (2016-10-17). https://arxiv.org/pdf/1610.02906.pdf. 61 Li C Z, Wang S Z, Yang D J, et al. PPNE: property preserving network embedding[C]// Proceedings of the International Conference on Database Systems for Advanced Applications. Cham: Springer, 2017: 163-179. 62 Ganguly S, Pudi V. Paper2vec: combining graph and text information for scientific paper representation[C]// Proceedings of the European Conference on Information Retrieval. Cham: Springer, 2017: 383-395. 63 Pan S R, Wu J, Zhu X Q, et al. Tri-party deep network representation[C]// Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2016: 1895-1901. 64 熊回香, 李晓敏, 杜瑾. 基于学术关键词与共被引的学者推荐研究[J]. 情报学报, 2021, 40(7): 725-733. 65 叶佳鑫, 熊回香, 童兆莉, 等. 在线医疗社区中面向医生的协同标注研究[J]. 数据分析与知识发现, 2020, 4(6): 118-128. 66 Voorhees E M. Variations in relevance judgments and the measurement of retrieval effectiveness[J]. Information Processing & Management, 2000, 36(5): 697-716. 67 谷勇浩, 黄博琪, 王继刚, 等. 基于半监督深度学习的木马流量检测方法[J]. 计算机研究与发展, 2022, 59(6): 1329-1342. 68 沈国际, 周燕辉. 2000—2017年无人机专利技术发展态势分析[J]. 国防科技, 2018, 39(6): 82-87. 69 Li K, Zou C Q, Bu S H, et al. Multi-modal feature fusion for geographic image annotation[J]. Pattern Recognition, 2018, 73: 1-14.
