|
|
|
| Graph Neural Network-based and Particle Swarm Optimization Technological Prediction Model |
| Lian Zhixuan, Wang Fang, Kang Jia, Yuan Chang |
| Department of Information Resources Management at Business School, Nankai University, Tianjin 300072 |
|
|
|
|
Abstract Effective prediction of technological development trends is crucial for policymaking in many technological industries. As patent application forms express technical features in an in-depth manner, they can be used to train a model to improve the technology prediction. This study constructs a technology prediction model on the basis of particle swarm optimization (PSO) and graph neural networks to help improve the prediction accuracy of the future development trend of a technology field and the characteristics of its emerging technologies. Using 594 artificial intelligence patent applications arranged in China by US companies over the last two decades as the research objects, this study conducted an experiment and found that the suggested model obtains higher accuracy than baseline algorithms, such as the exponential smoothing method, moving average method, support vector regression, gate recurrent unit, and recurrent neural network. Moreover, the model can reveal the formation process of the new characteristics of the technologies. The study predicts the layout of US patents in China to investigate the trend, characteristics, and gaps in the current US technology layout in China. The proposed graph neural network-based and PSO technological prediction model can improve the technology prediction accuracy and support the decision making in the layout of technological industries and the funding of scientific research.
|
|
Received: 19 January 2022
|
|
|
|
1 Liang Z T, Mao J, Lu K, et al. Combining deep neural network and bibliometric indicator for emerging research topic prediction[J]. Information Processing & Management, 2021, 58(5): 102611.
2 Tseng Y H, Lin C J, Lin Y I. Text mining techniques for patent analysis[J]. Information Processing & Management, 2007, 43(5): 1216-1247.
3 Lai K K, Wu S J. Using the patent co-citation approach to establish a new patent classification system[J]. Information Processing & Management, 2005, 41(2): 313-330.
4 Min C, Bu Y, Wu D, et al. Identifying citation patterns of scientific breakthroughs: a perspective of dynamic citation process[J]. Information Processing & Management, 2021, 58(1): 102428.
5 Lee C J, Kogler D F, Lee D. Capturing information on technology convergence, international collaboration, and knowledge flow from patent documents: a case of information and communication technology[J]. Information Processing & Management, 2019, 56(4): 1576-1591.
6 Guan J C, Chen Z F. Patent collaboration and international knowledge flow[J]. Information Processing & Management, 2012, 48(1): 170-181.
7 宋凯, 李秀霞, 赵思喆, 等. 基于LDA模型的国家间知识流动分析[J]. 情报杂志, 2017, 36(6): 55-60.
8 He X J, Meng X, Wu Y Y, et al. Semantic matching efficiency of supply and demand texts on online technology trading platforms: taking the electronic information of three platforms as an example[J]. Information Processing & Management, 2020, 57(5): 102258.
9 徐路路, 王芳. 基于支持向量机和改进粒子群算法的科学前沿预测模型研究[J]. 情报科学, 2019, 37(8): 22-28.
10 化柏林. 多源信息融合方法研究[J]. 情报理论与实践, 2013, 36(11): 16-19.
11 孙震. 基于科学论文多源数据的研究前沿集成识别模型研究[J]. 情报杂志, 2016, 35(8): 95-100.
12 刘琦岩, 曾文, 车尧. 面向重点领域科技前沿识别的情报体系构建研究[J]. 情报学报, 2020, 39(4): 345-356.
13 王筠. 文献老化的回归分析与预测[J]. 情报杂志, 2006, 25(6): 68-69.
14 张洋, 林宇航, 侯剑华. 基于融合数据和生命周期的技术预测方法: 以病毒核酸检测技术为例[J]. 情报学报, 2021, 40(5): 462-470.
15 王超, 倪静. 基于SIRS模型的高校科研团队内部隐性知识传播模型[J]. 科技和产业, 2021, 21(10): 234-238.
16 丁俊武, 韩玉启. 浅析技术系统演化理论在机器人技术发展方面的应用[J]. 机器人技术与应用, 2005(5): 37-40.
17 冯立杰, 王亚星, 冯奕程, 等. 基于多维技术创新地图的技术预测研究[J]. 情报杂志, 2017, 36(8): 58-63, 26.
18 刘玉梅, 温馨, 孟翔飞. 基于技术轨道跃迁的突破性技术预测方法及应用研究[J]. 情报杂志, 2021, 40(11): 39-45, 15.
19 杨红平, 马大川. 灰色预测理论在信息分析与预测中的应用[J]. 情报杂志, 2004, 23(3): 22-24.
20 钟昌宝, 聂茂林, 徐永其. 基于灰色马尔柯夫改进模型预测供应链独立需求[J]. 情报杂志, 2009, 28(7): 199-203, 198.
21 宋凯, 朱彦君. 专利前沿技术主题识别及趋势预测方法——以人工智能领域为例[J]. 情报杂志, 2021, 40(1): 33-38.
22 唐川, 唐卷, 房俊民, 等. 图灵奖得主识别与预测研究——基于多文献计量指标和支持向量机[J]. 情报杂志, 2015, 34(2): 69-72, 78.
23 董放, 刘宇飞, 周源. 基于LDA-SVM论文摘要多分类新兴技术预测[J]. 情报杂志, 2017, 36(7): 40-45, 133.
24 王雪. 基于时间序列模型的高水平学科预测研究[J]. 情报杂志, 2019, 38(6): 45-49, 117.
25 王廷满, 沈思. 基于BP神经网络的图书发行量预测模型研究[J]. 情报杂志, 2003, 22(6): 61-62, 65.
26 代鲁凡. 基于深度学习的商品期货趋势预测研究与实现[D]. 西安: 西北大学, 2021.
27 辛泽西. 基于专利数据的技术趋势预测算法的研究与实现[D]. 北京: 北京邮电大学, 2020.
28 曹志鹏, 潘定, 潘启亮. 基于表示学习的双层知识网络链路预测[J]. 情报学报, 2021, 40(2): 135-144.
29 许学国, 桂美增. 基于深度学习的技术预测方法——以机器人技术为例[J]. 情报杂志, 2020, 39(8): 53-62.
30 王兴旺, 董珏, 余婷婷, 等. 基于多种类型信息计量分析的前沿技术预测方法研究[J]. 情报杂志, 2018, 37(10): 70-75, 89.
31 刘汉, 王永莲. 基于混频预测模型改进预测精度——以入境旅游为例[J]. 情报杂志, 2016, 35(9): 75-79.
32 Casteigts A, Flocchini P, Quattrociocchi W, et al. Time-varying graphs and dynamic networks[J]. International Journal of Parallel, Emergent and Distributed Systems, 2012, 27(5): 387-408.
33 Holme P, Saram?ki J. Temporal networks[J]. Physics Reports, 2012, 519(3): 97-125.
34 Kostakos V. Temporal graphs[J]. Physica A: Statistical Mechanics and Its Applications, 2009, 388(6): 1007-1023.
35 Lakshmi T J, Bhavani S D. Temporal probabilistic measure for link prediction in collaborative networks[J]. Applied Intelligence, 2017, 47(1): 83-95.
36 赵蓉英. 知识网络研究(Ⅱ)——知识网络的概念、内涵和特征[J]. 情报学报, 2007, 26(3): 470-476.
37 赵蓉英, 邱均平. 知识网络研究(Ⅰ)——知识网络概念演进之探究[J]. 情报学报, 2007, 26(2): 198-209.
38 马费成, 刘向. 知识网络的演化(Ⅰ): 增长与老化动态[J]. 情报学报, 2011, 30(8): 787-795.
39 马费成, 刘向. 知识网络的演化(Ⅲ): 连接机制[J]. 情报学报, 2011, 30(10): 1015-1021.
40 Li M N, Porter A L, Suominen A, et al. An exploratory perspective to measure the emergence degree for a specific technology based on the philosophy of swarm intelligence[J]. Technological Forecasting and Social Change, 2021, 166: 120621.
41 Chen S H, Huang M H, Chen D Z, et al. Detecting the temporal gaps of technology fronts: a case study of smart grid field[J]. Technological Forecasting and Social Change, 2012, 79(9): 1705-1719.
42 祝娜, 王芳. 基于主题关联的知识演化路径识别研究——以3D打印领域为例[J]. 图书情报工作, 2016, 60(5): 101-109.
43 桂美增, 许学国. 基于深度学习的技术机会预测研究——以新能源汽车为例[J]. 图书情报工作, 2021, 65(19): 130-141.
44 王培妍, 段磊, 郭正山, 等. 基于张量分解的知识超图链接预测模型[J]. 计算机研究与发展, 2021, 58(8): 1599-1611.
45 伍杰华, 高学勤, 王涛. 融合链接预测相似度矩阵的属性网络嵌入算法[J]. 计算机应用研究, 2022, 39(4): 1080-1085.
46 Kashima H, Abe N. A parameterized probabilistic model of network evolution for supervised link prediction[C]// Proceedings of the Sixth International Conference on Data Mining. Piscataway: IEEE, 2007: 340-349.
47 Symeonidis P, Mantas N. Spectral clustering for link prediction in social networks with positive and negative links[J]. Social Network Analysis and Mining, 2013, 3(4): 1433-1447.
48 Yang X, Tian Z, Cui H Y, et al. Link prediction on evolving network using tensor-based node similarity[C]// Proceedings of the 2012 IEEE 2nd International Conference on Cloud Computing and Intelligence Systems. Piscataway: IEEE, 2012: 154-158.
49 Yang C, Liu Z Y. Comprehend DeepWalk as matrix factorization[OL]. (2015-01-02). https://arxiv.org/pdf/1501.00358.pdf.
50 Qiu J Z, Dong Y X, Ma H, et al. Network embedding as matrix factorization: unifying DeepWalk, LINE, PTE, and node2vec[C]// Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. New York: ACM Press, 2018: 459-467.
51 宋浩楠, 赵刚, 孙若莹. 基于深度强化学习的知识推理研究进展综述[J]. 计算机工程与应用, 2022, 58(1): 12-25.
52 吴安彪, 袁野, 马玉亮, 等. 时序图节点嵌入策略的研究[J]. 软件学报, 2021, 32(3): 650-668.
53 汪俊, 岳峰, 王刚, 等. 科研社交网络中基于链接预测的专家推荐研究[J]. 情报杂志, 2015, 34(6): 151-157.
54 任海英, 于立婷, 黄鲁成. 基于链接预测的科学研究机会发现方法研究[J]. 情报杂志, 2016, 35(10): 53-58, 36.
55 宫雪, 崔雷. 基于医学主题词共现网络的链接预测研究[J]. 情报杂志, 2018, 37(1): 66-71, 52.
56 张宁豫, 陈曦, 陈矫彦, 等. 基于位置的知识图谱链接预测[J]. 中文信息学报, 2018, 32(4): 80-86, 129.
57 孙蒙鸽, 王燕鹏, 韩涛, 等. 新兴技术的多指标量化识别研究——基于向量表征方法的探索[J]. 图书情报工作, 2022, 66(3): 130-139.
58 彭敏, 黄婷, 田纲, 等. 聚合邻域信息的联合知识表示模型[J]. 中文信息学报, 2021, 35(5): 46-54.
59 Zhou L K, Yang Y, Ren X, et al. Dynamic network embedding by modeling triadic closure process[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, 32(1): 571-578.
60 Zhu L H, Guo D, Yin J M, et al. Scalable temporal latent space inference for link prediction in dynamic social networks[J]. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(10): 2765-2777.
61 Kazemi S M, Goel R, Jain K, et al. Relational representation learning for dynamic (knowledge) graphs: a survey[OL]. (2019-05-27). https://arxiv.org/pdf/1905.11485v1.pdf.
62 Assun??o L R C, Mendes P A S, Matos S, et al. Technology roadmap of renewable natural gas: identifying trends for research and development to improve biogas upgrading technology management[J]. Applied Energy, 2021, 292: 116849.
63 Yu X, Zhang B. Obtaining advantages from technology revolution: a patent roadmap for competition analysis and strategy planning[J]. Technological Forecasting and Social Change, 2019, 145: 273-283.
64 Kim J, Geum Y. How to develop data-driven technology roadmaps: the integration of topic modeling and link prediction[J]. Technological Forecasting and Social Change, 2021, 171: 120972.
65 Jiang W W, Luo J Y. Graph neural network for traffic forecasting: a survey[J]. Expert Systems with Applications, 2022, 207: 117921.
66 Hisano R. Semi-supervised graph embedding approach to dynamic link prediction[C]// Proceedings of the International Workshop on Complex Networks. Cham: Springer, 2018: 109-121.
67 Zhu J, Xie Q, Chin E J. A hybrid time-series link prediction framework for large social network[C]// Proceedings of the International Conference on Database and Expert Systems Applications. Heidelberg: Springer, 2012: 345-359.
68 Moradabadi B, Meybodi M R. A novel time series link prediction method: learning automata approach[J]. Physica A: Statistical Mechanics and Its Applications, 2017, 482: 422-432.
69 Box G E P, Jenkins G M, Reinsel G C, et al. Time series analysis: forecasting and control[M]. New York: John Wiley & Sons, 2015.
70 Rabanser S, Shchur O, Günnemann S. Introduction to tensor decompositions and their applications in machine learning[OL]. (2017-11-29). https://arxiv.org/pdf/1711.10781.pdf.
71 Zuo Y, Liu G N, Lin H, et al. Embedding temporal network via neighborhood formation[C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM Press, 2018: 2857-2866.
72 Xiong L, Chen X, Huang T K, et al. Temporal collaborative filtering with Bayesian probabilistic tensor factorization[C]// Proceedings of the 2010 SIAM International Conference on Data Mining. Philadelphia: Society for Industrial and Applied Mathematics, 2010: 211-222.
73 Yu W C, Cheng W, Aggarwal C C, et al. Link prediction with spatial and temporal consistency in dynamic networks[C]// Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, 2017: 3343-3349.
74 Xu M K, Luo S T, Bengio Y, et al. Learning neural generative dynamics for molecular conformation generation[OL]. (2021-03-31). https://arxiv.org/pdf/2102.10240.pdf.
75 朱旭振. 基于链路预测的推荐系统: 原理、模型与算法[M]. 北京: 北京邮电大学出版社, 2018.
76 Liben-Nowell D, Kleinberg J. The link-prediction problem for social networks[J]. Journal of the American Society for Information Science and Technology, 2007, 58(7): 1019-1031.
77 Kennedy J, Eberhart R. Particle swarm optimization[C]// Proceedings of the International Conference on Neural Networks. Piscataway: IEEE, 1995: 1942-1948.
78 游丹丹, 陈福集. 基于改进粒子群和BP神经网络的网络舆情预测研究[J]. 情报杂志, 2016, 35(8): 156-161.
79 贾承丰, 韩华, 吕亚楠, 等. 基于Word2Vec和粒子群的链路预测算法[J]. 自动化学报, 2020, 46(8): 1703-1713.
80 刘静秋, 杜文莉, 张飞. 基于粒子群算法的函数复杂度分类法[J]. 控制工程, 2020, 27(8): 1337-1345.
81 王双月, 罗自炎. 一类基于L0/1软间隔损失函数的低秩支持张量机[J]. 运筹学学报, 2021, 25(3): 160-172.
82 Gordon M S, Humble J, Rumbell T, et al. Natural language processor for using speech to cognitively detect and analyze deviations from a baseline: US2020/0219529A1[P]. 2020-07-09.
83 Lee E, Cunnington D T, Chiarella G G, et al. Distinguishing voice commands: WO2021/033088A1[P]. 2021-02-25.
84 王小霞, 傅惠, 杨马进, 等. 动态交通数据丢失的补齐方法研究[J]. 内蒙古农业大学学报(自然科学版), 2016, 37(3): 93-96. |
|
|
|
