Original link: https://qq52o.me/2818.html
This article explains the method of crawling CSDN author’s articles.
Taking Kaiyuanshe as an example, the homepage of CSDN is: https://blog.csdn.net/kaiyuanshe , username is kaiyuanshe .
You can see that there are 1167 articles, and the creation process is distinguished by year and month, and it supports sorting by last release time and visits.
Open the browser developer tools, view Network , find Fetch/XHR , refresh the page and find that there will be some requests, and use home-api to filter.
After viewing the Response, you will find that these interfaces return some basic data and links of the article.
1. Creation process
https://blog.csdn.net/community/home-api/v1/get-place-on-file?username=kaiyuanshe
This interface can obtain links corresponding to the number of published articles and views per month.
{ "code": 200, "message": "success", "traceId": "869fa273-e85c-45d3-bcee-92db2dc59e50", "data": [ { "year": "2023", "months": [ { "url": "https://blog.csdn.net/kaiyuanshe?type=blog&year=2023&month=06", "count": 23, "month": "06" }, { "url": "https://blog.csdn.net/kaiyuanshe?type=blog&year=2023&month=05", "count": 18, "month": "05" }, { "url": "https://blog.csdn.net/kaiyuanshe?type=blog&year=2023&month=04", "count": 25, "month": "04" }, { "url": "https://blog.csdn.net/kaiyuanshe?type=blog&year=2023&month=03", "count": 20, "month": "03" }, { "url": "https://blog.csdn.net/kaiyuanshe?type=blog&year=2023&month=02", "count": 17, "month": "02" }, { "url": "https://blog.csdn.net/kaiyuanshe?type=blog&year=2023&month=01", "count": 16, "month": "01" } ], "count": 119, "currentYear": true } ] }
2. Article list
The default is to sort by the last release time, so the interface you see is:
After clicking Sort by Visits:
After clicking on the year and month in the authoring history:
in conclusion:
https://blog.csdn.net/community/home-api/v1/get-business-list?page={用页数}&size={分页数}&businessType=blog&orderby=&noMore=false&year={某年}&month={某月}&username={用户名}
After replacing the above variables, you can use this address to get: article ID, title, description, URL of article details, whether it is original, whether it is pinned, page views, number of comments, release time, cover image.
{ "code": 200, "message": "success", "traceId": "98eef2bc-cea8-4bb5-a0b6-cd2746342a28", "data": { "list": [ { "articleId": 123564439, "title": "官宣：开源社-开源战略研究组（ONES Group）正式成立~", "description": "| 作者：开源社-开源战略研究组（ONES Group）| 设计：王福政| 责编：李明康2022年3月11日开源社隆重宣布：开源战略研究组（ONESGroup）正式成立！这个研...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/123564439", "type": 1, "top": true, "forcePlan": false, "viewCount": 437, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/123564439", "postTime": "2022-03-11 21:00:00", "diggCount": 2, "formatTime": "2022.03.11", "picList": [ "https://img-blog.csdnimg.cn/img_convert/96e32ad402766811705556917a22b442.png" ], "collectCount": 2 }, { "articleId": 123568306, "title": "2021 开源社年度报告：开心开源", "description": "# 引言#2020年的开源社年度报告仿佛还在昨天，一转眼，2021年都已经过完了。在去年的年度报告上，我们说2020是动荡不安的一年。结果2021年，简直可以说是动荡加剧，令人应接不暇的...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/123568306", "type": 1, "top": true, "forcePlan": false, "viewCount": 161, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/123568306", "postTime": "2022-01-31 12:00:00", "diggCount": 0, "formatTime": "2022.01.31", "picList": [ "https://img-blog.csdnimg.cn/img_convert/504e177ee3c47d36f6fca1b68bfd722f.png" ], "collectCount": 0 }, { "articleId": 123568239, "title": "官宣:2021中国开源年报震撼首发~一篇报告带你读懂中国开源的2021~", "description": "| 封面：周颖| 责编：李明康开源，在2021年，火了，而且是燎原烈火，烧的焮天铄地，可以说是中国开源迅猛发展的元年也不为过。我们看到，在开源领域上，国策的重点扶持、资本的加码青睐、社区的...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/123568239", "type": 1, "top": true, "forcePlan": false, "viewCount": 825, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/123568239", "postTime": "2022-01-28 12:05:57", "diggCount": 0, "formatTime": "2022.01.28", "picList": [ "https://img-blog.csdnimg.cn/img_convert/844e67778c974a0cb56e8e68f6043d85.png" ], "collectCount": 0 }, { "articleId": 102699194, "title": "Apache 顶级开源大神加入COSCon'19 豪华讲师阵容", "description": "Apache 顶级开源大神加入COSCon'19 豪华讲师阵容开源社2019-10-22 12:28:39业界最具影响力的开源年度盛会2019中国开源年会(COSCon'19) 将于11月2-3日在华东师范大学(上海普陀区中山北路校区) 由开源社举办。COSCon 以其独特定位及日益增加的影响力，吸引越来越多的顶级企业和国际基金会的大力支持。COSCon 的独特定位就是成...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/102699194", "type": 1, "top": true, "forcePlan": false, "viewCount": 1412, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/102699194", "postTime": "2019-10-23 12:37:26", "diggCount": 0, "formatTime": "2019.10.23", "picList": [], "collectCount": 0 }, { "articleId": 131485700, "title": "中国开源发展所面临的几个问题和挑战", "description": "“如果你有正确的态度，有趣的事情自然会找到你。” ——埃里克·雷蒙德最近一段时间，开源很热闹，从GOTC大会，开放原子全球开源峰会，再到COPU第十八届开源中国开源世界高峰论坛,大会一场接一场。预计下半年还有一系列接踵而来的开源大会，足以让各路开源人乐此不疲。这些热闹的背后，我们既看到了国家对开源战略的重视，也看到了社会各界对开源的关注。而对长期以来默默耕耘的开源人来说，无疑是件好事...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131485700", "type": 2, "top": false, "forcePlan": false, "viewCount": 4, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131485700", "postTime": "2023-06-30 21:01:00", "diggCount": 0, "formatTime": "19 小时前", "picList": [], "collectCount": 0 }, { "articleId": 131485701, "title": "云原生活动预告｜Harbor & Narrows & Velero开源项目技术沙龙报名开启！", "description": "7月15日，由VMware主办，开源社、融科资讯中心协办，聚焦于Harbor、Narrows、Velero三个开源项目的技术分享活动将与大家见面。本次活动汇集了来自VMware、蚂蚁集团、微软、骥步科技、雅克云的多位云原生领域技术专家现场分享全栈式的开源项目讲解和云原生应用的落地与实践，希望您能通过本次云原生技术沙龙获得更多灵感和思路。本次活动干货满满，欢迎北京的朋友们免费报名线下参会，期待您的...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131485701", "type": 2, "top": false, "forcePlan": false, "viewCount": 2, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131485701", "postTime": "2023-06-30 21:01:00", "diggCount": 0, "formatTime": "19 小时前", "picList": [], "collectCount": 0 }, { "articleId": 131447885, "title": "在开源经历中成长，让开发者成为创新主体", "description": "本文背景积极推进“满天星”行动计划，促进开发者成长，重庆软件园产业学院“成长营”将聚焦“技术提升”“职业成长”“价值提升”等话题开展系列主题活动，赋能软件技术开发者更好发展、更快成长。本次直播访谈，聚焦在开源经历中成长，让开发者成为创新主体。本文为开源社KCC@成都站站长程诗杰个人分享的内容片段摘要，就从“如何帮助开发者成长” “如何做好开源社区”“如何成功举办一场开源活动”“如何推动开发者在成...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131447885", "type": 1, "top": false, "forcePlan": false, "viewCount": 330, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131447885", "postTime": "2023-06-28 21:02:58", "diggCount": 0, "formatTime": "2023.06.28", "picList": [], "collectCount": 0 }, { "articleId": 131407901, "title": "SCA技术进阶系列（二）：代码同源检测技术在供应链安全治理中的应用", "description": "随着“数字中国”建设的不断提速，企业在数字化转型的创新实践中不断加大对开源技术的应用，引入开源组件完成应用需求开发已经成为了大多数研发工程师开发软件代码的主要手段。随之而来的一个痛点问题是：绝大多数的应用程序都包含开源组件风险。因而，能够帮助管理和降低开源组件风险的SCA技术应运而生。常规的SCA软件成分分析工具可以通过分析组件版本及依赖，完成对引用的三方开源组件的检查，从而识别已知的组件漏洞以及...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131407901", "type": 2, "top": false, "forcePlan": false, "viewCount": 16, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131407901", "postTime": "2023-06-26 21:01:02", "diggCount": 0, "formatTime": "2023.06.26", "picList": [], "collectCount": 0 }, { "articleId": 131388499, "title": "开源社KCC@硅谷正式成立，搭建国际开源交流平台", "description": "大家好！我很高兴向大家宣布一个重要的消息：开源社在硅谷的KCC（Kaiyuanshe City Community）正式成立了！作为开源社的一项重要举措，KCC@硅谷将成为国际开源交流的桥梁，架起中国和全球开源社区之间的纽带。2023年以来，开源社观察到，疫情过后，开源社区的成员们渴望恢复面对面的交流和聚会，这对于长期在线上运营、远程协同工作的开源社区来说，是一个好消息，线下面对面的交流有助于加强...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131388499", "type": 1, "top": false, "forcePlan": false, "viewCount": 440, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131388499", "postTime": "2023-06-25 21:00:49", "diggCount": 0, "formatTime": "2023.06.25", "picList": [], "collectCount": 0 }, { "articleId": 131356038, "title": "国产开源社群的运营，为何总是画风奇特？", "description": "在过去几年的投入和关注下，国产开源社群如雨后春笋一般冒了出来。今天，以GPT 为首的AI 新势力接过话题度的接力棒，我们可以在降温周期里回顾一下过去几年间冒出来的国产开源社群都有什么样的成绩，有些什么样共性的问题可以改进。本文标题是国产开源社群的运营总是画风奇特，这也符合近几年来作为理论上开源社群的主要参与者，即软件开发者们的主流感受。从眼花缭乱的开源营销，到一次性开源、开关式开源的另类创新，...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131356038", "type": 2, "top": false, "forcePlan": false, "viewCount": 18, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131356038", "postTime": "2023-06-23 21:02:52", "diggCount": 0, "formatTime": "2023.06.23", "picList": [], "collectCount": 0 }, { "articleId": 131345919, "title": "粽子拍了拍你说“端午安康”", "description": "时临仲夏，岁至端午，开源社祝大家端午安康！", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131345919", "type": 2, "top": false, "forcePlan": false, "viewCount": 24, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131345919", "postTime": "2023-06-22 10:30:11", "diggCount": 0, "formatTime": "2023.06.22", "picList": [], "collectCount": 0 }, { "articleId": 131335844, "title": "如何加入开源社", "description": "开源社成立于2014 年，是由志愿贡献于开源事业的个人成员，依“贡献、共识、共治” 原则所组成，始终维持厂商中立、公益、非营利的特点，是最早以“开源治理、国际接轨、社区发展、项目孵化” 为使命的开源社区联合体。开源社积极与支持开源的社区、企业以及政府相关单位紧密合作，以“立足中国、贡献全球，推动开源成为新时代的生活方式” 为愿景，旨在共创健康可持续发展的开源生态，推动中国开源社区成为全球开...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131335844", "type": 1, "top": false, "forcePlan": false, "viewCount": 781, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131335844", "postTime": "2023-06-21 12:01:18", "diggCount": 0, "formatTime": "2023.06.21", "picList": [], "collectCount": 0 }, { "articleId": 131297842, "title": "2022 开源社年度报告：打开新世界", "description": "2022 年被认为是充满挑战的一年，对于开源社而言也不例外。开源社同样面临着外部环境的不断变化，然而，正是开源的力量和开放协作的模式，让我们能够在这个极端情况下做出卓越的回应。大家对于开源社的组织架构充满好奇，想知道我们是如何以公开透明、 自我管理和创新进化的方式运作的。通过阅读开源社年度报告，你会看到我们的组织架构就像一幅精心设计的水利工程的设计图，当水流经过时， 每一滴水都能自然流淌到能够发挥...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131297842", "type": 1, "top": false, "forcePlan": false, "viewCount": 78, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131297842", "postTime": "2023-06-19 21:00:47", "diggCount": 0, "formatTime": "2023.06.19", "picList": [], "collectCount": 0 }, { "articleId": 131278293, "title": "媒体组招募令| 开源社志愿者开始招募啦！", "description": "作者丨开源社媒体组编辑丨李楠相关阅读|RelatedReading❈ 2023年开源社项目委员会介绍❈ 翻译组招募| open source translators❈ 成员发展组招募令| 热情、细心，我们要找的就是你！开源社简介开源社成立于2014 年，是由志愿贡献于开源事业的个人成员，依“贡献、共识、共治”原则所组成，始终维持厂商中立、公益、非营利的特点，是最早以“开源治理、国...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131278293", "type": 1, "top": false, "forcePlan": false, "viewCount": 21, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131278293", "postTime": "2023-06-18 21:00:02", "diggCount": 0, "formatTime": "2023.06.18", "picList": [], "collectCount": 0 }, { "articleId": 131255489, "title": "助力开源商业，2023中关村科学城科创大赛报名中→", "description": "这是一个充满着无限可能的时代也是一个需要创新者不断探索的时代“互联世界、共链未来”中关村科学城科创大赛正式启动全球招募诚邀您共赴前沿盛宴让我们共同寻找下一个改变者点击阅读原文：开源社专属报名渠道：https://www.micecube.com/sign/4lMUwR/kys邀请码：kys欢迎大家报名！转载自丨中关村创业会客厅编辑丨肖钰雯相关阅读| Related Reading开源反分裂与数字...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131255489", "type": 2, "top": false, "forcePlan": false, "viewCount": 18, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131255489", "postTime": "2023-06-16 21:01:59", "diggCount": 0, "formatTime": "2023.06.16", "picList": [], "collectCount": 0 }, { "articleId": 131255492, "title": "KCC@北京启动，为一个城市做好开源！", "description": "KCC（开源社城市社区）北京区在6月12日正式成立，吸引了众多开源爱好者和技术开发者参与。现场举行的活动围绕“为一个城市做好开源”的话题展开讨论，包括开发者关系、开源背后的无用之用、开源在中国的变化、开源的困难与挑战以及个人能为开源做出的贡献等。在讨论中，与会者对开发者关系进行了深入探讨。开发者关系是指开源项目与开发者之间的互动和合作关系，包括了建立友好的沟通渠道、提供支持和培训等方面。与会者...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131255492", "type": 1, "top": false, "forcePlan": false, "viewCount": 915, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131255492", "postTime": "2023-06-16 21:01:59", "diggCount": 0, "formatTime": "2023.06.16", "picList": [], "collectCount": 0 }, { "articleId": 131218676, "title": "开源反分裂与数字大同世界", "description": "这是一篇报告的读后感，这篇报告是由Linux基金会研究部门发布的，名为《助力全球协作——开源代码的领导者如何面对分裂的挑战》。这份报告的主要内容包括：- 开源代码开发中的分裂及其利弊- 开源的国际化，以及项目领导者如何克服参与开源社区的潜在障碍- 技术民族主义对开源协作的影响，以及减少区域分裂风险的策略- 开源治理的现状，以及如何增加开源基金会之间的合作- 关键的发现和建议这份报告的发布背景是在当...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131218676", "type": 1, "top": false, "forcePlan": false, "viewCount": 1213, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131218676", "postTime": "2023-06-14 21:02:37", "diggCount": 0, "formatTime": "2023.06.14", "picList": [], "collectCount": 0 }, { "articleId": 131179463, "title": "我理解的参与开源贡献", "description": "坐标上海的E人，还在新手村练级的前端开发。曾闯荡过教育培训和零售管理行业，经历过大公司、外企，也和朋友的工作室合作过，斩获过一些能吓唬人的标签。好奇心很足，精力也很充沛，代码世界就是我现实生活中的霍格沃茨。虽然转码不易，但充满乐趣。—— PapayaHUANG在加入开源社一个多月，经历了几次例会，贡献过几个简单的PR，和不同的前辈交流，也浅带过其他新人之后，我收集了很多信息，提取了一些通用信息...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131179463", "type": 1, "top": false, "forcePlan": false, "viewCount": 460, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131179463", "postTime": "2023-06-12 21:00:13", "diggCount": 0, "formatTime": "2023.06.12", "picList": [], "collectCount": 0 }, { "articleId": 131179465, "title": "GLCC2023学生报名现已开放，超多课题等你来挑战| 来CCF GLCC掌握编程密码，一起开启开源世界大门！...", "description": "6月4日，第二届CCF确实开源编程夏令营（GLCC）正式开启学生报名通道，GLCC 2023收到了来自之江实验室、百度、字节跳动、蚂蚁集团、微众银行及OpenMMLab等40多个企业/社区的100多个课题，覆盖人工智能、操作系统、云原生及大数据等30 多个技术领域，奖金丰厚，等你来拿！奖金丰厚，等你来挑战！点击“阅读原文”，了解更多关于GLCC详情！转载自丨中国计算机学会编辑丨邵佳怡相关阅读|...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131179465", "type": 2, "top": false, "forcePlan": false, "viewCount": 37, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131179465", "postTime": "2023-06-12 21:00:13", "diggCount": 0, "formatTime": "2023.06.12", "picList": [], "collectCount": 0 }, { "articleId": 131136889, "title": "开源社章程（2023版）", "description": "第1 条开源社是由志愿贡献于开源事业的个人志愿者，依“贡献、共识、共治” 原则所组成的开源社区。第2 条开源社的英文名称为“KAIYUANSHE”，官方网站地址为https://kaiyuanshe.cn/第3 条开源社的愿景为：立足中国、贡献全球，推动开源成为新时代的生活方式。第4 条开源社的使命为：开源治理、国际接轨、社区发展、项目孵化，携手国内社区、高校、企业和政府，共创...", "url": "https://blog.csdn.net/kaiyuanshe/article/details/131136889", "type": 1, "top": false, "forcePlan": false, "viewCount": 790, "commentCount": 0, "editUrl": "https://mp.csdn.net/console/editor/html/131136889", "postTime": "2023-06-09 21:01:50", "diggCount": 0, "formatTime": "2023.06.09", "picList": [], "collectCount": 0 } ], "total": 1167 } }
So far, we have obtained most of the information. If you still need to obtain the details of the article, visit the corresponding url and crawl the HTML.
This article is transferred from: https://qq52o.me/2818.html
This site is only for collection, and the copyright belongs to the original author.