开始监听网页请求与结果
2024-04-08 10:16:13 373
功能介绍
监听浏览器打开网页时所发出的请求跟响应结果。能够抓取请求url、请求headers、请求body、响应headers、响应body。
监听结果以json对象的形式返回。监听通过url关键字进行匹配。符合条件的url将会被设置到监听结果中。
结果保存在变量中，以数组的形式保存。
标签页对象
选择一个之前通过【启动浏览器】或【创建新标签页】或【获取已打开标签页对象】流程创建的标签页对象
筛选资源类型
选择要获取的网页监听结果类型,可筛选JS,图片,文档等类型
匹配url方式
文本查找---通过文本关键字的方式进行匹配url链接
正则匹配---通过正则表达式的方式进行匹配url链接
全部获取---获取所有结果
规则或url
如果是文本查找的方式，填写完整的url链接或者关键字，例如 ：jquery.js
如果是正则匹配的方式，请填写正则表达式，例如：\S+jquery\.js
将资源直接保存到本地目录
当筛选的资源为JS、CSS、图片、媒体、字体时，并且勾选了当前选项后，可以将资源直接保存到本地磁盘中。实现对图片等资源的快速采集。
保存资源目录
要保存资源的目录。注意，目录必须已经创建好，否则会保存失败。
文件名
资源的文件名，如demo.png。也可以不填，不填默认按url的名称进行命名
监听结果保存至
保存监听到的json对象。json对象完整数据如下：
[{'frameid': '5DFCF841FBEB1D8E18E16570D020EB05', 'req_method': 'GET', 'request': {'headers': {'Accept': 'application/json, text/javascript, */*; q=0.01', 'Referer': 'https://news.baidu.com/', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36', 'X-Requested-With': 'XMLHttpRequest', 'sec-ch-ua': '"Chromium";v="107"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"Windows"'}, 'initialPriority': 'High', 'isSameSite': True, 'method': 'GET', 'mixedContentType': 'none', 'referrerPolicy': 'strict-origin-when-cross-origin', 'url': 'https://news.baidu.com/widget?id=LocalNews&ajax=json&t=1717409069738'}, 'respone': {'connectionId': 32, 'connectionReused': True, 'content': '{"errno":0,"request_id":"3080944338","timestamp":1717409073,"data":{"LocalNews":{"errno":0,"data":{"cityid":3373,"name":"\\u798f\\u5dde","rows":{"pic":{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800823411919977002","title":"\\u798f\\u5efa\\u9996\\u6b21\\u56fd\\u9645\\u9a6c\\u672f\\u8d5b\\u5728\\u7405\\u5c90\\u4e3e\\u884c","time":"15:17","imgUrl":"https:\\/\\/t12.baidu.com\\/it\\/u=4116889519,236725629&fm=30&app=106&f=JPEG?w=312&h=208&s=DA922FC049433EC00201C41D0300F0C3"},"first":[{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800520603097899704","title":"\\u798f\\u5efa\\u201c\\u6d77\\u4e0a\\u7267\\u573a\\u201d\\u851a\\u4e3a\\u58ee\\u89c2","time":"07:05","imgUrl":"http:\\/\\/t12.baidu.com\\/it\\/u=3485164233,236443616&fm=30&app=106&f=JPEG?w=312&h=208&s=E9A48F541EAB50053A8889D0030040BB"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800809759432230025","title":"\\u521a\\u521a\\u901a\\u77e5\\uff01\\u798f\\u5dde\\u5730\\u94c1\\u514d\\u8d39\\uff01\\u9762\\u5411\\u8fd9\\u4e9b\\u4eba","time":"11:40","imgUrl":"http:\\/\\/t11.baidu.com\\/it\\/u=3049340520,236712914&fm=30&app=106&f=JPEG?w=312&h=208&s=FD907B954DD778DA061DCCC00300E0B1"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800822342573465595","title":"\\u798f\\u5efa\\u7701\\u5e02\\u573a\\u76d1\\u7763\\u7ba1\\u7406\\u5c402024\\u5e74\\u7b2c17\\u671f\\u98df\\u54c1\\u5b89\\u5168...","time":"15:03","imgUrl":"http:\\/\\/t10.baidu.com\\/it\\/u=2109257245,236724751&fm=30&app=106&f=JPEG?w=312&h=208&s=7002BE1A94E84D0160F739CC0300C0BB"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800833099849509553","title":"\\u798f\\u5efa\\u7701\\u7ea2\\u5341\\u5b57\\u4f1a\\u60a6\\u57fa\\u91d1\\u6551\\u52a9\\u91d1\\u53d1\\u653e","time":"17:51","imgUrl":"http:\\/\\/t11.baidu.com\\/it\\/u=4175490006,236734650&fm=30&app=106&f=JPEG?w=312&h=208&s=77089D474D32549E81A9017803008070"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800528436828649331","title":"\\u5750\\u6807\\u798f\\u5dde\\uff01\\u4ece\\u201c1\\u4e2a\\u4eba\\u30011\\u4e32\\u94a5\\u5319\\u30013\\u4e2a\\u79d1\\u5ba4\\u201d\\u9006\\u88ad...","time":"09:09","imgUrl":"http:\\/\\/t10.baidu.com\\/it\\/u=311637426,236450918&fm=30&app=106&f=JPEG?w=312&h=208&s=C39B71852E1A4DCC4AAD21D2030070B0"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800554946457808787","title":"\\u798f\\u5efa\\u53ec\\u5f00\\u5168\\u7701\\u8b66\\u793a\\u6559\\u80b2\\u4f1a \\u8981\\u6c42\\u4ee5\\u6848\\u793a\\u8b66\\u3001\\u4ee5\\u6848\\u4e3a\\u9274","time":"16:11","imgUrl":"http:\\/\\/t12.baidu.com\\/it\\/u=2651842638,236475600&fm=30&app=106&f=JPEG?w=312&h=208&s=8C70538706C8094F5D1AC8BB03008013"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800830898021937432","title":"\\u83b7\\u5956\\u540d\\u5355\\u516c\\u5e03\\uff01\\u606d\\u559c\\u798f\\u5efa\\u8fd9\\u4e9b\\u5e08\\u751f\\uff01","time":"17:17","imgUrl":"http:\\/\\/t11.baidu.com\\/it\\/u=334119119,236732607&fm=30&app=106&f=JPEG?w=312&h=208&s=5CA8347287A34922185DB0DB0000E0B1"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800753137155193596","title":"188\\u5bb6\\uff01\\u798f\\u5efa\\u7701\\u521b\\u65b0\\u578b\\u4e2d\\u5c0f\\u4f01\\u4e1a\\u540d\\u5355\\u516c\\u5e03","time":"20:40","imgUrl":"http:\\/\\/t11.baidu.com\\/it\\/u=1525643625,236660179&fm=30&app=106&f=JPEG?w=312&h=208&s=0D287432191FC0CC185D94CA0000A0B1"}],"second":[{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800807809308657135","title":"\\u798f\\u5efa\\u542f\\u52a82024\\u5e74\\u201c\\u516d\\u4e00\\u201d\\u513f\\u7ae5\\u5173\\u7231\\u670d\\u52a1\\u4e3b\\u9898\\u6d3b\\u52a8","time":"11:09","imgUrl":"http:\\/\\/t12.baidu.com\\/it\\/u=3377135832,236711098&fm=30&app=106&f=JPEG?w=312&h=208&s=F7C0F3BA424078EE1C10D560030060F1"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800665277100392206","title":"\\u516c\\u793a\\uff01\\u798f\\u5efa\\u8fd9\\u4e9b\\u8003\\u751f\\u62df\\u83b72024\\u5e74\\u9ad8\\u62db\\u5f55\\u53d6\\u7167\\u987e\\u8d44\\u683c","time":"21:26","imgUrl":"http:\\/\\/t12.baidu.com\\/it\\/u=1248434145,236578358&fm=30&app=106&f=JPEG?w=312&h=208&s=4ACA7223A8DD51EB1ADD85D2000080B1"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800464548591885268","title":"\\u80e1\\u6210\\u6b66\\uff1a\\u949f\\u7231\\u5730\\u672f\\u62f3\\u7684\\u201c\\u6d77\\u897f\\u9556\\u5934\\u201d","time":"16:13","imgUrl":"http:\\/\\/t11.baidu.com\\/it\\/u=3955320237,236391410&fm=30&app=106&f=JPEG?w=312&h=208&s=F1521E9D033061884668D1DB03005031"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800792647910974073","title":"\\u63fd\\u5c71\\u89c2\\u6d77\\u63a2\\u6751\\uff01\\u5e73\\u6f6d\\u51e4\\u51f0\\u6e7e\\u5065\\u8eab\\u6b65\\u9053\\u542f\\u7528","time":"07:08","imgUrl":"http:\\/\\/t11.baidu.com\\/it\\/u=1425940007,236696979&fm=30&app=106&f=JPEG?w=312&h=208&s=B0135E90160F7F4D72C358410300B0B8"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800654644592020515","title":"\\u798f\\u5efa\\u4e8c\\u5efa\\u96c6\\u56e22024\\u5e74\\u201c\\u5b89\\u5168\\u751f\\u4ea7\\u6708\\u201d\\u542f\\u52a8","time":"18:35","imgUrl":"http:\\/\\/t10.baidu.com\\/it\\/u=1936941015,236568458&fm=30&app=106&f=JPEG?w=312&h=208&s=70535A968479483316A26BD9030030BB"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800741197999394613","title":"\\u201c\\u7f57\\u6e90\\u5de5\\u201d\\u56de\\u5f52\\uff0c\\u91cd\\u632f\\u7389\\u77f3\\u96d5\\u523b\\u4ea7\\u4e1a","time":"17:30","imgUrl":"http:\\/\\/t10.baidu.com\\/it\\/u=3871809022,236649061&fm=30&app=106&f=JPEG?w=312&h=208&s=797BA357CA5062CE4B9DC8E20300B033"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800832943156049579","title":"\\u6d89\\u53ca\\u798f\\u5efa\\u591a\\u4e2a\\u8f66\\u6b21\\uff01\\u7aef\\u5348\\u5047\\u671f\\u5357\\u94c1\\u589e\\u5f001366\\u5217\\u65c5...","time":"17:49","imgUrl":"http:\\/\\/t11.baidu.com\\/it\\/u=2834011146,236734511&fm=30&app=106&f=JPEG?w=312&h=208&s=FA04618C04EE94EB4C95D05E0300C090"},{"url":"http:\\/\\/baijiahao.baidu.com\\/s?id=1800805354388807871","title":"\\u798f\\u5dde\\u8fd9\\u7247\\u6d77\\u8d8a\\u6765\\u8d8a\\u5bbd\\u5e7f 2024\\u6e14\\u4e1a\\u5468\\u00b7\\u6e14\\u535a\\u4f1a\\u5706...","time":"10:30","imgUrl":"http:\\/\\/t12.baidu.com\\/it\\/u=4015633014,236708816&fm=30&app=106&f=JPEG?w=312&h=208&s=B0B393BA4402EFF9CAA3CA840300209F"}]}},"ad":null}}}', 'encodedDataLength': 465, 'fromDiskCache': False, 'fromPrefetchCache': False, 'fromServiceWorker': False, 'headers': {'Connection': 'keep-alive', 'Content-Encoding': 'br', 'Content-Length': '1834', 'Content-Type': 'application/json;charset=utf-8', 'Date': 'Mon, 03 Jun 2024 10:04:33 GMT', 'Server': 'nginx/1.9.1', 'Tracecode': '30809443383900232714060318', 'X-Bd-Api': 'news_widget_localnews', 'X-Bd-Status': '200', 'X-Logic-No': 'null'}, 'mimeType': 'application/json', 'protocol': 'http/1.1', 'remoteIPAddress': '14.215.183.213', 'remotePort': 443, 'responseTime': 1717409071749.122, 'securityDetails': {'certificateId': 0, 'certificateTransparencyCompliance': 'unknown', 'cipher': 'AES_128_GCM', 'encryptedClientHello': False, 'issuer': 'GlobalSign RSA OV SSL CA 2018', 'keyExchange': 'ECDHE_RSA', 'keyExchangeGroup': 'X25519', 'protocol': 'TLS 1.2', 'sanList': ['baidu.com', 'baifubao.com', 'www.baidu.cn', 'www.baidu.com.cn', 'mct.y.nuomi.com', 'apollo.auto', 'dwz.cn', '*.baidu.com', '*.baifubao.com', '*.baidustatic.com', '*.bdstatic.com', '*.bdimg.com', '*.hao123.com', '*.nuomi.com', '*.chuanke.com', '*.trustgo.com', '*.bce.baidu.com', '*.eyun.baidu.com', '*.map.baidu.com', '*.mbd.baidu.com', '*.fanyi.baidu.com', '*.baidubce.com', '*.mipcdn.com', '*.news.baidu.com', '*.baidupcs.com', '*.aipage.com', '*.aipage.cn', '*.bcehost.com', '*.safe.baidu.com', '*.im.baidu.com', '*.baiducontent.com', '*.dlnel.com', '*.dlnel.org', '*.dueros.baidu.com', '*.su.baidu.com', '*.91.com', '*.hao123.baidu.com', '*.apollo.auto', '*.xueshu.baidu.com', '*.bj.baidubce.com', '*.gz.baidubce.com', '*.smartapps.cn', '*.bdtjrcv.com', '*.hao222.com', '*.haokan.com', '*.pae.baidu.com', '*.vd.bdstatic.com', '*.cloud.baidu.com', 'click.hm.baidu.com', 'log.hm.baidu.com', 'cm.pos.baidu.com', 'wn.pos.baidu.com', 'update.pan.baidu.com'], 'serverSignatureAlgorithm': 2052, 'signedCertificateTimestampList': [], 'subjectName': 'baidu.com', 'validFrom': 1688608266, 'validTo': 1722909065}, 'securityState': 'secure', 'status': 200, 'statusText': 'OK', 'timing': {'connectEnd': -1, 'connectStart': -1, 'dnsEnd': -1, 'dnsStart': -1, 'proxyEnd': -1, 'proxyStart': -1, 'pushEnd': 0, 'pushStart': 0, 'receiveHeadersEnd': 2007.132, 'requestTime': 611387.282224, 'sendEnd': 0.65, 'sendStart': 0.528, 'sslEnd': -1, 'sslStart': -1, 'workerFetchStart': -1, 'workerReady': -1, 'workerRespondWithSettled': -1, 'workerStart': -1}, 'url': 'https://news.baidu.com/widget?id=LocalNews&ajax=json&t=1717409069738'}}]
其中列表项包含:
'req_method'(请求方式POST、GET)
'url'(发起请求的url)
'request' （请求对象,里面还包含列表项:'headers'(请求头部)、如果是post方式还包含'post_param'(post参数)）
'respone'(响应对象,里面还包含列表项:'status'(响应状态码)、'headers'(响应头部)、'content'(响应内容))
使用参考范例：
