Fetch Resource
Last update：2022-10-14 15:24:35
Fetch Resources
Description
This interface provides fetching resources from the specified URL and storing them in the specified space.
Request Description
POST /fmgr/fetch
HOST: <MgrDomain>
Authorization:<accessToken>
Header Description
| Parameter | Required | Description |
|---|---|---|
| HOST | Yes | Management domain name , which can be obtained in the user management interface |
| Authorization | Yes | Manage Credentials |
Parameter Description
Request parameters are organized in the following format and submitted as request content:
fops=fetchURL/<Urlsafe_Base64_Encoded_URL>/bucket/<Urlsafe_Base64_Encoded_bucket>/key/<Urlsafe_Base64_Encoded_key>/prefix/<Urlsafe_Base64_Encoded_prefix>/md5/<md5>/decompression/<decompression>/fetchTS/<fetchTS>¬ifyURL=<Urlsafe_Base64_Encoded_notifyUrl>&force=<Force>&separate=<Separate>
| Parameter | Required | Description |
|---|---|---|
| fops | Yes | The processing parameter list, the format is as Descriptiond above, and multiple processing requests are supported at the same time. The parameter lists are separated by ;. |
| fetchURL | Yes | Specifies the crawl URL. <Urlsafe_Base64_Encoded_URL> is the URL-safe Base64-encoded string for the specified URL. |
| bucket | Yes | Specify storage space. <Urlsafe_Base64_Encoded_bucket> is the URL-safe Base64-encoded string for the specified bucket name. |
| key | no | Specifies the key to store the resource. <Urlsafe_Base64_Encoded_key> is the URL-safe Base64-encoded string for the key of the specified resource. Note: When no key is specified, the key of the resource is the URI of the fetched URL . |
| prefix | no | Specifies the prefix for the resource key. <Urlsafe_Base64_Encoded_prefix> is a URL-safe Base64-encoded string for the prefix of the key of the specified resource. Note: When specifying the prefix parameter, the resource is finally saved as prefix/key . |
| md5 | no | Specifies the md5 value of the resource. If the md5 value of the resource captured by the specified URL is consistent with the specified md5 value, it will be saved, otherwise it will not be saved. If it is not specified, the md5 value will not be checked and saved directly. |
| decompression | no | Decompress the pulled files of a specified type, and support zip/tar/gzip . For example, configure /decompression/zip to unzip the pulled files. |
| fetchTS | no | When grabbing m3u8 files, specify whether to grab TS files: 0-means not grabbing TS files 1-represents grabbing TS files* Note: <1>If this parameter is not set, the TS files will not be grabbed by default. <2> Only m3u8 files are supported and the TS access path must be a relative path. <3> Multi-rate adaptive m3u8 files are not supported. * |
| notifyURL | no | The processing result is notified of the receiving URL. For the format of the notification content, see the returned content of the task query interface . |
| force | no | Whether to force processing. 1: Force data processing and overwrite the original result. 0: If the data processing result already exists, the processing is considered to be successful, and repeated processing is avoided to waste resources. The default value is 0. |
| separate | no | Whether processing Description are notified separately. Set 1 and 0 as option codes: 1: Indicates that notifyURL is notified after each command is executed. 0: Indicates that notifyURL is notified once after all commands are executed . The default value is 0. |
Request parameter example
fops=fetchURL/aHR0cDovL3d3dy5iYWlkdS5jb20vYWEuanBn/bucket/bXlidWNrZXQ=;fetchURL/aHR0cDovL3d3dy5iYWlkdS5jb20vYWEuanBn/bucket/bXlidWNrZXQ=/key/ZGlyL2FhLmpwZw==/prefix/YWFkaXIv/md5/0bee89b07a248e27c83fc3d5951213c1;fetchURL/aHR0cDovL3d3dy5iYWlkdS5jb20vYWEuemlw/bucket/bXlidWNrZXQ=/decompress/zip;fetchURL/aHR0cDovL3d3dy5iYWlkdS5jb20vYWEubTN1OA==/bucket/bXlidWNrZXQ=/fetchTS/1¬ifyURL=aHR0cDovL3Rlc3QuY29tL25vdGlmeVVSbA==&force=1&separate=1
Response Description
If the request is successful, a JSONstring with the following content is returned:
{ "persistentId": <persistentId> }
| Field name | Required | Description |
|---|---|---|
| persistentId | Yes | The task process ID, which can be used to query the task progress |
If the request fails, a Jsonstring with the following content is returned:
{
"code": "<code string>",
"message": "<ErrMsg string>"
}
| Field name | Required | Description |
|---|---|---|
| code | Yes | HTTP request response code, see HTTP response status code |
| message | Yes | Request failed message |
Example
curl -v -X POST -d "fops=fetchURL/aHR0cDovL3d3dy5iYWlkdS5jb20vYWEuanBn/bucket/bXlidWNrZXQ=/key/ZGlyL2FhLmpwZw==/prefix/YWFkaXIv/md5/0bee89b07a248e27c83fc3d5951213c1;fetchURL/aHR0cDovL3d3dy5iYWlkdS5jb20vYWEuemlw/bucket/bXlidWNrZXQ=/decompress/zip¬ifyURL=aHR0cDovL3Rlc3QuY29tL25vdGlmeVVSbA==&force=1&separate=1" -H "Authorization:86622e227a50d49d858c2494a935bc2e4ac543a7:Y2JmMGY0YTNhZmQxZmVlZGM2ZTA0YTdjNTcyOWQ1NjRkNTExYjg0Yg==" --url "http://mgrDomain/fmgr/fetch"
Is the content of this document helpful to you?
Yes
I have suggestion