Retrieve older data using download API list syntax

Products

Cloud Secure Web Gateway - Cloud SWG

Issue/Introduction

Starting on June 21st, 2022 and ending on June 22nd, 2022 the Web Security Service had a SyncAPI outage where log data was not provided over the API.

There was no log data loss, however, Administrators that wish to retrieve the data must retrieve the missing data manually.

Be aware that there may be a small amount of log duplication.

Affected Times

The data affected each of our three regions as follows all times UTC:

Region	Problem Began	Problem Ended
Asia/Pacific	June 21 15:47:48	June 22 16:42:34
Europe	June 21 17:42:35	June 22 18:17:34
North America	June 21 18:35:56	June 22 18:22:16

Environment

Web Security Service

Sync API

Download API

Resolution

There are two ways to download the relevant time. Note that the API is recommended, especially if there is a large amount of log data for the time period.

Via the WSS portal under Account Configuration > Log Export.
Ensure that the fields chosen are the same as the ones provided via the API as they can differ.
Using the Download API.

If using the API (recommended), the following commands will retrieve the relevant time periods:

For Asia/Pacific Region:

curl -s -o myfile.zip -H "X-APIUsername: [CustomerApiUsername]" \
-H "X-APIPassword: [CustomerApiPassword]" \
"https://portal.threatpulse.com/reportpod/logs/download?type=hour&start=0&"\
"items=1655830800000,1655834400000,1655838000000,1655841600000,1655845200000,"\
"1655848800000,1655852400000,1655856000000,1655859600000,1655863200000,1655866800000,"\
"1655870400000,1655874000000,1655877600000,1655881200000,1655884800000,1655888400000,"\
"1655892000000,1655895600000,1655899200000,1655902800000,1655906400000,1655910000000,1655913600000"

For Europe and North America Regions:

curl -s -o myfile.zip -H "X-APIUsername: [CustomerApiUsername]" \
-H "X-APIPassword: [CustomerApiPassword]" \
"https://portal.threatpulse.com/reportpod/logs/download?type=hour&start=0&"\
"items=1655834400000,1655838000000,1655841600000,1655845200000,"\
"1655848800000,1655852400000,1655856000000,1655859600000,1655863200000,1655866800000,"\
"1655870400000,1655874000000,1655877600000,1655881200000,1655884800000,1655888400000,"\
"1655892000000,1655895600000,1655899200000,1655902800000,1655906400000,1655910000000,"\
"1655913600000,1655917200000,1655920800000"

Notes:

myfile.zip is configurable to the name of your choice.
[CustomerApiUsername] and [CustomerApiPassword] are the API credentials. Make sure to replace these values.
See Generate an API Key
Epoch times are between when the issue began and ended in the different regions. Do not modify these epoch times

Additional Information

Documentation on the API can be found here: