Log File Delivery
Log files provide granular data for various types of events. Once log data is collected and aggregated into log files, they are then delivered to your Amazon S3 bucket.
Ad log data is delivered separately from the log events discussed in this article.
Log delivery, retention, and privacy
-
If log data cannot be delivered (for example, due to invalid credentials), we send you an email notification and retry delivery at a later time. Email notifications are sent at most once per 24-hour period. To avoid data loss, disable log pushing or fix the access credentials as soon as possible.
-
Undelivered log files may be deleted after 14 days. Once a log file is erased, the data is permanently lost and the associated events cannot be recovered.
-
After a log file is successfully delivered, it is erased from our system and cannot be regenerated.
-
Log data may contain sensitive information about your viewers (for example, a viewer’s IP address). While this information can also be accessed through an in-house media platform, you should treat all log data as confidential and take appropriate steps to protect user privacy.
-
Configure your Amazon S3 bucket to be private. Log delivery to a public S3 bucket is not supported.
-
Playback session IDs are generated when a viewer begins playing encrypted content. A session persists until the viewer closes their browser or playback environment, or until the session remains inactive for an extended period of time. As a result, a single session may represent playback across multiple assets and/or channels.
Enabling log delivery
Log data is pushed from our system to your Amazon S3 bucket. To enable log delivery, you must provide the required configuration details on the Log Pushing page (login required).
From the main menu, navigate to Settings > Log Pushing.
Configuration options
| Option | Description |
|---|---|
| AWS Access Key | An Amazon access key ID for a dedicated IAM user that has write-only permissions to the target S3 bucket. |
| AWS Secret Access Key | The corresponding Amazon secret access key for the same dedicated IAM user. |
| S3 Bucket Name | The name of the private S3 bucket where log files will be delivered. |
| Key Prefix | An optional virtual folder path and/or filename prefix that is applied to each log file delivered to your bucket. |
Log delivery requirements and behavior
-
Log delivery requires an Amazon S3 bucket. You may use an existing private bucket or create a new one using the AWS Management Console.
-
Do not provide credentials for an administrator account. Instead, create a dedicated IAM user with write-only permissions to the target S3 bucket. Use the AWS Management Console to generate credentials and verify that the user is authorized to upload objects.
-
Log delivery is only supported for private S3 buckets. Use of a public S3 bucket is not permitted and may result in account termination.
-
Once enabled, logs are delivered immediately and continue to be delivered until log pushing is disabled. Expect many small log files per day, as logs are generated and uploaded continuously.
-
Log files are delivered as soon as they are processed and are not guaranteed to arrive in sequential order.
Log Files
Log files are gzip-compressed plain text files with unique file names.
File Naming Convention
The naming convention for log files includes a prefix that categorizes objects by hour and then by event type.
Although the file extension is csv, fields are tab-separated.
Syntax
version=###/ exec_dt=YYYYMMDDhh/event_type=Event Type/part-Count-UUID.c000.csv.gz
The above date is only loosely tied to the data contained within a log file. Do not assume that all of the events contained within a log file occurred on that date.
Example
version=001/exec_dt=2023010810/event_type=slice_played/part-00000-3608d733-e96c-4575-8784-41154246e830.c000.csv.gz
In this example, a file called part-00000-3608d733-e96c-4575-8784-41154246e830.c000.csv.gz will be stored in the following virtual directory:
version=001/exec_dt=2023010810/event_type=slice_played.
Log Data
Key information
-
Events are reported below the format version and commented lines. Each line represents a single event.
-
Each line is composed of tab-separated values.
-
Empty values are represented by -.
-
The first 3 fields in every line are:
Name
Description
Date
Identifies the date (UTC) on which the event occurred.
Syntax:YYYY-MM-DDTime
Identifies the approximate time (UTC) at which the event occurred.
Syntax:hh:mm:ssEvent name
Identifies the type of event. Learn more.
-
Log entries are not guaranteed to be listed in sequential order. However, they are typically aggregated within a few minutes of each other.
-
The set of fields that will be reported after the initial 3 fields varies by event type. Learn more.
Sample Log File
2020-03-10 16:48:11 slice_played 24089701c7894902aebc3f1bd209ae2e 103 - - Roku/DVP-9.20 (519.20E04806A) - - 2b9fa9bd64804331b8da74c054b3cd67 - - - 0.8343682004336305 Spirit Games - - -
2020-03-10 16:52:58 slice_played 080ca7672c1043599818cd823cb7e144 252 - - Roku/DVP-9.20 (289.20E04804A) - - 60614f69a767462aabe6152611d73aa1 - - - 0.636021907119814 My Fiancé - - -Log Events
This section details log events and their fields.
Certain events support custom log fields. Each event type indicates the set of objects from which custom log fields may be added.
ad_beacon_sent
Indicates that an event beacon was sent to an ad server.
Field | Description |
|---|---|
group ID | Indicates an ID that identifies multiple URLs that correspond to a single event. For example, multiple impressions may be sent to various ad servers (e.g., Google Ad Manager, FreeWheel) for a single ad. |
session ID | Indicates a unique playback session ID. |
ad asset ID | Indicates the asset ID corresponding to the ad that was played. |
event description | Indicates the type of event. Valid values are: |
ad break position | Indicates the zero-based position of the ad break.Syntax: |
ad index in break/pod | Indicates the zero-based index for the ad in the ad break / ad pod. For example, |
ad ID | Indicates the ad ID provided by the ad server. FreeWheel defines this ID via ads/ad@adId. |
creative ID (FreeWheel only) | Indicates the creative ID returned by FreeWheel via ad/creatives/creative@creativeId. |
url | Indicates the event's URL. |
HTTP Status | Indicates the HTTP status code for the response from the ad server. |
ad_request_error
Indicates that an error occurred when making an HTTP request to the ad server.
Field | Description |
|---|---|
session ID | Indicates a unique playback session ID. |
asset ID | Indicates the asset ID corresponding to the asset that was played. |
ad break position | Indicates the zero-based position of the ad break. |
ad request data | Returns either of the following:
|
error description | Indicates the description or error event generated by the exception. |
slicer build type | Provides the type and operating system for the Slicer that processed the requested content. |
channel ID | Indicates a channel ID when an ad was requested during the playback of a live channel. |
event ID | Indicates an event ID when an ad was requested during the playback of a live event. |
asset_deleted
Indicates that an asset was deleted.
| Field | Description |
|---|---|
| asset ID | Indicates the asset ID corresponding to the asset that was deleted. |
| cause | Indicates the reason why the asset was deleted. For example, an asset may be manually deleted, automatically deleted due to an auto-expiration policy, or automatically deleted because an error occurred during the slicing job. |
asset_play_started
Indicates that a viewer initiated the playback of live or VOD content.
Field | Description |
|---|---|
asset ID | Indicates the asset ID corresponding to the asset that was played. |
user IP | Indicates the IP address of the playback device. |
player user agent | Indicates the player's user agent as reported by the |
player referrer URL | Indicates the referrer as reported by the |
channel ID | Indicates a channel ID if a live channel was played. |
asset is ad | Indicates whether the asset was an ad. Valid values are:
|
euid | Indicates the external user ID provided in the playback URL. |
session ID | Indicates a unique playback session ID. |
playing owner ID | Identifies the owner of shared content by user ID. |
event ID | Indicates an event ID if a live event was played. |
channel_deleted
Indicates that the live channel was deleted.
| Field | Description |
|---|---|
| channel ID | Identifies the live channel that was deleted by its channel ID. |
| cause | Identifies the user that deleted the channel by their user ID. |
channel_ping
Indicates that the viewer continued to watch the live channel. This event, which is triggered every 10 minutes during the playback of a live channel, allows us to estimate simultaneous viewers during live playback.
Time is divided into ten-minute intervals that are known as buckets. During live playback, a single channel_ping event is triggered once per bucket. Each bucket is assigned a unique number starting from 0. Bucket 0 refers to the Unix epoch (00:00:00 on January 1, 1970 UTC), bucket 1 starts 10 minutes after that, and so on.
Field | Description |
|---|---|
channel ID | Identifies a live channel by its channel ID. |
asset ID | Indicates the asset ID corresponding to the asset that was played. |
bucket number | Identifies the number assigned to the bucket during which playback occurred. |
user IP | Indicates the IP address of the playback device. |
player user agent | Indicates the player's user agent as reported by the |
player referrer URL | Indicates the referrer as reported by the |
euid | Indicates the external user ID provided in the playback URL. |
session ID | Indicates a unique playback session ID. |
channel_play_started
Indicates that a viewer initiated the playback of a live channel.
Field | Description |
|---|---|
user IP | Indicates the IP address of the playback device. |
player user agent | Indicates the player's user agent as reported by the |
player referrer URL | Indicates the referrer as reported by the |
euid | Indicates the external user ID provided in the playback URL. |
session ID | Indicates a unique playback session ID. |
drm_error
Indicates that a DRM-related error occurred. For example, a request from a player that uses a deprecated Content Decryption Module (CDM) will generate this type of error.
| Field | Description | ||
|---|---|---|---|
| beam ID | Indicates the asset ID corresponding to the asset, live channel, or live event that triggered the DRM error. | ||
| drm_type | Identifies the DRM solution. Valid values are: `fairplay \ | widevine \ | playready` |
| event_error_message | Indicates the DRM error message. |
An
is generated for errors that are unrelated to DRM.
drm_key_read
Indicates that a DRM key was read. This event occurs when a player requests a DRM license.
Field | Description |
|---|---|
timestamp | Indicates the timestamp, in Unix time (milliseconds), that the DRM key was read. |
beam ID | Indicates the asset ID corresponding to the asset, live channel, or live event that triggered the DRM license request. |
playing owner ID | Identifies the owner of shared content by user ID. |
user IP | Indicates the IP address of the playback device. |
player user agent | Indicates the player's user agent as reported by the |
slicer build type | Provides the type and operating system for the Slicer that processed the requested content. |
beam description | Indicates the description for an asset, live channel, or live event. |
error
Indicates that an input error occurred. For example, the player may have requested an invalid playback URL.
| Field | Description |
|---|---|
| formatType | This field is reserved for future use. It currently returns 0. |
| subType | This field is reserved for future use. It currently returns web. |
| URL | Indicates the playback URL that caused the error. |
| user IP | Indicates the IP address of the playback device. |
| HTTP code | Indicates the status code (e.g., 404) for the response to the playback URL. |
| user agent | Indicates the player's user agent as reported by the User-Agent request header. |
| message | Provides a short description of why the error occurred. |
event_copied
Indicates that a live event was copied.
| Field | Description |
|---|---|
| event ID | Identifies a live event by its event ID. |
| cause | Indicates the user ID for the user that copied the live event. |
event_ping
Indicates that the viewer continued to watch the live event. This event, which is triggered every 10 minutes during the playback of a live event, allows us to estimate simultaneous viewers during live playback.
Time is divided into 10 minute intervals that are known as buckets. During live playback, a single event_ping event is triggered once per bucket. Each bucket is assigned a unique number starting from 0. Bucket 0 refers to the Unix epoch (00:00:00 on January 1, 1970 UTC), bucket 1 starts 10 minutes after that, and so on.
Field | Description |
|---|---|
event ID | Identifies a live event by its event ID. |
asset ID | Indicates the asset ID corresponding to the asset that was played. |
bucket number | Identifies the number assigned to the bucket during which playback occurred. |
user IP | Indicates the IP address of the playback device. |
player user agent | Indicates the player's user agent as reported by the User-Agent request header. |
player referrer URL | Indicates the referrer as reported by the Referer request header. |
euid | Indicates the external user ID provided in the playback URL. |
session ID | Indicates a unique playback session ID. |
event_play_started
Indicates that a user started watching a live event.
Field | Description |
|---|---|
event ID | Identifies a live event by its event ID. |
user IP | Indicates the IP address of the playback device. |
player user agent | Indicates the player's user agent as reported by the User-Agent request header. |
player referrer URL | Indicates the referrer as reported by the Referer request header. |
euid | Indicates the external user ID provided in the playback URL. |
session ID | Indicates a unique playback session ID. |
session_created
Indicates that a new playback session was created.
Field | Description |
|---|---|
session ID | Indicates a unique playback session ID. |
content ID | Indicates a channel ID, asset ID, or event ID for the playback session. |
content type | Indicates the type of content that was played, Valid values are: |
playing owner ID | Identifies the owner of shared content by user ID. |
user agent | Indicates the player's user agent as reported by the User-Agent request header. |
euid | Indicates the external user ID provided in the playback URL. |
referer | Indicates the referrer as reported by the |
user IP | Indicates the IP address of the playback device. |
slice_played
Indicates that a single temporal slice of media was delivered. Each slice represents approximately 4 seconds of media. The timestamp corresponds to time at which the slice was delivered to the player.
Field | Description |
|---|---|
asset ID | Indicates the asset ID corresponding to the asset that was played. |
slice number | Identifies a slice using a zero-based number. |
is live | Indicates whether the slice corresponds to a live channel or a live event. Valid values are:
|
user IP | Indicates the IP address of the playback device. |
player user agent | Indicates the player's user agent as reported by the |
player referrer URL | Indicates the referrer as reported by the |
euid | Indicates the external user ID provided in the playback URL.
|
session ID | Indicates a unique playback session ID. |
playing owner ID | Identifies the owner of shared content by user ID. |
channel ID | Indicates a channel ID when the slice was played in the context of a live channel. |
event ID | Indicates an event ID when the slice was played in the context of a live event. |
duration of content delivered | Indicates either the duration of the video segment that was delivered or the byte range corresponding to the duration of the requested content. |
ray_letter | Identifies the quality of the ray corresponding to the current slice by its letter. |
slicing_began
Indicates that an asset was created due to the start of either a live or VOD slicing job.
Field | Description | |
|---|---|---|
job type | Valid values are: `live \ | vod` |
asset ID | Indicates the asset ID corresponding to the new asset. | |
external ID | Indicates the external ID corresponding to the new asset. | |
channel ID | Indicates a channel ID if a live channel was played. | |
description | Indicates the description assigned to the new asset. | |
slicer build type | Provides the type and operating system for the Slicer that processed the requested content. | |
slicer version | Indicates the slicer's version number. | |
event ID | Indicates an event ID if a live event was played. |
slicing_ended
Indicates that the slicing job finished.
Field | Description |
|---|---|
asset ID | Indicates the asset ID corresponding to the last asset in the slicing job. |
had errors | Indicates whether an error occurred. Valid values are:
|
slices | Indicates the number of slices that were delivered for encoding. |
Format History
Log file format is tracked via version numbers defined within the prefix assigned to your log file. This version number is incremented whenever we add a new event type or field. Upon incrementing the log file version, we will deliver both versions for a limited time period. Take advantage of this window to update your code to process log data from the new virtual directory and to account for the new event type(s) and/or field(s).
Your code should either raise a proper error or ignore unknown event types and fields.
Format history is provided below.
| Version | Description |
|---|---|
| 001 | Initial release |
Updated 17 days ago