Envelopes

This document defines the Envelope and Item formats used by Sentry for data ingestion, forwarding, and offline storage. The target audience of this document is Sentry SDK developers and maintainers of the ingestion pipeline.

Envelopes are a data format similar to HTTP form data, comprising common Headers and a set of Items with their own headers and payloads. Envelopes are optimized for fast parsing and human readability. They support a combination of multiple Items in a single payload, such as:

• Submit events with large binary attachments.
• Enable communication between hops, for instance, between different SDKs (Native and Mobile, ReactNative and Android) and between Relays.
• Allow batching of certain Items into a single submission.
• Offline storage for deferred sending after connection issues.

Sentry specifies a dedicated endpoint at for ingesting Envelopes:

POST /api/<project_id>/envelope/

Terminology

• required: The implementation may emit an error if this field is missing.
• recommended: This field should be emitted when writing, but can be missing during a read.
• optional: Can be omitted freely during writing and can be missing during a read.

Serialization Format

This section defines the Envelope data format and serialization. For details on data integrity and a list of valid Item types refer to Envelope Items.

Prerequisites

These definitions apply to all parts of the Envelope data format:

1. Newlines are defined as UNIX newlines, represented by \n and ASCII code 10. If newlines are preceded with \r, this character is considered part of the previous line or payload and may emit an error.
2. UUIDs are declared as either 32 character hexadecimal strings without dashes ("12c2d058d58442709aa2eca08bf20986"), or 36 character strings with dashes ("12c2d058-d584-4270-9aa2-eca08bf20986"). It is recommended to omit dashes and use UUID v4 in all cases.
3. Envelopes do not offer a mechanism for compression. However, an entire Envelope may be compressed or decompressed in an implementation defined way by any component handling Envelopes. For example, Ingestion allows compression via content encoding.

Headers

Envelopes contain Headers in several places. Headers are JSON-encoded objects (key-value mappings) that follow these rules:

• Always encoded in UTF-8
• Must be valid JSON
• Must be declared in a single line; no newlines
• Always followed by a newline (\n) or the end of the file
• Must not be padded by leading or trailing whitespace
• Should be serialized in their most compact form without additional white space. Whitespace within the JSON headers is permitted, though discouraged.
• Unknown attributes are allowed and should be retained by all implementations; however, attributes not covered in this spec must not be actively emitted by any implementation.
• All known headers and their data types can be validated by an implementation; if validation fails, the Envelope may be rejected as malformed.
• Empty headers {} are technically valid

Header-only Example:

{ "event_id": "12c2d058d58442709aa2eca08bf20986" }

Envelopes

The full grammar for an Envelope is:

Envelope = Headers { "\n" Item } [ "\n" ] ;
Item = Headers "\n" Payload ;
Payload = { * } ;

• Headers are a single line containing a JSON object, as defined in the Headers section. Attributes defined in the Envelope header scope the contents of the Envelope and can be thought of as applying to all Items.
• Based on the contents of the Envelope, certain header attributes may be required. See Envelope Items for a specification of required attributes.
• Items comprise their own headers and a payload. There can be an arbitrary number of Items in an Envelope separated by a newline. An implementation should consume Items until the file ends.
• Envelopes should be terminated with a trailing newline. This newline is optional. After the final newline, no whitespace is allowed.
• Envelopes may be empty, terminating immediately after the headers.
• The end of file (EOF) does not implicitly terminate an Envelope if more data is expected, such as a Payload.

Envelope Headers

Envelopes can have a number of headers which are valid in all situations:

dsn

String, recommended. An envelope can be self authenticated. This means that

the envelope has all the information necessary to be sent to sentry. In this

case the full DSN must be stored in this key.

sdk

Object, recommended. This can carry the same payload as the sdk interface

in the event payload but can be carried for all events. This means that SDK

information can be carried for minidumps, session data and other submissions.

sent_at

String, recommended. The timestamp when the event was sent from the SDK as string in

RFC 3339 format. Used for clock drift

correction of the event timestamp. The time zone must be UTC.

Items

Items supply the data of an Envelope. Without Items, an Envelope is considered empty and can safely be discarded.

There are two generic headers for every Item:

type

String, required. Specifies the type of this Item and its contents. Based

on the Item type, more headers may be required. See Envelope Items for a list

of all Item types.

length

int, recommended. The length of the payload in bytes. If no length is

specified, the payload implicitly goes to the next newline. For payloads

containing newline characters, the length must be specified.

Notes for implementors:

• Envelope header is required, but it can be empty.
• Implementations must gracefully skip and retain Items of unknown type, along with their payload.
• Unknown attributes must be forwarded to the upstream.
• Length-prefixed payloads must terminate with \n or EOF. The newline is not considered part of the payload. Any other character, including whitespace, means the Envelope is malformed.
• If length cannot be consumed, that is, the Envelope is EOF before the number of bytes has been consumed, then the Envelope is malformed.
• If an Item with implicit length is terminated by \r\n, then \r is considered an arbitrary character not part of the newline, and thus part of the payload.

Full Examples

These examples contain full Envelope payloads. Newlines are explicitly marked with \n, unprintable characters are escaped with \x<><>. all other characters are literal.

Envelope with 2 Items:

Note that the attachment contains a Windows newline at the end of its payload which is included in length:

{"event_id":"9ec79c33ec9942ab8353589fcb2e04dc","dsn":"https://e12d836b15bb49d7bbf99e64295d995b:@sentry.io/42"}\n
{"type":"attachment","length":10,"content_type":"text/plain","filename":"hello.txt"}\n
\xef\xbb\xbfHello\r\n\n
{"type":"event","length":41,"content_type":"application/json","filename":"application.log"}\n
{"message":"hello world","level":"error"}\n

Envelope with 2 Items, last newline omitted:

Note that the attachment contains a Windows newline at the end of its payload which is included in length:

{"event_id":"9ec79c33ec9942ab8353589fcb2e04dc","dsn":"https://e12d836b15bb49d7bbf99e64295d995b:@sentry.io/42"}\n
{"type":"attachment","length":10,"content_type":"text/plain","filename":"hello.txt"}\n
\xef\xbb\xbfHello\r\n\n
{"type":"event","length":41,"content_type":"application/json","filename":"application.log"}\n
{"message":"hello world","level":"error"}

Envelope with 2 empty attachments:

{"event_id":"9ec79c33ec9942ab8353589fcb2e04dc"}\n
{"type":"attachment","length":0}\n
\n
{"type":"attachment","length":0}\n
\n

Envelope with 2 empty attachments, last newline omitted:

{"event_id":"9ec79c33ec9942ab8353589fcb2e04dc"}\n
{"type":"attachment","length":0}\n
\n
{"type":"attachment","length":0}\n

Item with implicit length, terminated by newline:

{"event_id":"9ec79c33ec9942ab8353589fcb2e04dc"}\n
{"type":"attachment"}\n
helloworld\n

Item with implicit length, last newline omitted, terminated by EOF:

{"event_id":"9ec79c33ec9942ab8353589fcb2e04dc"}\n
{"type":"attachment"}\n
helloworld

Envelope without headers, implicit length, last newline omitted, terminated by EOF:

{}\n
{"type":"session"}\n
{"started": "2020-02-07T14:16:00Z","attrs":{"release":"sentry-test@1.0.0"}}

Data Model

This section has been moved to Envelope Items.

Ingestion

This section describes how to ingest Envelopes into Relay or Sentry. The main ingestion endpoint for Envelopes is:

POST /api/<project_id>/envelope/

HTTP Headers

Envelope requests may contain all headers as regular store requests. The only accepted content-type is application/x-sentry-envelope, which is implied if it is missing. To minimize the necessity for CORS preflights it's acceptable to send text/plain, multipart/form-data and application/x-www-form-urlencoded as well. In either of those cases the behavior however is the same as using application/x-sentry-envelope.

Authentication

In addition to regular HTTP header- and querystring authentication, the Envelope endpoint allows to authenticate via an Envelope header. To choose this authentication method, set the "dsn" Envelope header to the full DSN string.

If multiple forms of authentication are given, the endpoint validates that the information matches and otherwise rejects the request. If both are missing, the Envelope is rejected with status code 403 Forbidden.

Size Limits

Event ingestion imposes limits on the size and number of Items in Envelopes. These limits are subject to future change and defined currently as:

• 20MB for a compressed envelope request
• 100MB for a full envelope after decompression
• 100MB for all attachments combined
• 100MB for each attachment item
• 1MB for event (errors and transactions), span, and metric (statsd, buckets, meta) items.
• 100KB for monitor check-in items
• 50MB for profile items
• 10MB for compressed replay item
• 100MB for replay item after decompression
• 100 sessions per envelope
• 100 pre-aggregated session buckets per each "sessions" item

External References

• Multi Part Form Data
• Chunked Transfer Encoding