The Conviva Connect App Flow Data Feed and App Event Data Feed deliver the fields listed below, each with its type, mode and meaning. For the metric definitions and SQL built on these fields, see Metric calculations. For the end-to-end load, see Build your first App pipeline.
Both feeds use PascalCase field names. Legacy App feeds use lowercase names with underscores, so the casing tells you which pipeline produced a file.
Row grain and identity
In the Flow Feed, each row is one instance of one configured flow. FlowId is the per-instance identifier and is unique across the feed. Flow is the configured flow name and repeats whenever the same flow runs again, so two rows can share a Flow value and never share a FlowId. Conviva writes each flow instance once, when it completes or a session or flow timeout closes it.
In the Event Feed, each row is one event. ECOSessionId identifies the app session the event belongs to, and an app session closes after 90 seconds with no user interaction. Each ECOSessionId belongs to exactly one ClientId, and one ClientId produces many sessions. Each event appears in exactly one hourly file.
The two feeds share no session key. ClientId identifies the device and UserId identifies the user, and the pair is the join between them. See Build your first App pipeline, step 8.
Neither feed carries a partition column. The delivery window is carried by the folder, so record it as you load.
The JSON columns
Four columns hold JSON objects. In Parquet and Avro they arrive as maps. In CSV they are serialized into a single cell as a JSON string.
MetricsIntandMetricsFloat(Flow Feed) hold the flow's measures, keyed by measure name.Tags(Flow Feed) andEventDetails(Event Feed) hold the metadata your own app reports.
Keys vary from row to row, and a key that does not apply to a row is missing from the object. Extracting a missing key returns null, so wrap every extraction in COALESCE where a missing key should count as zero. The keys listed below are the ones the sample files carry. Your account's keys depend on how your flows are configured and what your app reports. Confirm your set with your Conviva Representative.
Row kinds and their keys
Each Flow Feed row is one of five kinds, depending on how it's configured in Conviva. The kind decides which keys MetricsInt and MetricsFloat carry. Your flow configuration decides which kind each row is, so filter on Flow names where you know them and use the keys below as a guide.
| Row kind | Keys | Meaning |
|---|---|---|
| Standard flow | success, failure, total_duration_ms; optionally error and the network request keys | success and failure are 0 or 1 flags, and exactly one is set. success is 1 when the flow completed within its configured success threshold. error counts the configured error events during the flow, so a successful flow can carry a non-zero error. The network request keys are network_request_success_count, network_request_failure_count, network_request_success_duration_ms and network_request_failure_duration_ms. The two duration keys hold totals: divide by the matching count for an average. |
| Conversion metric | success, failure | Outcome flags with no duration. |
| Duration metric | total_duration_ms only | How long a step took. Conviva emits it only for positive durations. |
| Event aggregation metric | keys named after the metric | A Count writes one integer key. Sum and Average results go to MetricsFloat. An Average writes <name>_count to MetricsInt and <name>_value, already averaged for that row, to MetricsFloat. |
| App session | the session metric keys | Rows whose Flow value is session. Keys include app_startup_count, user_active_time_ms and page_load_count. See Metric calculations for the full list. |
The sample carries seven standard flows, five event aggregation Counts (each with a value of 1) and two duration metrics. It has no conversion metrics, no Sum or Average metrics and no session rows.
Standard flows are the only rows that carry both success and total_duration_ms, so filtering on both keys keeps them and drops every other kind.
Sentinel values
On a standard flow, a negative total_duration_ms means a timeout closed the flow before it completed:
-1: the app session timed out first.-2: the flow's completion timeout was reached first.
Treat both as status codes, and exclude them from every sum, average and ratio over the column. A value of 0 or more means the flow completed. In the sample, four standard flows carry -1.
Flow duration comes from total_duration_ms. EndTimeStampMs minus StartTimeStampMs is the time from the flow's start until it closed, whether or not it completed. In the sample, two duration metrics with identical start and end timestamps report 130 ms and 648 ms, and each of the four timed-out flows spans 2,035,669 ms while reporting no duration.
Sample delivered Flow file
Below are four rows from an anonymized Flow file, exactly as the columns arrive. Scroll the table horizontally to see all 33 columns. The values are synthetic (test-network IP addresses and placeholder names), but the column order, casing and formatting match a real delivered file. The JSON columns are truncated for display. Download the full sample App Flow Connect file to open it in a spreadsheet or feed it through your pipeline.
The four rows cover the three kinds of row in the sample. Application Launch Flow completed in 876 ms and still reports error of 7. conviva_video_attempt is an event aggregation metric with a single key. Video Attempt to Stream Auth Request is a duration metric reporting 130 ms with identical start and end timestamps. Accept Offer was closed by a session timeout: total_duration_ms is -1 while its timestamps span 2,035,669 ms.
| FlowId | Flow | UserId | ClientId | SensorVersion | City | State | ZipCode | Country | Platform | PlatformSubcategory | DeviceCategory | DeviceManufacturer | DeviceName | DeviceModel | DeviceMarketingName | DeviceOperatingSystemFamily | DeviceOperatingSystem | DeviceOperatingSystemVersion | DeviceHardwareType | BrowserName | BrowserVersion | AppName | AppVersion | AppBuild | AppType | Asn | Isp | StartTimeStampMs | EndTimeStampMs | MetricsInt | MetricsFloat | Tags |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 05af294f1421f51520908e7e4db83839 | Application Launch Flow | 28e3c5f8-4609-f29d-af3b-fc17e1287690 | 1426529805.1287270902.1785263917.1212759394 | js-1.5.5 | dublin | leinster | d02 | ireland | web | web | Web | Apple | Mac | Mac | Apple Mac | macOS | macOS | macOS 10.15.7 | Desktop | Chrome | Chrome 148.0.0.0 | ExampleTV-browser | 7.1.35 | 7.1.35 | Web | EXAMPLENET-AS Example Networks Ltd, IE | example networks | 1781866691358 | 1781866692234 | {"error":7,"failure":0,"success":1,"total_duration_ms":876… | {} | {"First Launch":"Yes","DeepLinkPlacement":"browser","Profi… |
| 6873ed215491727373dabf14c8f64ba6 | conviva_video_attempt | 28e3c5f8-4609-f29d-af3b-fc17e1287690 | 1426529805.1287270902.1785263917.1212759394 | js-1.5.5 | dublin | leinster | d02 | ireland | web | web | Web | Apple | Mac | Mac | Apple Mac | macOS | macOS | macOS 10.15.7 | Desktop | Chrome | Chrome 148.0.0.0 | ExampleTV-browser | 7.1.35 | 7.1.35 | Web | EXAMPLENET-AS Example Networks Ltd, IE | example networks | 1781866693974 | 1781866693974 | {"Video Attempt":1} | {} | {"First Launch":"Yes","Video Event Url":"https://example.c… |
| 42bd2230d64f6436b5360952f0f85d38 | Video Attempt to Stream Auth Request | 28e3c5f8-4609-f29d-af3b-fc17e1287690 | 1426529805.1287270902.1785263917.1212759394 | js-1.5.5 | dublin | leinster | d02 | ireland | web | web | Web | Apple | Mac | Mac | Apple Mac | macOS | macOS | macOS 10.15.7 | Desktop | Chrome | Chrome 148.0.0.0 | ExampleTV-browser | 7.1.35 | 7.1.35 | Web | EXAMPLENET-AS Example Networks Ltd, IE | example networks | 1781866694752 | 1781866694752 | {"total_duration_ms":130} | {} | {"First Launch":"Yes","Network Request Response Code":"200… |
| 00d26102f3b6af39242135eaaf70eb77 | Accept Offer | 8c8819d7-81c1-ae4a-2c02-5bc20fefeea8 | 1984561384.1202246918.148301006.1095262305 | js-1.5.5 | bengaluru | karnataka | 560001 | india | web | web | Web | Apple | Mac | Mac | Apple Mac | macOS | macOS | macOS 10.15.7 | Desktop | Chrome | Chrome 148.0.0.0 | ExampleTV-browser | 7.1.35 | 7.1.35 | Web | EXAMPLEISP-AS Example Broadband, IN | example broadband | 1781864670281 | 1781866705950 | {"error":0,"failure":1,"success":0,"total_duration_ms":-1} | {} | {"First Launch":"Yes","IsSubscribed":"false","Channel":"UN… |
Sample delivered Event file
Below are six rows from an anonymized Event file, one per EventCategory the sample carries, ordered by EventTimeMs. Download the full sample App Event Connect file for the rest.
The Frustration Signal row has an empty IpV4, which is expected. IpV6 is empty on every row of the sample.
| ECOSessionId | EventTimeMs | ClientId | UserId | City | State | Country | IpV4 | IpV6 | Isp | Asn | ConnectionType | EventCategory | EventName | EventDetails |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 12304258176999194035520940129463743443729745416184822825533400183382 | 1781870462316 | 1426529805.1287270902.1785263917.1212759394 | 28e3c5f8-4609-f29d-af3b-fc17e1287690 | dublin | leinster | ireland | 192.0.2.10 | example networks | EXAMPLENET-AS Example Networks Ltd, IE | cable/dsl | Media | Video Start Time | {"path":"/1/program/b3d043e0a1f08bd11c82aeb34e58968b/watch… | |
| 12304258176999194035520940129463743443729745416184822825533400183382 | 1781870820595 | 1426529805.1287270902.1785263917.1212759394 | 28e3c5f8-4609-f29d-af3b-fc17e1287690 | dublin | leinster | ireland | 192.0.2.10 | example networks | EXAMPLENET-AS Example Networks Ltd, IE | cable/dsl | Custom | Experience | {"path":"/dashboard/home","ProfileGuid":"28e3c5f8-4609-f29… | |
| 12304258178242400463119547128129233717989280609545352196117613135999 | 1781870843423 | 1891968572.1634682297.450814740.1030318875 | 414a8338-db00-43c6-9934-1c84ea9c1248 | austin | texas | united states | example corporate network | EXAMPLECORP-AS Example Corporate Network, US | corporate | Frustration Signal | conviva_click_to_error | {"Stack Trace":"TypeError: exampleAnalytics.trackEvent is … | ||
| 12304258176734351282259336171015022955966743790941446409590003981875 | 1781871894890 | 1308352753.970593429.1675942206.1688127617 | a3172022-3d49-c3b7-ec62-bf6dc6683944 | bengaluru | karnataka | india | 192.0.2.11 | example broadband | EXAMPLEISP-AS Example Broadband, IN | cable/dsl | Page | Tab Not Equals Guide | {"ProfileGuid":"a3172022-3d49-c3b7-ec62-bf6dc6683944","Dev… | |
| 12304258175065347482524944647493139221198610858275473417189570682261 | 1781873691585 | 1551812005.1224339696.530433194.932408738 | C5XRF0hT7XlFjzsTDwaz4u9aYmoeTLAV | bengaluru | karnataka | india | 192.0.2.13 | example broadband | EXAMPLEISP-AS Example Broadband, IN | cable/dsl | Search | Unentitled Product View | {"IsSubscribed":"false","path":"/","Channel":"|ChannelKind… | |
| 12304258175065347482524944647493139221198610858275473417189570682261 | 1781873691585 | 1551812005.1224339696.530433194.932408738 | C5XRF0hT7XlFjzsTDwaz4u9aYmoeTLAV | bengaluru | karnataka | india | 192.0.2.13 | example broadband | EXAMPLEISP-AS Example Broadband, IN | cable/dsl | Payment | offer_click | {"IsSubscribed":"false","path":"/","Channel":"|ChannelKind… |
Flow Feed field dictionary
The Flow Feed delivers 33 columns. Your account's column list is configured when the feed is set up, so confirm it with your Conviva Representative.
Identity and timing
| Field name | Type | Mode | Description |
|---|---|---|---|
FlowId | VARCHAR(32) | REQUIRED | Identifier for this instance of the flow, generated from the ClientId, the flow name and the start time. Two runs of the same configured flow get different values. |
Flow | VARCHAR(128) | REQUIRED | Name of the configured flow, as it appears in Pulse. Group by this column to report per flow. Names prefixed with conviva_ are Conviva-defined; the rest are yours. |
UserId | VARCHAR(128) | NULLABLE | Identifier for the user, as reported by your app. Half of the join key to the Event Feed. Empty when your integration does not report it. |
ClientId | VARCHAR(64) | NULLABLE | Conviva device identifier, four integers separated by dots. The other half of the join key to the Event Feed. |
StartTimeStampMs | BIGINT | NULLABLE | When the flow started, as Unix epoch time in milliseconds. |
EndTimeStampMs | BIGINT | NULLABLE | When the flow closed, as Unix epoch time in milliseconds, whether it completed or a timeout closed it. Take the flow's duration from total_duration_ms. |
SensorVersion | VARCHAR(32) | NULLABLE | Version of the Conviva Sensor that reported the flow, for example js-1.5.5. The prefix names the Sensor platform. |
Measures and metadata
| Field name | Type | Mode | Description |
|---|---|---|---|
MetricsInt | MAP<VARCHAR, BIGINT> | REQUIRED | The flow's whole-number measures, keyed by measure name. See the row kinds above. In CSV this arrives as a JSON string. |
MetricsFloat | MAP<VARCHAR, DOUBLE> | REQUIRED | The flow's floating-point measures, keyed by measure name, including event aggregation Sum and Average results. Empty on every row of the sample, because the sample has no Sum or Average metrics. |
Tags | MAP<VARCHAR, VARCHAR> | REQUIRED | Custom metadata your app reports with the flow, keyed by tag name. Every value is a string, including numbers. The sample carries keys such as SessionID, ScreenName, ActionName, AssetGuid and ContentType. Your set is whatever your integration reports. |
Geography and network
| Field name | Type | Mode | Description |
|---|---|---|---|
City | VARCHAR(128) | NULLABLE | City derived from the IP address, lowercase. |
State | VARCHAR(128) | NULLABLE | State, province or region derived from the IP address, lowercase. |
ZipCode | VARCHAR(32) | NULLABLE | Postal code derived from the IP address. Formats differ by country, so keep it a string. |
Country | VARCHAR(128) | NULLABLE | Country derived from the IP address, lowercase. |
Asn | VARCHAR(128) | NULLABLE | Autonomous system that owns the IP address, as the registry name plus country code. |
Isp | VARCHAR(128) | NULLABLE | Internet service provider for the IP address, lowercase. Narrower than Asn and the one to group by for network reporting. |
Device, platform and app
| Field name | Type | Mode | Description |
|---|---|---|---|
Platform | VARCHAR(64) | NULLABLE | Broad platform the app ran on, for example web. |
PlatformSubcategory | VARCHAR(64) | NULLABLE | Finer platform split within Platform. |
DeviceCategory | VARCHAR(64) | NULLABLE | Conviva device category, for example Web. This is the column to group by for device-level reporting; it matches the Device Category dimension in Pulse. |
DeviceManufacturer | VARCHAR(64) | NULLABLE | Manufacturer of the device, for example Apple. |
DeviceName | VARCHAR(128) | NULLABLE | Conviva's normalized device name. |
DeviceModel | VARCHAR(128) | NULLABLE | Model string for the device. |
DeviceMarketingName | VARCHAR(128) | NULLABLE | Consumer-facing name for the device, for example Apple Mac. Use this one in anything a business reader sees. |
DeviceOperatingSystemFamily | VARCHAR(64) | NULLABLE | Operating system family, for example macOS. |
DeviceOperatingSystem | VARCHAR(64) | NULLABLE | Operating system without the version. |
DeviceOperatingSystemVersion | VARCHAR(64) | NULLABLE | Operating system including the version, for example macOS 10.15.7. The OS name is repeated inside the value, so parse rather than concatenate. |
DeviceHardwareType | VARCHAR(64) | NULLABLE | Form factor of the device, for example Desktop. |
BrowserName | VARCHAR(64) | NULLABLE | Browser the app ran in. Empty for apps that are not browser based. |
BrowserVersion | VARCHAR(64) | NULLABLE | Browser including the version, for example Chrome 148.0.0.0. The browser name is repeated inside the value. |
AppName | VARCHAR(128) | NULLABLE | Name of your application as reported by the Sensor. |
AppVersion | VARCHAR(64) | NULLABLE | Release version of your application. Group by this to compare flow outcomes across releases. |
AppBuild | VARCHAR(64) | NULLABLE | Build identifier of your application. Equal to AppVersion unless your build pipeline sets it separately. |
AppType | VARCHAR(64) | NULLABLE | Kind of application, for example Web. |
Event Feed field dictionary
The Event Feed delivers 15 columns and no metric columns. Compute counts, rates and time spans over the event rows.
| Field name | Type | Mode | Description |
|---|---|---|---|
ECOSessionId | VARCHAR(128) | REQUIRED | Identifier for the app session the event belongs to, a long numeric string. It does not embed the ClientId, so keep both. Group by it to roll events up to a session. |
EventTimeMs | BIGINT | REQUIRED | When the event occurred, as Unix epoch time in milliseconds. Order by this within a session to reconstruct the user's path. |
ClientId | VARCHAR(64) | NULLABLE | Conviva device identifier, four integers separated by dots. Half of the join key to the Flow Feed. |
UserId | VARCHAR(128) | NULLABLE | Identifier for the user, as reported by your app. The other half of the join key to the Flow Feed. Empty on a minority of rows, typically before the user signs in. |
EventCategory | VARCHAR(64) | NULLABLE | Grouping for the event, set by your event-mapping configuration. The sample carries Custom, Media, Page, Search, Payment and Frustration Signal. The set isn't fixed, so write roll-ups that show a new category in the output. |
EventName | VARCHAR(128) | NULLABLE | Name of the event. Names prefixed with conviva_, such as conviva_page_view and conviva_video_error, are generated by the Conviva Sensor. Everything else is a name your app reports. |
EventDetails | MAP<VARCHAR, VARCHAR> | REQUIRED | Custom metadata your app reports with the event, keyed by detail name. Keys vary by event. The sample carries keys such as url, path, referrer, ScreenName, Element Name, Error Message and Stack Trace. In CSV this arrives as a JSON string. |
City | VARCHAR(128) | NULLABLE | City derived from the IP address, lowercase. |
State | VARCHAR(128) | NULLABLE | State, province or region derived from the IP address, lowercase. |
Country | VARCHAR(128) | NULLABLE | Country derived from the IP address, lowercase. The Event Feed carries no ZipCode. |
IpV4 | VARCHAR(16) | NULLABLE | IPv4 address the event came from. Empty when the client connected over IPv6 or when the address was not resolved. |
IpV6 | VARCHAR(64) | NULLABLE | IPv6 address the event came from. Empty on the overwhelming majority of rows. |
Isp | VARCHAR(128) | NULLABLE | Internet service provider for the IP address, lowercase. |
Asn | VARCHAR(128) | NULLABLE | Autonomous system that owns the IP address, as the registry name plus country code. |
ConnectionType | VARCHAR(64) | NULLABLE | Kind of network connection, for example cable/dsl or corporate. Derived from the IP address, so it describes the network rather than the device radio. |
Next steps
Metric calculations for the flow and event metric definitions and their SQL.
Build your first App pipeline for the end-to-end load.