Firehose is an event delivery mechanism introduced as a premium feature for high-volume HasOffers clients, allowing push delivery of tracking and adjustment events to external event consumers. This feature requires you to have an Amazon Web Services (AWS) account as well as have or are a developer available to implement consuming messages from either Amazon's Simple Queue Service (SQS) or Amazon's Kinesis Stream.
Event Types
There are two types of events: standard events (a.k.a. tracking events) and adjustment events.
Determining Event Type
To determine if an event is a standard event or an adjustment event, check for the presence of an adjust_action field. If so, the event is an adjustment. If not, the event is standard.
From there, use the action field to determine if the event is for an impression, click, or conversion.
Standard Events
Standard events cover the raw tracking events from our ad servers: impressions, clicks, and conversions. These come from your raw, unadjusted stats. You can see these events in the application through the event tracer and stat reports.
JSON objects for standard events contain all applicable fields for the event. Certain fields, if they do not apply specifically to that event will be excluded. See Event Structure above for a link to all potential fields.
Adjustment Events
Adjustment events cover the difference applied to your stats, such as rejecting conversion.
JSON objects for adjustment events contain fields relating to the adjustments amounts along with other information that affects stats aggregation where applicable such as affiliate sub IDs and advertiser sub IDs. See Event Structure above for a link to all potential fields.
For example, if you reject a conversion that has a payout of $10 and revenue of $20, the adjustment event data includes the following fields:
"action":"conversion", "num":-1, "payout":-10, "revenue":-20
Adjustment events for impressions and clicks are handled in a similar fashion, with the action set to "impression" or "click".
Adjustment Events Stream
The adjustment events stream is a combination of:
- New events created through the Network API
- New events created by the Conversion Upload tool
- Events and adjustments to existing conversions through the Network API or Conversion Upload tool.
The Adjustment Events stream needs to be included to properly reconcile statistics and update existing conversions with Firehose.
How to Reconcile the Events Stream with the Adjustment Events Stream
First, the adjust_action value will need to be examined in order to determine the proper Adjustment Event action approach.
If adjust_action = 1 (create): Append the adjustment event to the regular event stream
If adjust_action = 2 (update): These events serve as a ledger of changes to the original event. The event identified via the TUNE Event ID value in the adjustment record will need to be applied to the event record with the same TUNE Event ID value. The TUNE Event ID serves as our primary/unique key for all event records.
Example adjustment outputs:
Creating new Conversions via API
{""action"": ""conversion"",
""ad_campaign_creative_id"": 0,
""ad_campaign_id"": 0,
""adjust_action"": ""create"",
""adjustment_sequence_id"": 16648208123366,
""adjustment_timestamp"": 1664820812,
""adv_sub"": """",
""adv_unique1"": """",
""adv_unique2"": """",
""adv_unique3"": """",
""adv_unique4"": """",
""adv_unique5"": """",
""advertiser_id"": 0,
""advertiser_manager_id"": ""0"",
""aff_sub"": """",
""aff_sub2"": """",
""aff_sub3"": """",
""aff_sub4"": """",
""aff_sub5"": """",
""affiliate_click_id"": """",
""affiliate_id"": 1029,
""affiliate_manager_id"": 1,
""affiliate_unique1"": """",
""affiliate_unique2"": """",
""affiliate_unique3"": """",
""affiliate_unique4"": """",
""affiliate_unique5"": """",
""app_version"": """",
""browser_id"": 0,
""conversion_id"": 923242,
""country_code"": ""INT"",
""created_from_adjustment"": true,
""created_timestamp"": 1664810032,
""creative_url_id"": 0,
""currency"": ""EUR"",
""current_payout"": 1.0,
""current_revenue"": 2.0,
""customer_id"": 0,
""date"": ""2022-10-03"",
""datetime"": ""2022-10-03 11:13:52"",
""event_id"": ""a6f0653c-ace9-4e63-bfaf-8c953d30fb99"",
""goal_id"": 0,
""hour"": 11,
""ip"": """",
""network_id"": ""demo"",
""num"": 1,
""offer_file_id"": 0,
""offer_id"": 1,
""order_id"": """",
""payout"": 1.0,
""payout_cents"": 1.0,
""payout_type"": ""cpa_flat"",
""pixel_refer"": """",
""product_category"": """",
""promo_code"": """",
""refer"": """",
""revenue"": 2.0,
""revenue_cents"": 2.0,
""revenue_type"": ""cpa_flat"",
""sale_amount"": 0.0,
""sale_cents"": 0.0,
""session_datetime"": """",
""session_ip"": """",
""sku_id"": """",
""source"": """",
""status"": ""approved"",
""status_code"": 52,
""timezone"": ""America/New_York"",
""transaction_id"": """",
""tune_event_id"": ""adc6c18f-5d50-4a0f-a8cc-ff63f06b6336"",
""update_stats"": true,
""user_agent"": """"}Updating an existing Conversion
{
"""action""":"""conversion""",
"""ad_campaign_creative_id""":0,
"""ad_campaign_id""":0,
"""adjust_action""":"""update""",
"""adjustment_sequence_id""":16648209853209,
"""adjustment_timestamp""":1664820985,
"""adv_sub""":"""",
"""advertiser_id""":0,
"""advertiser_manager_id""":""1"",
"""aff_sub""":"""",
""aff_sub2"":"""",
""aff_sub3"":"""",
""aff_sub4"":"""",
""aff_sub5"":"""",
"""affiliate_id""":1029,
"""affiliate_manager_id""":0,
"""browser_id""":7,
"""conversion_id""":329536393,
"""country_code""":"""UK""",
"""created_from_adjustment""":false,
"""created_timestamp""":1664820662,
"""creative_url_id""":0,
"""currency""":"""USD""",
"""current_payout""":7.0,
"""current_revenue""":0.0,
"""current_sale_amount""":0.0,
"""customer_id""":0,
"""date""":""2022-10-03"",
"""datetime""":"""2022-10-03 14":"11":02"",
"""event_id""":""e9cf2e32-3024-46a1-a089-bb56967f83aa"",
"""goal_id""":0,
"""hour""":14,
"""ip""":""10.1.1.10"",
"""network_id""":"""demo""",
"""offer_file_id""":0,
"""offer_id""":1,
"""payout""":0.0,
"""payout_cents""":0.0,
"""payout_type""":"""cpa_flat""",
"""pixel_refer""":"""https":"",
"""promo_code""":"""",
"""refer""":"""",
"""revenue""":0.0,
"""revenue_cents""":0.0,
"""revenue_type""":"""cpa_flat""",
"""sale_amount""":0.0,
"""sale_cents""":0.0,
"""session_datetime""":"""2022-10-03 14":"08":34"",
"""session_ip""":""10.1.1.2"",
"""source""":"""",
"""status""":"""rejected""",
"""status_code""":12,
"""timezone""":"""America/New_York""",
"""transaction_id""":""102dab988204e09a8a6fd048f5c480"",
"""tune_event_id""":""455226b0-e485-4284-9853-4c152dbdece6"",
"""update_stats""":false,
"""user_agent""":""Mozilla/5.0 (iPhone; CPU iPhone OS 15_5 like Mac OS X) AppleWebKit/605.1.15 (KHTML,
like Gecko) Mobile/15E148""
}
Field Updates
Updates to existing events allow a subset of fields to be modified. These fields will need to either be updated on the original event record or a window function must be used to take the latest adjustment_sequence_id field. The value of the adjustment_sequence_id field can also order the adjustment ledger for each TUNE Event ID.
*The TUNE Event ID is the unique conversion identifier used by TUNE. This ID will need to be included along with the adjustment stream to update existing conversions in the 3rd party destination location.
Handling Event Deduplication
Each tracking event has an alphanumeric identifier, event_id. Use this value for deduplication purposes. Do not use the dispatch_id in the message envelope for deduplication.
The value of event_id is guaranteed to be unique only within a given network. If developing code to work with multiple networks, you must validate against network_id in conjunction with event_id.