Mixpanel to Keen.io Migration

Alex Egg,

At LuckyBolt we used mixpanel for over a year to aggregate all our business metrics. It has pretty good analysis and reporting tools too. It has a nice funnel feature where we can track the conversion of a user from their first site visit to signup to first order. However, doing simple things like displaying our order $ volume per day is not possible w/ mix panel (I know it’s not possible b/c I’ve gone through support with them a few times). Also, another issue with mix panel is that the software feels like it is abandoned – the clunky UI has not been updated in the whole year we were using it. For these reasons the stakeholders at LuckyBolt just didn’t adopt the platform w/ the same gusto that I did. I also tried just using mixpanel as a time-series object store a-la influxdb (which is all I’m convinced mix panel is on the backend), but I was stymied by the lack of tooling to run sophisticated queries on my data.

To get your data out of Mixpanel you have to use their API. I’m using it w/ their ruby client, which has a simple DSL:

require 'mixpanel_client'

client = Mixpanel::Client.new(
  api_key:    m_api_key,
  api_secret: m_api_sec
)

  data = client.request(
    'export',
    from_date: "2016-01-17",
    to_date:   "2016-01-18"
  )

This will spit back all your events for 15 days in January 2015, e.g.:

[{
	"event": "Signup",
	"properties": {
		"time": 1453098295,
		"distinct_id": "15748",
		"$lib_version": "1.7.0",
		"City": "San Diego",
		"Company": "BioLegend",
		"DeliveryPoint": "BioLegend",
		"email": "[redacted]@biolegend.com",
		"mp_lib": "ruby"
	}
}, {
	"event": "Order",
	"properties": {
		"time": 1453100786,
		"distinct_id": "15729",
		"$lib_version": "1.7.0",
		"DeliveryArea": "Longmont",
		"DeliveryPoint": "DigitalGlobe - Longmont",
		"Items purchased": ["Turkey, Bacon, Avocado Sandwich / Off Campus Cafe"],
		"mp_lib": "ruby",
		"tip": "0.9",
		"total": "10.4306"
	}
},...
.
.
.
]

The suggested way to import into Keen.io is by using the CLI they provide: https://github.com/keen/keen-cli . However, it’s just a wrapper around their ruby API bindings, so we can just use the keep ruby API bindings and make the migration in one script. The CLI import command uses Batch Publishing behind the scenes. It reads your input file line-by-line and adds it to the batch collection.

{
    "keen": {
        "timestamp": "2016-01-22T06:20:07.483Z",
        "created_at": "2016-01-22T06:20:07.483Z",
        "id": "56a1ca172fd4b15539cc28ca"
    },
    "items": [
        "Mahi Mahi Bowl / Embargo Grill",
        "Roast Beast / The Curious Fork",
        "Tuna Avocado Wrap (GF) / The Curious Fork",
        "Tuna Avocado Wrap (GF) / The Curious Fork",
        "Shrimp Bowl / Embargo Grill"
    ],
    "tip": "9.03",
    "delivery_area": "Genesee",
    "delivery_point": "Applied Proteomics",
    "time": "2016-01-18 08:25:16 -0800",
    "total": "71.566"
}    

Permalink: mixpanel-to-keenio-migration

Tags:

Last edited by Alex Egg, 2016-01-22 06:21:23
View Revision History