Thread time! Google just announced a thing called "The Data Transfer Project." You might be wondering why they did this and what it means: read on!

I started The Data Liberation Front in 2007 and we basically went from product to product inside of Google adding export buttons. This was pretty cool since it made it easy for folks to copy their data out of Google.

This was cool, but Google was adding products at a pretty good clip and just keeping up was hard, and, if you were a user of Google products, having to go to every. individual. product. to export your data was rather onerous. We needed something better.

So in 2009 we started working on a product that we called "Foil Swan" -- as in the packaging that some restaurants would wrap your leftovers in for you to take them home. The goal of Foil Swan was to create one place for users to download ALL of their data in a zip file.

This launched (on the same day as Google+!) in 2011 as "Google Takeout" with just a few products included: Buzz (!), Contacts, Picasa Web Albums (now Photos), your Profile, and your G+ stream. We even made a ridiculous video announcing it:

Over the course of the next four years, the team added EVERY Google product to Google Takeout. This is actually an incredible technical feat, and while it's now called "Download Your Data" (boring!), you can still find it at takeout.google.com

But.

Downloading your data in a zipfile isn't all that useful, especially if you're on a mobile device, or a Chromebook. And you'd then have to upload it to another service. This was doubly inconvenient if you were just using Takeout to make a snapshot/backup of your data.

Of course, we knew this all along--but you have to walk before you run, so Takeout to zip file was what we did, and I'm super proud of that work. The next step, of course, was to allow folks to download their data directly into another service.

And over the last three years, that's what the team did (although without me around because I left Google in 2014 to start a little thing called Tock (exploretock.com)

So, thanks to the hard work of the Data Liberation team, now you can use Google Takeout to push your data *directly* to Dropbox, Microsoft OneDrive, Box, or even Google Drive. This is absolutely amazing.

But.

The Data Liberation team doesn't have the time to write code to push your data to every service out on the internet (or new services that pop up). Clearly they chose the big ones. What if you start a new company like Daisy's Delightful Data Depot?

The answer is that you need some sort of connector -- an interface that Daisy's Delightful Data Depot can use to say "Hey! We're over here and you can export your data to us!" That was the missing piece. That piece is, yep, you guessed it, The Data Transfer Project.

So now, instead of Google having to do the engineering work to add Daisy's Delightful Data Depot and every other company to the list of places you can take your data to, Daisy's team does the engineering work to use the Data Transfer Project's API.

And all Google has to do is do some detective work to make sure that Daisy's Delightful Data Depot is a legitimate business and allows for users to download data from their service as well. What was once a ton of eng work for Google (or Microsoft, or Twitter), is now much easier.

So, in short, not only is this really cool, but it's one of the final steps (but not THE final one!) in a journey that I started *eleven* years ago. And I couldn't be prouder of the team that I left behind at Google to continue this work.

They've done everything I've dreamed of doing and more. Can't wait to see what cool stuff comes out of this!

There are more stories of the Data Liberation Front, but those are for another time. Just for fun, I'll leave you with this team photo from 6 or 7 years ago. Let's just say that we had a lot of fun getting the word out in The Early Days. :-)

You all did *amazing* work on this. It was interesting to see it get mirrored in Twitter, Facebook, Vine, etc downloads. Was there any discussion about using small client side webapps (like Twitter's Grailbird) for browsing the downloaded data?

That was never a goal of mine -- my goal was to get the data out so people could write things like that :)

Been waiting here 5 hours for another "But", can I go offline yet?

I'm not telling!

Nice! Congratulations. Everything you do is great!

that is an extraordinary feat!

Thanks!

SWANS FOR THE SWAN GOD! PACKAGING FOR THE PACKAGING QUEEN

This project has had such an influence. Not just on Microsoft, Twitter, and Facebook joining the initiative. Also GDPR's inclusion of data portability as a consumer right. Thanks for getting not only a discussion going, but a working implementation.

The DTP is all the current team, not me, but thanks 😀

Right, but you started it. If y'all hadn't done that work 11 years ago I never would have believed it could have happened.

I'll take that! 😀

as someone that seems to enjoy convincing computers to do data plumbing like this for a living i am thoroughly impressed in what you've started. as @nelson points out, it takes a lot of work to keep the ball rolling, so thanks!

also PS: thanks for writing the thread. it's good context that i can use when i try to get my employer to participate :)

Dude I wish you well on this insane project I won't even tell you why you're dooned

Not my project but thanks!

I love you

I know.

assuming you're already familiar with this?

, you are so behind on this. You gotta get some real agents in the field. I mean, srsly:

I was on vacation last Friday :)

This will teach you to take vacations.

How 'bout it!