Convopage
See the entire conversation
Time to put on my Cloud Economics Pants and do a bit of math around @Cloudflare's R2 pricing model as described herein.
Announcing Cloudflare R2 Storage: Rapid and Reliable Object Storage, minus the egress fees
Introducing Cloudflare’s S3-compatible Object Storage service, with zero egress bandwidth charges and automatic migration from S3-compatible services.
blog.cloudflare.com
188 replies and sub-replies as of Oct 02 2021
So today I'm going to store 1GB of data in @awscloud's S3 and serve it out to the internet. The storage charge is 2.3¢ per month the tier 1 regions.
Someone on the internet grabs that 1GB of data once. I'm paying 9¢ to send it to them. You read that right; just shy of four months' of storage charges to send it to the internet once.
As described, R2 sits in CloudFlare's world. The first time you request an object from S3 via CloudFlare, I pay 9¢ to send it out, then 1.5¢ a month to keep it in R2.
And from that point forward egress becomes free. But I'm not done.
Now look, @eastdakota strikes me as a stand-up guy, but he's a network guy; my data is SUPER important. I want to keep it on S3.
I can cut it over to use S3 Infrequent Access. This drops the price on AWS to 1.25¢ per GB per month. Should R2 break and need to re-retrieve it again, I'll pay another 9¢ to transfer it out, plus a 1¢ surcharge for retrieving it from Infrequent Access.
Let's tie this together. I can pay 2.3¢ per GB plus a whopping 9¢ per GB of transfer, *OR* I can pay 2.75¢ per GB to keep it in both places, secure in the knowledge that my egress traffic is a one-time 9¢ charge, the end.
Would I take that deal?
Dear reader, yesterday I would have sold you for glue in order to secure that deal. Today I don't have to.
The only response @awscloud realistically has is to significantly cut their egress pricing in one form or another, in which case customers win.
(A "surcharge to CloudFlare" or whatnot would destroy trust in their business and is untenable.)
This is frankly brilliant of @Cloudflare. I'm just waiting for an astroturf campaign ineptly trying to cast shade their way about how dangerous / risky their object storage is, but as mentioned upthread I can mitigate data loss risk by keeping it both places simultaneously.
All of this is of course the worst case cost model if you distrust @Cloudflare not to lose data. Trust it (something that comes with longevity) and the economics improve by a lot.
One final point: Now let’s remember that the internet is 1-to-many. If 1 million people download that 1GB this month, my cost with @Cloudflare R2 this way rounds up to 13¢. With @awscloud S3 it’s $59,247.52.
THAT is why people are losing their minds over this.
Slight correction: $53,891.16. Apologies, the @awscloud pricing calculator LOVES to slip "developer support" onto the tab.
Who doesn't like to slip "developer support" onto the tab?
I mean, come on. "Tips welcome".
Ugh. Those kind of prices make you think about your own iron. I’d love to see a comparison to @storj
Is this an introductory price by CF to steal aws customers? This doesn't seem sustainable?
Steal is placing an inherent value to the question in terms of rightness or wrongness which is unintended.
That’s the hell of it: no. It’s complicated, but this gets at it:
The Relative Cost of Bandwidth Around the World
Over the last few months, there’s been increased attention on networks and how they interconnect. CloudFlare runs a large network that interconnects with many others around the world.
blog.cloudflare.com
It’s fairly clear to anyone bandwidth adjacent that egress must be one of AWS’ highest margin lines. Honest question: does it subsidize the others?
AWS invests heavily in their network. I’m honestly not sure what the internal incentives are (and if I were I wouldn’t be able to say): it’s definitely a profit center and that’s sad.
Google often trumpeted their worldwide private network 3 or 4 years ago when selling their cloud. Is AWS now competing in terms of private under sea cable offering worldwide transmission of traffic on a private network?
They charge extra for it; it’s called Global Accelerator.
It seems to me that a significant motivator is the stickiness of data. Once it is in you don’t want to move it. Doubly so if it costs an arm and a leg to move it. The only way to make that true is costly egress. It makes moving more to aws compelling.
When will Corey discuss VMware and how they allow backdoors to everyone's accounts upon simple requests and saying that you're in cybersecurity. Wowsers!
In fact, @VMware doesn't even keep track of their customers! Then they blame the ones who end up with backdoored VMware crap all over their systems that don't know how to fix it, because they're not in cybersecurity! My God, they've got nerve!
Who in their right minds would think it's okay to put 10 VMware accounts on top of someone's YouTube account so that anytime they use YouTube, it burns their CPU up within 5 minutes? @VMware, because they don't feel responsible to know what their customers do w/ their products.
Even as a fiber customer lucky enough to get gig up and down at home for $80 a month, it still blows my mind how affordable bandwidth has become. I still treat my home internet as if it's dsl or cable.
still not giving Cloudflare a cent of my money if I can help it given their steadfast support for the people who doxx me and my loved ones :(
This is pretty much my stance. Love seeing competition in the market. Inspired by some fantastic technology individuals have developed.
Putting my money where my mouth is (as much as I can afford it) because I think CF have made (and are still making) atrocious ethical decisions.
Honest question here, who is the ethical option?
None. Perfectly. But CF is more blatantly bad than others are perfectly innocent.
Ohhh good call. I should revisit their pricing for my next project!
Unfortunately fastly due to egress and request costs per month would make it prohibitively expensive. I couldn't sell management on that. Is that the best option?
There’s that “mass exodus” phrase again.
Related- at a recent all hands the CEO of Fastly invited anyone who lost faith in the company or its culture to leave. Not surprising that a large, and growing, number are taking him up on that invitation.
Have multiple friends at Fastly who haven't told me about serious problems there; you have linked a single Glassdoor review rather than multiple pieces of evidence
also we're talking about ethics here and the bare minimum of refusing to serve Nazis, heh
Do you know for sure about an yet-unreleased product that they will allow infinite egress for "free"?
On the other hand, I'm sure you know that those who deliver 1PB of data from S3 are not paying the price you listed.
1PB per month egress on AWS is where discounting starts.
If I hit that kind of transfer on CloudFlare, they call me. If I hit it on AWS I get billed an amazing kitchen remodel.
The first statement is untrue, but hey I am just an AWS Solutions Architect while you are world renowned Cloud Economist so 🤷🏻♂️
Okay, post negotiation you’re then comparing a new Toyota to half a quarter.
And yeah, the 1PB commit starting point is a different data transfer dimension I was misremembering. Oops. It’s been a long day.
And lastly, if my 1GB file blows up and goes viral, yeah: I’m paying that rate for the first surprise month. I can negotiate going forward; getting a concession retroactively is going to cost me some favors.
accessor pays works if people are already AWS customers, but not for HTTP transport to net etc
Transfer from S3 doesn’t cost the consumer unless requestor pays is enabled. “Receiving side pays” is only cross-AZ, and S3 has no cross-AZ charges unless you’re doing something that’s just deranged architecturally.
I appreciate the way you compare prices.
I personally had been equating a RedShift Cluster to “a gently used off-lease Camry every month”
now I wish Cloudflare built AWS VPC equivalent (EC2, etc) and I won't need to pay expensive AWS NAT Gateway anymore 😁
Them and everyone else is scrambling to build it.
tried Azure and GCP, still easy on AWS for me, and maybe just because I am lazy to change my Cloudformation script to something else 😂
Huge. Plus, if I'm reading the article right, they're going to make infrequent access (under a few requests per second) completely free — also game changing.
I really want to kick the tires on this.
How does Workers KV pricing compare against S3 for sub 25MB obejcts?
That storage price is cheaper than you can buy it yourself. $15 a TB. That's below average hardware cost.
S3’s $23 per TB could be accused of the same, and yet here we are.
I'd say that's right on hardware price per TB. But the killer is the access cost like you said. Economics just got a big delta.
Remember that that’s per month.
Ah. Yeah. Missed that.
B2 is $5/TB/mo...
$15/TB/m is nowhere near cost of they are spreading over 3 years. Even as a tiny provider we could come in at below that for the hardware side and we have nowhere near their purchasing power or the same power efficiency levels with their custom hardware etc.
Yeah I messed up on the monthly. Over a three year period you could have full raid, backups, footprint, everything and a big margin.
My experience with DO Spaces. They probably improved things since then, YMMV. I can't find their post mortem anymore though, dead link.
Yes, so basically there is a $59k range to play with. Problem with egress cost is that it is unevenly distributed worldwide. What do you suggest? (at @Scaleway the cost for your use case is between zero and 10k€ depending on how you code it, without CDN that is).
We would like to use Scaleway Object Storage as origin bucket for Cloudflare R2 with a 1 month TTL, is there any plan to honour the Bandwidth Alliance agreement waiving the egress cost to R2? The last time I checked with support they confirmed that the alliance was not applicable
umm, shouldn't you be comparing Cloudflare to AWS CloudFront (and getting a much less exciting headline...oh, I see my mistake)?
No, because they just announced their own object store.
But the economics of CloudFront data transfer are both more variable and in many cases *worse* than these examples.
Object storage with default cdn in front of it?
WHOAH THAT’S CRAZY
So... wait. Is this just Egress to other Bandwidth Alliance endpoints (ex: Backblaze)? Or could ex: indie game developers use it as a download mirror?
This sounds a lot like the edge caching that was supposed to be a major upgrade to the internet but never really materialised. Cloudflare seems to be really into building the missing infrastructure that never got built because it would lower rents
If you want to serve an object that frequently on the internet, S3 is not the solution anyway. A CDN/object cache is the way to go. AWS offers Cloudfront for this exact purpose and R2 sounds very similar to that. Might be a little cheaper.
R2 is very much unlike CloudFront. For R2 transfer is free, for CloudFront just slightly cheaper in the best case scenario, and much more expensive in others.
CDN Pricing | Free Tier Eligible, Pay-as-you-go | Amazon CloudFront
Pricing details for Amazon CloudFront's global content delivery network (CDN), including the AWS Free Tier. You pay only for the data transfer and requests used to deliver content to your customers. With CloudFront, there are no upfront payments or fixed platform fees, no long-term commitments, no premiums for dynamic content, and no requirements for professional services to get started.
aws.amazon.com
Thanks for pointing out my half-assed comment of these services being similar. They totally aren't now I see. I'll test the service out in coming weeks!
save thread
First public save of this thread! 🏆
Readwise users: Like this reply to save QuinnyPig's thread to your account without cluttering their replies 📚
Stats:
• 109 total saves of QuinnyPig's threads (ranked #290)
One question if application is at initial stage then will it be ok to start with self hosted and then switch to managed as user base increases ??
I'm genuinely excited. @Cloudflare had a solid answer to Dynamo DB I could almost move my entire cloud workload over to them.
Nonsense. I'm spinning up my 'proxy to CloudFlare' startup on the assumption that's the next step.
I will simply do 12 9s of durability and pay out SLA breaches exclusively in @Swagbucks
That's the best part; it works in-line that way natively. Point Cloudflare at your S3 bucket and it does this seamlessly.
Once its reliability is established, though, if you have a read-heavy workload why would you even utilize aws s3 in the first place? Wouldn’t you just skip directly to this instead where you have the choice?
Eventually, but I'm not one to trust a new storage service at launch.
My modeling above is if you completely distrust CloudFlare not to lose your data. It's the *worst case scenario.*
Understood. Kinda wild that they priced it to be compelling even in that light.
S3 storage prices were last reduced in 2016 and obviously *looks up* the egress discussion.
I would hope they’re positioning for a service expansion after launching Cloudflare workers then this. They seem to do serverless really well. The two could even combine as a better version of Object Lambda Access Point but who knows lol.
What about the use case for backups, where I'm not planning on retrieving much at all, and when I do it's probably only once, but it will be large and "one time transfer fee" kills it.
WHHHYY does it make most sense to store this on a CDN? The world is broken.
I want to see this GA and at scale before I trust backups to it, but the one time transfer fee isn't usually THAT egregious; it's 4 months of storage in standard, 8 months of storage in Infrequent Access, and I Must Consult the Bones for Glacier timing / pricing.
Why wouldn’t storage at a CDN make the most sense? A CDN of sufficient scale necessarily has mountains of storage at each POP. They’ve just found another way to monetize it by letting you exceed overstay their cache algorithm’s lifecycle time by paying $0.015/month/gig.
I am with you. I really cannot wait to try it too.
It’ll be interesting to see reports on these things:
1. Will the major compute providers provide adequate peering to efficiently use this service?
2. What will rate limiting on writes look like?
3. How does their eventual consistency model perform with a distributed application?
1. Yes. Cloudflare isn't fly by night; they have a 100Tbps network today.
2. I suspect it'll mostly be a pricing question but we'll find out.
3. I'm as curious as you are!
1. This depends on the cloud providers, too, and any (theoretical) anticompetitive steps they might take to limit the peering bandwidth, absent net neutrality.
What's in it for CF? How are they making money on this?
It costs them less than 1.5¢ per GB to store your data, and their transit / peering agreements aren't slapped with a markup to afford sending @eastdakota to space.
It’s revenue they wouldn’t otherwise have (since all that revenue was previously just going to AWS?)
As to exactly how that revenue is converted into profit (and how profitable it is) I would have to defer to a cloud economist like @QuinnyPig..
I do love the idea of putting it in front of S3 as a permanent cache/backup solution that’s significantly cheaper to serve from.
I just wish most of my S3 workloads weren’t write-once/read-*mostly*-once but keep around forever.
Right. That’s what’s going to be interesting to see.
“Your margin is my opportunity” stings when it means having to cut back on the il’ space program eh.
It’s already a no-brainer if you download your object just twice…
Although metadata consistency might have a lower isolation level?
Well AWS won’t join bandwidth alliance. So yeah y’all pay that $0.09 per GB egress
Right. *Once*. Then that object lives in R2.
Yes sir that is correct. However, I doubt that R2 would suffer the same headaches as US-East-1 in AWS 🙃
I think I'm missing something: wouldn't it be cheaper and simpler to just put the bucket (S3 or R2, whatever... wherever you feel more secure) behind a traditional Cloudflare CDN, and not use a second bucket at all?
I'd have to disagree. He, along with others at the company, play games and refuse to answer simple, direct questions with straightforward answers.
They still claim they don't "host", as if the only definition of hosting is storing the files served for a web site. Disingenuous.
Great thread, Corey! Now the question is - how the hell can Cloudflare afford to provide such liberal pricing? Network ain't free for them either, right?
AWS charges 1998 prices for bandwidth. The markup on it is intense.
Thanks, Corey. That makes sense but even if it is low it is not zero. How can Cloudflare afford to make egress free? Is it just me being suspicious of the free cheese?
Because it gets subsidized by their upmarket offerings. AWS charges (in some cases) a roughly 80x markup or more.
AWS’s Egregious Egress
Amazon’s mission statement is: “We strive to offer our customers the lowest possible prices, the best available selection, and the utmost convenience.” And yet, when it comes to egress, their prices are far from the lowest possible.
blog.cloudflare.com
Also, their entire platform is one giant global network, handling 100Tbps currently. They can move a data anywhere for pretty near nothing.
I think the subsidized bit is that this distributed object store needs a ton of storage (across 250+ POPs) to support replication.
This! Precisely this @inadarei.
And having their own object storage means you can build more with their other (paid) services like Workers and Sites.
I think if you create something like Netflix2, Youtube2, DockerHub2 ... they want to talk with you about completely free egress. But at that point you have 3 options:
1.) Pay the charge and price it in
2.) Create your own distribution network like Netflix
3.) Buy Cloudflare 🤣
To continue his train of thought, everything you are saying makes sense. However, I find it EXTREMELY hard to believe that 13 cents covers even the variable costs of sending 1 mil GB, let alone contributing to fixed costs as well. Loss leader approach perhaps?
Utility bandwidth is bought by the size of the pipe, not how much you send. Also CF has very low fixed costs because of network peering.
Real news here is a storage product that allows them to capture revenue they missed since they always had free bandwidth regardless of origin.
They did write a blog post to answer this question.
AWS’s Egregious Egress
Amazon’s mission statement is: “We strive to offer our customers the lowest possible prices, the best available selection, and the utmost convenience.” And yet, when it comes to egress, their prices are far from the lowest possible.
blog.cloudflare.com
there's a very good analysis (as usual) from @stratechery here:
Cloudflare’s Disruption
Cloudflare’s new storage offering is potentially disruptive both economically and strategically.
stratechery.com
Thanks for doing the math lord knows I wasn’t gonna
S3 -> R2 -> ... Q1 ?
So the next big thing is someone launching Quantum storage? with subatomic billing I presume.
Didn't that cosmos db already have a big bang recently?
P0 storage is when it raises the on call guy to get the data from the data fridge and mail it to you.
make P0 if you ever do object storage
Hmm.
How much money do I need to pay to get it renamed
Followed by O-1 which can be read as "O/I" as in "the inverse of I/O" meaning you get paid when your object gets accessed over the network.
i think that's called advertising
Now I want this.
Q1 ... OMG 😱
Q + 1 ... 1 like the first letter of the alphabet ... i see.
OH MY GOD ITS QAnon QAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnon
Cloudflare and their global network of bandwidth.
They are the REAL GLOBALISTS! #XFiles
Great thread! I hope you do a blog version of this as well, with some more examples, comparing various scenarios & % savings - I’d like to share that with a few folks!!
Well okay. 🙂 Next week’s @LastWeekinAWS blog post it is.
Check the authorization story as well (ofc it's not IAM compatible)
so what would be the fair (and self-sustainable) market price for AWS S3 egress in 2021?
That’s a fascinating question. A lot plays into it. “Less than three times what a hard drive of that size costs to buy on Amazon” is a good starting point.
we just need AWS to add some cloudfront<->s3 pricing improvements to compete this “s3 cache” solution
CloudFront’s obnoxious problem is that what it costs me varies depending upon where you’re pulling my content from.
and, keep in mind that my cloud is where my data sleeps. so being a cache for AWS S3 for zero 💲 won’t do the job
IMHO, because of this offer, it's "best-effort" = Free, with a premium tier that promises "measures" and "guarantees" and what not.
I think this has the effect of draining any remaining object-like work out of VM's/containers/functions really really fast...
you sound like a person who didn't know that you can get water from the tap instead of by the bottle a dollar at a time
go look into traditional hosting. this is supposed to be free
Q1 ... OMG 😱
Q + 1 ... 1 like the first letter of the alphabet ... i see.
OH MY GOD ITS QAnon QAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnonQAnon
Wowser. That’s cheap. And object storage is becoming increasingly useful thanks to Jamstack.
Surely you need to compare this with Cloudfront, rather than S3 directly?
Okay, then it’ll be 8.5¢ to 12¢ per GB depending upon where you happen to be downloading it from.
If you compare Cloudfront vs Cloudflare directly as CDNs, then Cloudfront has lost “relative” marketshare the last 4 years. But it isn’t just a storage service
How does this all compare to Azure?
Want to do this for @Azure ?
Yeah I recently switched to CloudFlare and it's making me even more angry at how expensive AWS is.
Bezos’ space flight won’t fund itself
Switched to CloudFlare for the CDN or everything? What about databases?
Still use cognito and dynanodb. The integrated IAM is worth it
How is latency between CloudFlare workers and AWS?
Dynamo is v low latency
CloudFlare workers also come with its own k-v style db you can use that is included (free + pay). But I haven't used it yet.
What a fascinating thread. Great insight here. This is why we help people avoid vendor lock-in! Thanks for sharing.
Wait, I am confused, their egress is $0?
It is hard to believe, because we so used to paying insane fees to all the other CSPs... 👀
There's much to like about CloudFlare.
CloudFlare workers, for example.
Redirection done
Creating a redirect is in itself not hard. A redirect based on a mix of certain conditions, now that can get tedious. Often, you end up…
mohammedbrueckner.medium.com
and workers aren't just for the occasional requirement, I run our entire API out of them, 500 req/sec without a hiccup
I suppose "With no egress fees" means free from everywhere.
This type of economics allowed me to bootstrap an adtech company vs raising money.
Do I correctly understand that if we have outbound traffic 200TB/month from cloudflare it costs nothing for us? Or I misunderstand that new pricing model?
(No comment)
This is an excellent development and as business/consumer I'm really liking this race to the bottom. It won't help my use case (infrequent egress transfers) but as you say it should drive AWS own prices down.
Something I haven't seen yet is whether or not there's a limit on egress amounts. With services like Wasabi, you're expected to transfer out less data per month than you store. Do we know if R2 will have similar restrictions?
I am in love with CloudFlare's slow expansion in to the same space the big clouds are in. It doesn't do much to solve the "big US megacorp" phobia -- but I have always loved the idea of using the more modern cloud technologies... I refuse to pay the $90/TB AWS tax though.