See the entire conversation

Programmers worrying about whether their architecture will Web Scale is like buying a lottery coupon and fretting about which yacht to buy.
71 replies and sub-replies as of Jul 13 2017

Strong advocates of the Law of attraction lol
Clients are also guilty of this - worrying if their totally new & unheard of platform will cope with the 100k users that they don't have yet
This is so true. You are wise!
When you scale, it's because you're prepared, there's no need to worry! If you're worried, don't, because you won't scale until u prepare
OMG! The best of the day.
programmers NEVER worry about being webscale. If it works on their workstation it will work anywhere ;)
Don't fret, just use #FunctionalJavaScript, immutable data, and promise.then(buyYacht) to unlock #FutureScale 😂
given that a post on reddit can bring a flash mob,isn't it worthwhile to have proper skills to milk them.With awareness attention is fickle
especially with how some things are ephemeral like calls to action like the present net neutrality thing, fads, games-they still matter
i don't think it's crazy with market entrance being so hard to enter coz of the big fish tinyurl.com/y986xjky whose main weapon is scale
The end of the internet startup
We haven't had a major new technology company in more than 10 years.
vox.com
front paged on Reddit, HN and Product Hunt simultaneously last week and peaked at 10 pageviews/sec. Social traffic isn't that big
Also he did it with PHP *shudder*
Yep, this week I was @ProductHunt #1, @newsyc #1 and @reddit frontpage with hoodmaps.com; it was 3 page requests per second max
Hoodmaps 🗺 Neighborhood Maps
hoodmaps.com
Making pages static kept the site up. Took 5 mins to write script that generated cached pages. One-man-band!
For reference: a properly configured VPS can do 1M reqs/s
Do you serve it with nginx/apache directly or it has to go through php level? I love to hear more
First served NGINX -> PHP then too many requests cuz @reddit frontpage, then cached PHP > HTML, then NGINX -> HTML fast
Yup. That is what I have in mind if I ever need to serve that much traffic, thank you
Think this is a big key: ensure the server is configured to handle the requests.
Most Aus sites/providers I've seen the back end of aren't ready to handle a Reddit load because they haven't experienced near those numbers.
1M as in one million right? One TCP server cannot serve this many by design
returning 502 Bad Gateway and stuck at loading as we speak
i think i may be misinformed by some articles i've come across about sites crashing under heavy load due to sudden hype
They do crash under sudden load. That's why most sites don't even reach @reddit frontpage. But making site static HTML quickly fixes it!
Yeah it's the gradient, not the total load. Scaling w/in minutes requires horizontal scaling & shared-nothing architecture, like Rails.
The Hyperion, obviously.
Depends on where they work
exactly. code now and get it working. when you run into scaling issues (a good problem to have), get funding and hire ppl to scale it.
I think there's a difference between over engineering for scale prematurely & making design decisions that don't paint u into a corner later
The slowest software is the one that no one wants to use.
Although it's good to be aware of potential problems. The art is spending the right amount of time on the right problems at the right time.
That said I agree, in general many devs would be better off if they could let go of fear of things that 99.99% likely will not happen.
a guy failed to withdraw his winning bcz of citizenship. Had he declared two weeks later money would have been in his account
Do you agree Peter?🙂
Yeap. Been there made that mistake.
to make sure I understand was that an argument for "worrying about architecture once you have a problem?"
This seems to be for "if you're big enough to worry about scale, you can figure out the 'perfect' architecture later". But I don't ...
... agree with this totally. If you can, be good as possible to begin with. But not at the cost of time waste if it's just a PoC, forexample
seems reasonable, that's why I wanted to make sure I understand the author :)
You build for today's customers first and worry about tomorrow's customer tomorrow. No point scaling a dead sheep.
Wise words @panondo. Also you don't know the bottlenecks you'll have at that time. Could be in a part you didn't event build yet
Especially in the prototype phase. If / when it's time to scale, you should be able to hire the correct team.
Until you get to the stage where you do need to scale and have to spend 6-12 months re-writing everything out of RoR.
when it's time, know ye that rails can be made to grow huge. dcos mesos docker containers yes rails can run there too
A lot of people do this! Doing an overkill of tech for a platform of 20 people!
I feel that programmers worrying about web scale are more like that one pig who used brick to build his house.
That's preparing for a known, likely situation. This would be more like if he built a nuke bunker when he only had a wolf to worry about.
Things are only as complicated as you want them to be. You can prepare for a level of scale without having to prepare for the worst case.
That's the point, right? Someone who is starting something from scratch and preparing for Facebook-level traffic is likely wasting time.
I CAN DREAM CANT I
tell that to that pokemon guys = )
What kind of programmers are we talking about here? It actually matters.
+1 for Sorin. Context matters here (as usual)
I like though how people worry about maintenance costs as well, and sometimes having smaller services for example helps with that.
This is really really good!
The fucked up part is: almost every startup with 0 customers now interviews devs like they already have 1m rps.
Wait, I spend a lot of time fretting about which lotto yacht to buy. Is that not normal? Maybe I'm just a boat person ...
Programmers have no idea what a shitload of traffic you can serve from a single box.
Fwiw, there is a major difference between having scalable arch out the gate and not coding yourself in a corner for the future/having a plan
They are worrying about which programming language is the best one !!
And they bought the lottery ticket with a credit card.
i love that thought! ... 100% agree that unfortunately mostly wasted thoughts as never reach (or come close) to that point :/
...what precisely is web scale? Sounds like an ambiguous, and dated term, that doesn't characterize what most large-scale apps are: mobile.