Convopage
See the entire conversation
512 replies and sub-replies as of Nov 07 2017
Didn't read it thinking it was a click bait.
why do you think it's garbage? It's talking about the raise of federated networks vs centralized cloud providers.
I don't think anyone wants to give their private data to a random peer on the network. And it has to be plaintext to deliver most apps.
think of all the DB queries you do in a normal app and what %age you could do if you didn't have access to the plaintext
It's not hard to have privacy in a P2P network. DAT protocol, for example encrypts everything in transit. DHT lookups use opaque keys.
We have really strong technical and product reasons for existing!
Let me paint the picture- you start with P2P publishing protocols like @dat_project, bittorrent, and @IPFSbot to distribute apps & user data
You use these networks as a replacement backend. You can abstract them to feel like databases.
They're async networking protocols that support offline writes. So, technical advantage: offline-first.
You optimistically sync a fair amount of data and do the reads and writes locally. Technical advantage: lower latency because a local db.
The @dat_project protocol uses a signed ledger to record all data. Technical advantage: offline-chekable data integrity.
(As opposed to the live only, connection-based authentication we have now with https. See beakerbrowser.com/2017/06/19/cry… for the innards.)
Cryptographically-secure change feeds in the Dat protocol, and on the Web
Dat uses an append-only feed that’s inspired by Certificate Transparency to create auditable change logs in file archives. We use this to deliver Web applications securely in Beaker.
beakerbrowser.com
From a *product* standpoint, the huge opportunity of a P2P Web is how it simplifies and relocates the stack
Instead of the client/server model, you're moving all data and networking and business logic into the clients
Which is superb because that means users can modify the code and move their data from app to app. Super customizable.
And because all data is being published on a crypto network, we'll be building key distribution into the network, so we can e2e encrypt
In @BeakerBrowser, you can go to a website and "fork" it, just like forking a repo in github. You get the code, the data, all of it.
It allocates a new pubkey -- the address of your new site -- and it's deployed. You can share it; the forked site is live immediately.
Now @floydophone if you don't see a little value there for an open and user-first Web, then I can't help ya 🍺
Slow down there. What you’re saying sounds nice but it’s not going to work with today’s tech
On fb you can full text search for posts and only the stuff you’re allowed to see is returned. Privacy setting changes dynamically
How do you implement this p2p without sacrificing availability (ie making people host their own shit)
Did this make any sense I’m jet lagged haha
Totally makes sense. Managing privacy & perms with e2e crypto and distributed networks is much harder than in centrally managed systems.
If you change a privacy setting, you basically have to reencrypt the content and ask nicely for the previous recipients to lose their copy
Technically the same is true of centralized systems, but we don't cache aggressively enough in centralized systems for most ppl to realize
(That is: If I constantly sucked down everything that FB was willing to tell me, then a change in privacy settings would be equally weak)
Availability is also a challenge. It's offset by the ability to reseed other ppl's content, which helps quite a bit.
And there's also public peer services, like hashbase.io
Both of these challenges have to be compared against the possible gains though. It's new tech. It'll have shortcomings
But if we can make it work well enough to do fancy new things, then we're on our way, right?
see i think the p2p adherents way underestimate how bad the UX will be, and how hard (impossible?) the soln is w/o CS breakthrough
and even if all that happens, it will still be far more expensive and slower to iterate on features.
I think the p2p adherents are *terrible* at UX, but that a good UX is more depending on the UX effort than the CS
And the argument we're making is, if we solve UX, then p2p will be an obvious win
But you'll just have to let us prove that out. It can't be argued ahead of time.
no i totally disagree with you. there are fundamental limitations to the flexibility of public key crypto that will always influence ux
Well, note the time and the date, and in a couple of years or so if we meet, beers are on the loser of this bet
Is that unique to p2p?
there is a lot more of this sort of discussion on scuttlebutt.nz an actually existing p2p social network
False equivalence imo: the disks are kept secret. Making every table and search index public presents a ton of likely-intractable problems.
i am having trouble seeing how you'd build the same or similar features fully decentralized while maintaining privacy
Since 2012, I read the end of PC, the end of tablet, then the end of web, then the end of native apps... They are all still there. 😒
The cloud won't die.
It just won't be as important as it is now.
The only thing I don't understand with decentralized apps is economic incentive. No one can really profit from it so adoption will b limited
Still no argument there. It would be GREAT to have useful tools where privacy is protected. But I don't see the path to get there.
Sounds great but unrealistic. I just don't see people spending money to fund a distributed social network. People want free.
I see the tech becoming viable but I dont see people building quality apps w/ polished UI & maintaining it w/o financial incentive.
NOTE: I'm not trying to poo-poo p2p web, just trying to attract attention to a problem that is mostly ignored.
You should see #rotonde!
not sure i agree with this. you could say the same about nascent web.
That’s true. But the nascent web was new. P2P web is competing with the existing web.
I think all that will end up happening is that centralized co's will partially use distributed tech to lower their own costs. (e.g. Spotify)
You. Don't. Have. Privacy. Now. You only have as much privacy as Facebook wants you to pretend to have. You are moving goalposts.
Delegating the management of content to a third party (which is an attention mkt) doesn't create privacy between the original parties.
You're right that, fundamentally, digital content once shared is impossible to unshare. But introducing Big Brother doesn't solve anything.
Well, other than creating easy UX to sucker people into giving up all their information.
If a P2P revokable sharing facility is a hard problem to solve, at least it's the *right* hard problem to solve.
does anyone have *any inkling of an idea* of how to solve this?
it gets way more complicated once you introduce searching and tagging & data gets too big to replicate to everyone
Actually my first thoughts on de-cent emerged when I realized I could fit my year's data input & output onto a single microSD.
P2P web will push more traffic across the network, but I don't believe storage requirements are a major problem.
For large content like video, one can park the raw BLOB on a central server or use IPFS-style backing store.
You don’t need to sell me on immutable data. I get it. But every app needs some shared mutable state somewhere, right? Crdts have limits
I responded to this elsewhere
If you do get beyond user device capacity, you use beefier machines and stream from them, using p2p
That model doesnt have to go away
i wish someone in this thread would take my concern seriously. this is a very simple and clear technical question that no one can answer
the only one who has addressed it is @substack when he said you'd build different features. denial from everyone else
thank you for listening to me. i don't disagree with this. my concern is these constraints have negative UX and dev cost problems today
I’ve actually used it (your Hawaiian house and Dominic’s boat are sweet). Listened to dominic tarr and @andrestaltz podcasts too
Can you help me understand why the most common ssb impls look more like social networks and less like email/listservs?
I ask bcc I feel social networks are in many ways defined by features that must be centralized (recommenders search deletion notifs etc)
Yeah I thought git ssb was cool. Didn’t know about npm though
It can have worse abuse vectors once you build discovery mechanisms into it (those defining centralized things). But I get your point
Actually I think that de-cent identity & distributed karma ledger is the way to allow for discovery while avoiding spam.
can there be an unconscious bias from working at FB in a past life? This is a different product with different requirements.
Oh it’s totally conscious :)
Does it make my concerns invalid though?
You are def right though. I just think users will want some of these features and your p2p app will hit a brick wall
And if I meet you at a conf and we share no friends, you give me a business card with a pub key on it, yes?
Has anyone built a wiki on top of ssb? Or anything that has to resolve a merge conflict or relies on ordering (ie can’t use a crdt)?
I worked on osm-p2p-db which deals with these kinds of problems on top of hyperlog, not ssb
OpenStreetMap Without Servers [Part 2]: A peer-to-peer OSM database
osm-p2p is a decentralized peer-to-peer database for storing and editing OpenStreetMap nodes, ways, and relations. Learn about how it works and how to create your own OSM database and synchronize with other peers offline.
digital-democracy.org
I find it hilarious that a guy from FB is telling us not to trust randoms with our private data.
No ur hilarious
Fwiw I agree, high consistency and aggregation at scale don’t work in pure p2p so I suggest services for that
Thus my pinned tweet! It’ll be a hybrid model. Wrote this too: pfrazee.github.io/blog/achieving…
I do think it’s possible actually & would love to see someone take a stab at an impl. See scs.stanford.edu/14au-cs244b/la…
If you’re willing to use blockchain style consensus it’ll work p2p, but I’m not. Just use a coordinator node when you need high consistency
And/or grow an ecosystem around a protocol that makes fungible the benefits of centralization. E.g. basic storage is already fungible today.
One thing that's been in my mind is how well git repos are fungible today, ... except GitHub issues and comments aren't.
Yes, and Github is not really to blame; TCP/IP, DNS, and client-server model are. To build a website, Github had to own all the data.
Although git-ssb git.scuttlebot.io/%25n92DiQh7iet… managed to pull it off :-)
Will give that a read though
I’m not convinced they’ll figure out how to scale blockchain-style consensus :) electing a master is probably a better bet
Yeah totally agree
Another point is that "consistency" is a big word. Webscale social media defn of "global consistency" is overkill for many kinds of apps.
For almost all apps, you’ll need to replicate a state machine somewhere. Even if it’s a tiny one.
I don't fully understand, the Merkle tree is the app data. Can you elaborate?
^ Im going to be putting out a blog about that today. A traditional service with blockchain-style cryptographic auditability
Apps written for the p2p data web should be OK with Eventual Consistency or even "It's Inconsistent So What". Dat-split is the new netsplit
Ideally the UI wouldn't constantly be bugging the user about data merges. I expect that for most apps, diffs should merge cleanly.
A wiki is actually the hardest type of collaborative document to get right. Most social apps (comment threads etc) have a larger granularity
And a notion of user ownership of the posts that removes the need for coordination
so you are very unlikely to get merge conflicts of e.g. comments on a shared photo.
In SSB, the key is that *only local edits* exist, so a merge conflict would be someone's local edit. There is no concept of an arbiter.
So "leader election" happens on the social level, not on the protocol level. Some people elect someone's edits as the current "consensus".
Exactly. When we all jump into the same Google doc, we don't need CS-level election protocols. If someone behaves badly you boot them :-)
A bit unrelated, but there are more discussion in the scuttleverse about this viewer.scuttlebot.io/%253woQ9BGzDsm…
Ehmmm that's kind of flowery language. You're basically saying you either avoid merges or do them manually, and have no single data view
In some cases that works but in cases where you NEED high consistency, I think you have to be willing to concede to leader nodes
Yes, CAP theorem. I'm just saying that consistency is in the end of the day about socially electing some node (or cloud of nodes).
Either that, or a blockchain-like approach. Or block-DAG, like IOTA Tangle.
What you’re saying is this can’t be done today, right? We have to make ux changes, right?
UX changes compared to what? Facebook com?
Mediawiki
This isn’t even a major point of mine since I think it’s solvable by electing a master. Secrecy is the hard one
Why do you need secrecy for global public content? What kind of secrecy?
I may want to show different aggregates to different people that can’t be precomputed
Ie the aggregate is public but the underlying data is not
If I can’t precompute then I need to either keep a service online (availability issue) or replicate to someone else
The “replicate to someone else” is the secrecy problem and I don’t believe there’s an answer
There is a body of work I can’t find that says a vm can’t hide secrets from the host. Same problem. Also described
Privacy on the Blockchain - Ethereum Blog
Blockchains are a powerful technology, as regular readers of the blog already likely agree. They allow for a large number of interactions to be codified and carried out in a way that greatly increases reliability, removes business and political risks associated with the process being managed by a central entity, and reduces the need for …
blog.ethereum.org
I'm tapping out of this thread for productivity's sake. HMU elsewhere if you want to talk, happy to continue
You ever make it out to the Bay Area lmk and we will debate over overpriced beers
You bet!
Secrecy plus different aggregates to different people: this in the context of mediawiki or what? Sounds like personalized FB news feed.
Yeah sorry. I switched topics on you since I didnt want to debate a problem I don’t think exists.
The aggregates thing shows up everywhere though. Including fb photos newsfeed etc
Can you give other examples?
I'm fairly certain that a P2P web will require users to have agents that crawl their "local" net for updates, info, etc.
Normies can use a service like hashbase.io that does this for you; geeks will run their own. But many such svcs can exist. No one owns graph
maybe users'll publish standing queries, with micropayment bounties for 1st-to-deliver new result, & mercenary agents will race to deliver
Spotify has a lot of features that would fall into this bucket. Probably a lot of b2b too: salesforce, zendesk etc.
Well, it barely makes sense to mix decentralization and business. The former's purpose is often to prevent the possibility of the latter.
The way to maintain decentralization *and* make money is through physical products, more here: staltz.com/layers-of-the-…
I think there’s plenty of room to have capitalist incentives on top of p2p networks that protect privacy. No argument here.
Well, I'm all ears, but skeptic. Recent ideas for monetization in P2P so far have been detrimental to the user.
I'm personally not in the business of doing capitalist business. So the problems you describe are non-problems to me.
Decent monetization models I've seen are either open markets (Filecoin style) or making products (iPhone, goTenna, Beartooth, etc)
I’m not describing these as problems because I don’t think they’re problems. This is a pure tech issue as far as I’m concerned.
I don't have the same view as @andrestaltz about avoiding business; I just think p2p exposes previously hidden costs, and that's good.
w/o capitalist entity in middle, who'd pay for the infrastructure—physical & development—for a hypothetical de-central Facebook replacement?
How do you pay for your email?
Google shows me ads
\O/
But that still doesn't even address my question.
You'd pay for handling of raw bits. I don't disagree that people have to pay for some things.
The point is to avoid convolving that payment with an attention market you can't opt out of.
The state, just like public libraries and roads
Interestingly, this has (for better or worse) already happened in China, and the state paid for it
What infrastructure? I already pay my ISP.
How about database for starters? Where are all those posts and comments stored? Who's gonna pay to have that db hosted / maintained?
All my data stored on my device. Except for pictures, you don't have that much data in a news feed.
For pictures, you can either opt to auto-delete old ones, or use Syncthing at home, or Filecoin.
Where is the feed stored? Where are other people's posts stored, from which they're read into my news feed?
My feed and my friends feeds are all stored on my device. That's how scuttlebutt.nz works today.
Your entire year's worth of data (DL and UL), outside of videos, could easily fit on a modest SD card.
Yup. A tiny SD card that's affordable even in developing countries. That's the "infrastructure".
I not infrequently use Twitter's Advanced Search to search for posts from other people posted years ago.
That's one of the many core use cases I have in mind.
You want a global database. The P2P community doesn't want one, except the part of the community concerned with blockchains.
I can't stress enough that one person's desired feature is another person's perceived bug/issue.
I think a large % of current social network users want one too. If that's true it'll hurt the p2p option from ever being realized.
The decentralized database is global, it's just permission to each piece of it is managed locally.
You cannot query for real-life conversations happening globally. Is that a problem for socializing globally? No.
Well, sure. Requirements for a successful social network differ from the requirements for conversing in real life.
Yes this is the big problem
Shouldn’t this be the job of a search engine?
I've used both - depending on what I remember about the tweet, either one may be more suited.
On my computer right now I have 600mb worth of news feed, excluding pictures. And it's a lot a lot of content.
Ok sure. Now how do you search content posted years ago? How does the system search arbitrary posts from other people u don't have saved?
1. There is a search feature based on leveldb and indexes. 2. I'm not connected to those people, can't search them.
Cool. Getting closer. That leveldb must get massive at scale. Who maintains it? Who hosts it?
What do you mean with scale? How large is one person's mutual social graph?
Partly I'm assuming I want to search people I'm not connected to, like you can w/ me, even tho you don't follow me.
Even if not, if most people follow, say, around 500 people, then years upon years upon years of media, posts, etc add up.
If I buy a new phone, and download whatever app this social network connects to, will it suck down GB's worth of old data to sync up?
Will have to download at least your previous feed, but we are discussing multiple ways of doing this.
Bottomline: if you dig enough, you *will* find problems with decentralization. So what? Everything has a downside. Centralization has too.
What matters to me is the tradeoff I'm making favors freedom over global consistency/discoverability.
As long as we all agree that the decentralized version of X will always be slower, harder to use and less featureful than X we’re good :)
To be more precise: will never exceed X. And will gain you some privacy and control.
Imprecise.
What can a p2p app do that a client server one can’t, excluding improved privacy/control?
Improved privacy/control. :)
And that's not a small feature.
Don’t disagree. Does not conflict with previous tweet either :)
It's underrated though, with full control you actually gain performance in many cases, specially for developing countries with bad internet.
How the new decentralized web can be a more attractive foundation for product development.
Thread:
Paul Frazee✌️ on Twitter
“@floydophone @alexewerlof We have really strong technical and product reasons for existing!”
twitter.com
You use these networks as a replacement backend. You can abstract them to feel like databases.
This is my whole point... the p2p system is really a distributed database. Let’s think about what it can and can’t do through that lens
I think the biggest use case we are missing is caching, and in updating local caches w/ Service Workers before you even request content.
Fine, but don't ignore the fact that that database has extremely different approach to CAP theorem.
I don’t understand
When you don't need concurrent multi-writes to a single row/field/object, CAP becomes much easier. Consider threads in email.
If some EmailMegaCorp centralized all inboxes, their internal db problems would be massive, trying to linearizing every email in the world
However, with UUIDs embedded into RFC822, clients can show threaded conversations without needing to solve really hard DB problem.
Sure, we all use Gmail now, but we don't *have* to. SMTP works great to facilitate meaningful global social dialogue.
Take a step back, pause, and really think about what the user experience of "just using" SMTP w/o a nice front end UX over it would be like
What are you talking about? I use an email client every day. I used pine in 1993 and MailMate and Gmail now. That's IMAP/POP+SMTP.
I've lost the point - I was just saying the centralized entity of Google adds tons of value to SMTP.
The major problem with SMTP is lack of real identity protocols. DNSSEC provides a solution based on centralized namespace.
But what if I do? I would say very close to 100% of multiplayer apps I have built need this.
Yep. It's emblematic that Skype initially was heavily P2P. Its creators made Kazaa prior to that.
The social web today is actually P2P in a higher abstraction. Lower level P2P is dismissed.
Pseudo-peer-to-peer
Peer-to-peer (P2P) is the essence of the internet. Tim Berners-Lee's first drafts of the World Wide Web suggested a platform where content is universally accessible, including universal authorship, essentially what today we understand as blogs and wikis. Prior to that, Usenet and Email SMTP were distributed systems enabling peer-to-peer communications without a single dominant node. Technologically, the internet was an innovative peer-to-peer infrastructure, a "network of intranets". However, ultimately its…
telegra.ph
That said, a lack of lower level P2P will eventually leak fundamental limitations to user requirements concerning freedom.
And Spotify was made by uTorrent guys. (And OpenTTD, randomly..)
Both of which are no longer p2p!
you don't say...
Yep, for very different reasons, but both tied to a fundamental arch choice: connection-oriented >> user privacy & control.
I know the Skype story was “oh shit, protocol updates are a nightmare”
For Skype: client-server connections provides much nicer facility for centralized government surveillance. Metadata alone is useful for LE.
For Spotify: You don't want the users to have the music. You want to watch their activity, and also be on MPAA's good side.
That’s not what I heard
re: the reason, or about metadata?
I heard the primary motivator was inability to push breaking changes. If you're LE, wouldn't you just snoop VOIP traffic anyway?
requires warrant. metadata is better (and oftentimes enough to get the job done), but harder to obtain in p2p system.
i don't see how it's different from looking at IP metadata
This is a great feature but not exclusive to p2p
We've been thru this: what if I change devices? What if I have multiple devices? What if I want to search for Dan Abramov's tweets from 2011
Has anyone honestly ever modeled how much data are in a typical twitter user's feed, for, say, a year, following around 500 people?
It gives me the shudders imagining that all getting sucked onto my device - I want no part of that.
You can easily have an expiration policy set for ephemeral content. But it's not very large; you should be able to do the math.
You are conflating "use of a server" with "centralization". You can run or rent a server that syncs between multiple devices.
If you change devices, you put on your big boy pants and copy a directory or use the handy graphical Dat Desktop tool.
If you want to search someone's posts, you query your neighbors or archival svcs for his full post history. Or you use a search service
Maybe he wrote posts that are not publicly visible to the search crawlers, so you have to ask your friends for it or proxy a search to them
It's really no different than email. Email, plus DHT/content-based addressing, plus a richer document model that allows self-contained apps.
That's an objectively inferior user ex compared to status quo. It may offer better privacy but we need to be more honest abt the tradeoffs
How can you say that when I haven't described the user experience? I'm describing data and architectural model by analogy.
You're telling me to put on my big boy pants and do work that Twitter currently does *for me*
Do I have to do that manually, or will the p2p network do it for me transparently?
Keep a small blog site up in the face of 660Gbps DDoS: krebsonsecurity.com/2016/09/krebso…
Which was my original point: users long ago decided they were willing to trade privacy for convenience. I don't see that changing.
Original article argued the opposite which is why I said it was garbage.
That was then. This is now.
Tell that to the Germans. You're in America. Many users have no idea how data privacy and the cloud works at all.
In fact many user's intuition of Facebook app is that they're communicating to their friends directly with radio waves. (P2P)
Bruh—Americans are pretty fucking stupid - I mean we elected Donald Trump ;)
Still tho, it's kinda obv Facebook is creeping on you when 1/
the thing you wee just looking at on some website suddenly appears in Facebook ads 5 minutes later. People get it - they just don't care.
Yeah and it's that attitude that got Trump. Hopefully you're not defending that attitude. Hopefully you're not defending fossil fuels.
I don't defend fossil fuels, or plenty else. But I don't have prob w/ Facebook aggregating my data & showing ads. Plz don't conflate the 2
They don't not care. They have no alternatives. So more and more I see friends "taking a break" from FB.
They don't want to commit social suicide, but don't want to stay plugged in to the attention mill.
Right, sure, they "care" just less than they care about being connected to their friends. I suspect they also care more than they do about
convenience and ease of use, but we've yet to see an almost-as-convenient FB/Twitter surface to test that. (Anyone remember Diaspora?)
Younger crowd is/has moved to less textual "feed" social nets like instagram, snapchat etc. Others use WhatsApp, Telegram, etc.
Sure but aren't ALL of those centralized? I'm not arguing for Facebook specifically (I do VERY little on there anymore)—just centralization
Your contention was that ppl don't mind FB ads & intrusiveness. I'm saying that ppl are voting with their feet to other services.
Lol this is inaccurate, you should know. In many cases interaction with the app will be faster because of local db.
In many cases easier to use and onboard because no upfront login and password workflow required.
In many case much more featureful because of open source you could customize anything or have diverse plugins.
Plenty of centralized apps support optimistic updates
Not for everything. The ultimate test of the service is when fully offline, e.g. airplane
Google docs
So explain to me how a decentralized version of Google docs would be slower than Google docs.
I gotta scuttle all my collaborators edits. Hope they propagated before I make a conflicting edit!
Conflicts seem like a HUGE problem with P2P, right?
last paragraph in this section
digidem/osm-p2p-db
osm-p2p-db - Peer-to-peer database for OpenStreetMap data
github.com
What's faster? Alice uploads to Google, Bob downloads from Google. Or: Alice's upload is Bob's download.
In the real world we probably aren’t on the same network so going to google faster than the nat punch
And here's the crux of the problem: NAT. It's time for multiple transport channels. Bluetooth Mesh, WiFi direct, regional are networks, etc.
this is technically correct but misses many use cases where the network can be established before you ask for content out of it.
The music festival use case, right?
There's much bigger cases. We're overly focused on the "first time" you visit a website, what about the second?
I actually have no idea what’s going on anymore lol
Can we establish the network and maintain it with Service Workers from that first visit? Can we give you content before you ask for it?
I think you’re talking about a particular system and I’m missing it. Can you share more info
Here's one, static website. On first visit spins up a Service Worker and gets on a p2p network for that site. 1/2
New version of the site gets pushed, one of the peers in the network notices it and distributes the content to the rest of the network 2/2
Now you're faster than any CDN, you literally have all the updated content before the user loads that page again.
This is like what Netflix does with pops, but automatic because we built that awareness into the protocol. Right?
Similar yes, Netflix is aggressive about caching. It definitely requires some method of content addressability to work correctly.
Yeah, I’m pretty onboard with p2p for CDN use case. It’s the “distributed apps” thing that makes me cringe
The hard part is distributed authority and trust models. CDN use cases are easy because you still get a central authority.
TBH, I think there are classes of *new* applications we'll see from these technologies rather than replacements for existing systems.
What about encryption? All these posts from other users are individually encrypted, right? Is decrypting them all a perf problem?
If they are public messages they are signed, not fully encrypted.
Others in this thread indicated a core feature would be that access limited to people you're connected to. Perhaps many diff ideas on this
We're a little too conditioned to the limitations of centralized services. Like, we're chatting right now but I can't send you a large file.
I'd say we're too conditioned to the whole modern web stack, it's hard to see how different our user requirements would be in a p2p future.
You have it backwards. p2p vs centralized doesn't dictate user requirements. User requirements dictate whether we *use* p2p vs centralized.
The infrastructure influences users, it creates intuitions and expectations. It's not like you were born wanting to *sign up* for a web app.
People intuitively want things to work, fast, conveniently with little to no effort on their part. That definitely influences architecture.
it's like there's been this endless tension between centralization and decentralization for all of human history
Sending large files is still an unbelievable pain in the ass, and not easily integrated into the communication apps we already use everyday.
It matters the kinds of apps we are looking at. We already have exemplars of working, non-centralized, p2p social comms systems: SMTP, NNTP
They're old, but they work (except for lack of identity leading to spam), and they don't have any "distributed database" issue.
Availability?
Availability is either moot or impossible. If you have the data, it can't go away. If you don't, nothing can guarantee you can get it. :-/
The app continues to function on the data you do have. This then turns into a consistency prob: guarantee of freshness requires connection.
SMTP is quite available in practice but that's due to incredibly specialized and reliable federated authories.
Yeah thought we were distinguishing federation and truly distributed
I'm starting to find the separation inadequate in this new blockchain future where networks are fully distributed but highly specialized.
The reality of trying to participate in Bitcoin vs. Federated Authority networks is roughly the same, or possibly worse w/ Bitcoin.
Agreed. I'm not saying SMTP is end-all/be-all. Just providing example of de-cent comms with stable threads/msgs, w/o massive central DB.
those are federated authorities though, decentralized authority is whole other hill to climb :)
now, that's static content, so it's easy to reason about, but you can expand these ideas for other types of site data.
I can actually see Google Docs as a great example of something that can work p2p just as well.
I'm dying to get some kind of plugin for LibreOffice that does versioned replication like in a merkle DAG
I'm not sure LibreOffice is the right foundation for that...
Is there anything else that is open source and beats feature coverage compared to Google Docs?
Nope. I think we will have to start from scratch...
Perhaps something based on Draft.js that is super simple and adds more features over time.
The decentralized version can be faster if it's popular. E.g. torrents are often faster than direct downloads.
I wanted to give a diversity of examples. On consumer side, Dropbox, Spotify, Starcraft 2 and every fb feature has this problem.
I’d also love to see a device for in person 🔑 exchange. The handshake device
Yes, there must be auxiliary services that crawl, aggregate, etc. - but they build on *top* of a privacy-respecting human comms net.
These "global view" features are easy to impl on centralized social platforms, b/c they started as client-server web sites.
A P2P data web starts with privacy and transport-independence as the main things it's solving for; builds everything else on top.
Don’t we just end up in the same situation as today in that case?
No. Users&apps can control how much they share, & charge such services a satoshi for each query & for timeliness.
A market or economy can emerge b/c there is scarcity around timely, granular access to user activity. People can opt out For Reals.
Right now we all make devil's bargain - use FB for free! I don't even have option to pay them my ARPU to opt out of all datasets, ML, ads.
Furthermore, this infrastructure can be deployed in authoritarian locales, for journalistic freedom and as a secure comms net for dissidents
Google couldn't beat China's draconian policies and left. FB can't b/c WeChat. We can create unstoppable p2p tech w/ no throat to throttle.
This *does* mean needing to go down to PHY mesh nets on RaspPi and even low-bitrate shortwave radio for key exchange etc. /cc @andrestaltz
But all this stuff is totally doable now, and needs to be done.
I want to dig in here because I don’t think this is meaningfully different than a centralized system. But need a sec
Sure. Also twitter kind of sucks for this. If you want to post a link to gist instead, go for it
On phone in a foreign country. Train in 2h will hopefully have WiFi :)
The p2p, decentralization types often have trouble coming to terms w/ fact that users value features that're hard/impossible in a p2p system
By like token, users (and their data) are taken advantage of in unavoidable ways in a centralized system.
P2P proponents are asking the Q: what if we pick a different set of fundamental tradeoffs. We are not in denial that there are tradeoffs.
Totally cool. I'm honestly just curious who would pay for the infrastructure? Dev time? etc.
So computing is spread across all users' machines? Even accepting that users'd accept that, still, how does the development get funded?
Assume you get the crowd funding, grants, etc. Who decides which devs get hired to work on which features?
Point being, it's hard to see anything scalable getting created that lacks some sort of centralized authority.
How about email? It's the largest social network.
Centralization only factors into DNS for email. ISPs, companies pay costs of running servers & bandwidth.
No. To have an inbox you can navigate to, do useful things with, you need someone to code that UX up and make it available to you.
You underestimate the amortization of dev cost in an OSS world. Most of cost of tech big 4 is in operating a centralized web systems.
Building useful UI for some of these systems is very straightforward, and is easily done by OSS volunteer effort.
I imagine a whole lot of Twitter and Facebook devs would lol at that argument.
Let them. Centralization magnifies/exponentiate the complexity of what they have to solve. The problem they chose to solve *is* hard.
My point is that we don't *have* to solve the "social web" problem *that* way.
Which is provided by some sort of centralized authority: Google, Microsoft, etc. They manage features, UX, etc. QED.
I'm not solving for "zero capital" schemes. Money & businesses can definitely help this ecosystem. Bandwidth still needs to be paid for.
Read again plz. I'm assuming capital - I'm wondering who *manages* the capital to drive this platform forward,
It's a space for innovation again. It's an open protocol, not an open platform. Significant difference.
Value of social networks = volume of people in them. *Everyone's* on Twitter. "lots of different ecosystem projects" would lose that value
This is value of large, centralized social networks. *Everything* available in one place. Users can choose what they want to carve out & see
Actually, no. Very little avail in one place. FB & Twitter have a dearth of features. They gate innovation compared to open data platform.
By everything I meant people. Users can choose whom to follow. They don't need to poke around N disparate, one-off communities.
Ah, sure. But that's because internet lacks robust Identity system. This is a *core* thing that the p2p de-cent web is trying to solve.
This "open social graph" is a core goal. But prev efforts failed b/c they were not ambitious enough; open graph is a means to an end.
I find decentralized identity terrifying and subject to massive abuse. Who says, definitively, that "Peter Wang" is in fact Peter Wang?
Web of Trust & personal keys/certs, through physical or other offline exchange. To your example, how many Peter Wangs exist in the world?
Do we all belong to the same central sovereign authority? Then how in the world does the world function at all?
De-cent identity of *people* who are already in a meatspace social net is a simpler problem than de-cent trust of random boxen on Internet
Few, but, what if you piss off some horrible person; how do you stop him & his many asshole friends from impersonating you w/ dummy account?
I've no doubt. I'm sure the few early adopters are brilliant people sharing brilliant things.
If you want it public and searchable, put it on the old web. Most people don't want that all the time, though, and many want it never.
And it is often completely inappropriate for web apps. When's the last time you searched, got a Google Doc in results, and clicked it?
The son isn't actually gonna kill the father; services will still exist. But I want the p2p web to kill public/private confusion.
That will lead (back) to multiple, fractured publics, and that has downsides. But at least tools to deal w/downsides will belong to *users.*
This is true. P2P arch will influence some aspects of UX. But also clustered arch will influence some aspects of UX, e.g. login & offline.
Each arch decision has UX pros and UX cons. That said, the UX goal shouldn't be primary. UX has gone too far: it's creating addiction.
P2P services will not and should not win adoption based solely on UX. It's fundamentally about freedom first.
And freedom (offline, all data on my device, no login) is, in a way, good UX.
Keep up the good fight Paul!!
What happens if the encrypt algorithm is compromised? You can migrate to a new one, but anybody can still access the old data and decrypt it
or am I missing something?
Yeah that's true. But the same counter-argument exists for hosted services. "What happens if they get hacked?"
And services get hacked way more often than e2e algorithms get compromised
If you have a really high need for security, then you should avoid putting the data (encrypted or no) on 3rd party hosts
I'm not thinking about "high need", just... a Facebook clone? What happens if pics of some user leaks? What if he/she's famous?
A totally valid concern, but e2e isn't even employed on the current FB, so this is kind of a weak argument against p2p
In fact, you can use p2p to provide direct ACLs. E2E for my target recipient, and then *only connect* to that recipient
No cloud at all
I lost you on ACLs sorry 😅
Sorry sorry - "Access Control Lists", aka permissions
Ok but, still, FB has on-server encryption of the data, with SSL for the transport.
If the encryption is compromised they can patch it
But I'm probably missing your point here
Yeah but they have to:
1. access the server
2. get the data out
3. un-encrypt it
(thinking about best case - for service provider)
Right. If you need e2e plus access control via a host, the latter can be stacked on top of p2p
"this wont work with today's tech", mostly translates into "this doesn't fit into the model that i'm familiar with"
Hopefully their love is an orthogonal concern! If users are empowered to self-build and can create high enough quality, that's the game
What does this do to sites like youtube, where the media that it stores takes up an uncountable amount of storage space?
When we hit the limits of user devices, then we'll find ways to involve beefier computers and just stream p2p. Same underlying model
I'd love to see a @datomic_team style architecture backed by @IPFSbot or similar as storage. Seems really powerful.
Have you seen cockroachdb?
Only very briefly. Doesn't seem that similar? SQL, Raft consensus, no 1st class history, place-oriented, but interesting consistency story
Can't really see how it would benefit from p2p distribution
it's the interface people will want to build their distributed apps i think.
i.e. option to have single-shard and multi-shard strong consistency
Yea, if you favour C over A, it looks great. But looks distinctly designed for relatively small stable clusters in always on DCs
I would love some people to chime in on ZeroNet... Seems more capable than @dat_project but seems mostly ignored.
That's a mischaracterization of how many decentralized application stacks work
Yeah, I'd much prefer to give it to Zuckerberg, who's far more trustworthy. </sarcasm>
this is objectively true
Sure it is. "They trust me, dumb fucks" Zuck
Sure, it is.
i'd prefer to share w/ a known quantity than an unknown one. wouldn't you? also, i was making a technical argument - not a political one
Actually technically your argument was flawed. These protocols are built with people distrust in mind, where trust is on numbers/crypto.
It was probably the dumbest idea ever, for society, to build massive tech systems on top of trusting some particular for-profit organization
Yeah, I actually value security more than privacy. If the security is lacking, you will end up having no privacy.
That's objectively ("security over privacy") the speech that authoritarian regimes have used to convince people.
i wish y'all would channel your activism into something that can work well and that we actually need -- decentralized DNS
It exists already, on top of these protocols.
ansuz/dnssb
dnssb - ssb meets dns
github.com
What's the relationship between DAT and other W3C suggestions
and namecoin. but these are so hard for me to use with my web browser. why? i think it's because that part of the problem is tedious
Anyway, do a little bit of research before saying things are garbage. For your own sake you don't want to say wrong things on stage.
there's no need to be rude about this. i have done my research. there are real problems that are ignored because of political overidealism
Easy. Don’t make personal attacks, direct or implied. I’ve written plenty of garbage too.
You're a funny chap!
"Implied" personal attacks are bound to culture. You *interpreted* a personal attack from *your* cultural lenses, which I don't share.
Explicitly, I recommended researching facts, that's all.
This conversation went in the shitter so I’m unplugging. Tremendous respect for what you guys are working on.
Also think it’s important and timely given what private companies are capable of these days and what we’ve seen this year
With that said, there are serious technical issues that everyone pretends don’t exist or can be designed around and it’s a huge bummer
Please consider that secrecy is a real problem in p2p today, as are problem spaces that can’t be represented as a crdt (ie wiki), & agility
SSB has encrypted messages, and even non-encrypted messages still have a good amount of secrecy as long as you control how to share data.
One final plea is please give me the benefit of the doubt; I’ve listened to your podcast, built a dht, built spam system, etc
Doubt is good, it brings honest discussions. Labelling as garbage isn't good. Claiming we pretend problems don't exist isn't good.
Such as...?
I don't care if there are serious technical issues or not. You were rude and you got a response in the appropriate tone. fin.
btw if an article doesnt cover every technical problem, it doesn't mean it is misleading or pretending they're not there or "garbage"
but this is irrelevant, how fast this conversation went in the shitter is just a reflex of how you started, to put it simply.
Sure caused a great discussion though eh?
If sounding angry and inflammatory to gather more replies is your favourite method to generate discussion, then yes, sure.
You are right that I shouldn’t have tone policed @andrestaltz. Somewhat hypocritical. Sorry. Wanted to sleep :)
For sure, I guess the difference is I'm choosing to go with Facebook or Google for my data as opposed to be forced into security.
And current stockholder too!
Also: isn't gab, like, the ideal use case for a fully distributed architecture? why didn't they go that route?
That's why we invented encryption :P
it's more complicated than that :) see rest of thread
Depends on the guarantees that you want but this is likely true in 99.99% of cases
Blockchain is expensive on purpose. While it's the most hyped up p2p tech, it's certainly not all there is.
even in Blockchain, many don’t use high CPU proofs, there’s an entire authenticated blockchain space that isn’t proof of work.
that said, this article is terrible, and premise is wrong (Cloud CPU is approaching zero cost, storage is costly but more complicated p2p)
The title is inflammatory sure, but I feel many of the points, especially around centralized services being big targets are valid.
In my ideal world, we use both P2P and cloud services. Each has unique strengths and weaknesses.
This I completely agree with :)
We overly centralize right now.
The main technical blocker I see to decentralization is home ISPs.
Slow uplink, no stable public IPs, MITM by ISPs, hostile TOS, etc...
You simply can't compete with cloud if you want a stable public IP and fast bandwidth.
Meh, I don't see this as a big blocker. WebRTC accounts for these limitations already.
WebRTC can't connect without public services helping. Also the bandwidth issue is real with rural DSL typically getting 0.3Mbit up.
Sure, but in the aggregate you can make up for a few slow upstream nodes.
for popular content yes. Bittorrent proved this. Webtorrent and DAT are also interesting for this.
I think a much larger problem is figuring out how to establish a network before you ask for content. That's much harder.
do you mean computer network or social network?
Anyone can build a small twitter clone, but if nobody is on it, what is it worth?
computer network. establishing the connections to piece together a network takes time, that perf hit is a big barrier to some use cases.
I'm less concerned about this. Maybe I'm not seeing the real problems.
Cheap storage and edge caching are probably the strongest points towards more decentralized approaches.
it's not like you can't attack p2p networks, there aren't as many so they aren't as ripe a target.
the way you attack a centralized service is quite different than how you attack a p2p network, sure, but neither is immune to attack.
Need distributed global consensus (like btc)? Yeah, gonna require a lot of compute. Not true for distributed storage and transport
Potentially? Many peers duplicate content, but that's required for a p2p network to scale. That doesn't bother me. The most important win...
...in a distributed network is that users get to own and manage their data instead of FB or GOOG
Yeah I have no comment on that. I can and have made that argument, but I'm not keen to do it now or in 140 chars 🍻
I agree with the cost thing though. it can become really expensive with scaling . the rest is well . . . you said it
Yeah... intended to spur discussion. But it looks to me like there's an entire generation of university students who believe stuff like this
2012: the cloud sucks, why would I run code on machines I don't own?
2017: the cloud is over, I trust everyone with my code.
If you have a better idea for how we're going to land a big account at our consulting company on the cheap ID LIKE TO HEAR IT!
In my mind I just hear "data centers are OVER"
Portlandia - OVER
The return of the cyclist character. Clip from Portlandia, first season, third episode.
youtube.com
Beautiful
You don't think distributed is better than centralized? Look at Git vs SVN
There’s no secrecy in git. And no availability requirements
Git is 100% available till my laptop breaks down
Because noone is there to moderate content on the actual web? Wouldn't it be simpler if we all used the same platform to publish?
Not exactly what this article is getting at but I've heard good arguments re the return of "edge computing". Good example is Apple FaceID
This is pretty good a16z.com/2016/12/16/the…
Autonomous driving is a classic example. But I'm more impressed by @BillRuh_GE's GE Digital pitch of jet engine/power-plant optimizations!
Do you have a link for this?
See e.g. ge.com/digital/blog/w… @GE_Digital
What is Edge Computing? | GE Digital
Edge computing is not a new concept, but several trends have come together to create an opportunity to help industrial organizations turn massive amounts of machine-based data into actionable intelligence closer to the source of the data. Learn more.
ge.com