A senior Twitter engineer told me in private this is actually true in a lot of cases. Shadowbanning is often just a temporary infrastructure issue. Twitter is massively reliant on distributed indexes which can fail and require rebuilding. Not sure I'm using the right words.

A very important cognitive bias in humans that served us well in 99% of our history was assuming in simple, purposeful explanation for results. That has broken down in a specialized society relying on complex systems. We assume all failure modes in other industries are simple.

In this case, we assume a Twitter infrastructure failure is a purposeful, vengeful act because we both (1) Do not understand how a complex system can fail (2) Do not like believing in bad things happening to us for random reasons nobody is purposely at fault for

Additionally, when shadowbanning does happen, it's, as of the last time I read about it, an automated spam response triggered by an algorithm. Spam systems are notoriously obscure magic. Twitter staff saying THEY don't shadowban people can strictly correct in the right context.

Remember when Wikileaks made a bunch of hoopla about Facebook blocking their links? You really think a human did that? No, I talked to an engineer there, it was an automated security threat system. Side note, Wikileaks actually hosts malicious emails in some of their dumps.

We are all pattern-matching creatures who assume the worst living in a world that iterates at a pace that makes it physically impossible for any one person to understand more than a small percentage of it. Welcome to how 2018 happened.

Furries are a manifestation of our overactive anthropomorphic pattern-matching

Twitter engineer from 2010-2014 who worked on rearchitecting the core data services, along with other former engineers in the thread twitter.com/olix0r/status/…

Got a second affirmative confirmation that this thread is correct, if you think you’re shadowbanned, some human didn’t do that to you, it’s a system issue, or you look like a robot because you’re in a mindless bleating retweet cult.

RE: Facebook blocking Guardian story about Facebook being hacked. Worldwide instant-pickup URL referencing “Facebook” and “hacking” being blocked is easily explained by machine learning to stop “mercenary hacker phishing” spam, with improper goodlisting. twitter.com/swiftonsecurit…

I literally run spam filters for my job. I swear to god they’re dumb as fuck.

also, the url has a proper Grauniad tyop in it.. last word is 'berach' not 'breach' ..

Ha ha I know that -- I got banned on Facebook for sharing the cover of the magazine where Roseanne was dressed like Hitler (friends had not seen it was not sharing in support) and got banned for "hate speech" for 3 days!

What job? Singer?

I know for a fact that twitter has problems. I pull my timeline from their API and periodically do a full re-take of my TL. I inevitably get tweets that somehow weren't delivered the first time. Deliver of "likes" is even less reliable. /1

I'm guessing there's a lot of "best effort" delivery in their systems and that best isn't as good as it should be. (I used to do spam filters in the 90's. They're a bitch to get right, but I managed. Dunno if my techniques would scale to today's net, though.) /end

Don't you mean your second job, Taylor?

Sure. However, the fact that such a poorly-written system decides whether important news can be spread on what is now probably the #1 news-gathering platform for most people, is problematic in and of itself.

It’s worth pointing out that it’s also the main reason there’s no edit button: crazy amounts of immutability-assuming infrastructure.

or you replied in Cyrillic to a tweet by celebrity. That gets you instant ban, and you have to verify your account again. Definitely not humans doing that, but a very ham-fisted approach to “Kremlin bots”

Bleating

Re-read this tweet for affirmative confirmation (wait, don't affirmative and confirmation mean the same thing) that you don't know English

An account's number of tweets can be multiple inconsistent numbers. The number on your profile, number of tweets if you just scroll down to the beginning, number in the data download, and number when you search are often all different (even if you include/exclude RTs).

Haven't dug down into which tweets are missing in which counts yet, but it seems to affect accounts over longer timeframes more. Answer is probably distributed caches similar to shadowbans.

Trump’s probably wishing his old tweets were unfindable about now… theindychannel.com/news/national/… @nytimes @theindychannel

who??

Likes are worse. Twitter only actively tracks a very limited number of likes. I constantly get deja vu because I think to myself: didn't you like that a little while ago? Admittedly, as @SwiftOnSecurity and @AwfulyPrideful et al. can attest to, I kind of like a lot of tweets, if

usually for the mundane reason that I want to express my sincere gratitude for something nice/interesting etc. and hope that it encourages people. Those who don't find o... Those who don't incorrectly assume I'm a bot, anyway.

I always thought my replies disappearing in threads with a large number or replies was a bug. I had no idea that is also what people thought was shadowbanning.

@MotherResister Oh that’s a different newer algorithm and thread design :/

I think this thread is amazing as presented to my timeline by Twitter.

yes but this #subthread is even better #BTW here is proof #Shadowbanning is a thing @jack why did @Twitter i was put in 12hour #TwitterJail c #BS 1stAmendment Foul docs.google.com/document/d/1w9… & @TwitterDev infrastructure needs 2b better @ #EventualConsistency mobile.twitter.com/alicemazzy/sta…

So this explains why I keep getting notis that arent valid huh? Smh

Twitter doesn't do themselves any favors here by deliberayely giving the same error message for blocks/suspensions/deletions as they do for network errors.

Is this sonehow inherrent with cloud applications ?

Threads like these are precisely why I follow you.

When you bring it back to furries at the end of every thread, it feels like getting Bel Aired

I love how everything comes down to Furries. 🦊

Like OCD on furroids.

Good points, but there are other times when the shadowbans are purposeful. While we have to recognize that it might be the tech, it could also be malicious and we can’t discredit that off the bat either.

The ability to purposefully shadowban someone may have been anti-spam featureset that was later given a human interface, yes, I don't know.

Short and sweet - there it is¡ Brilliantly simple explanation.

😜twitter.com/FemiAbodunde/s…

Goethe, 1774 “And I have again observed, my dear friend, in this trifling affair, that misunderstandings and neglect occasion more mischief in the world than even malice and wickedness. At all events, the two latter are of less frequent occurrence.”

Hanlon’s razor is also common, but with complex distributed systems it’s not usually incompetence or stupidity, as Hanlon states. Parts of the internet are transiently unavailable all the time. Same for anything that rides on top of that network.

Another case in point was the burp in IP reputation in 2013 which led to Facebook blocking Tor; people jumped on bogus hypotheses like "FB hates anonymity because adverts" or "FB hates activists", not "tech screwup".

Aside: it was this event, and @runasand's / Tor's help in defusing the bullshit, which directly led to the onionifying of Facebook, the .onion TLD, certs, etc. It's amazing what a little understanding will achieve, if one tries. Certainly better than @ioerror's random hatred.

I will remind you of this often, my friend. 💕 “It's amazing what a little understanding will achieve, if one tries.” - @AlecMuffett, May 19, 2018

I'm sure you will; of course, so does obstinate pursuit of a grand idea. :-)

2018 is just an overflow error from 2017

none of these things would be serious problems if these platforms could be expected to have timely human responses to algorithmic errors

heck, IRC networks have pages to report erroneous automated K-lines

when you make algorithmic failure look identical to a human saying "fuck you", you don't get to complain when people assume the latter

fair

how stupid of people to think ever cooperative facebook would actively suppress information coming from one of the few real

information distributes, in the interests of the political group that has very succesfully coopted it.

and furthermore wikileaks dumps have harmful emails in them so pretty much distributing malware,zuckerburg can fuk my holes

Does Hanlon’s apply? If I leave someone a voicemail and they don’t call me back, did that person shadowban me? Possibly. Also possible they forgot, put it down for later, etc. Links etc disappearing could be sinister or it could be other. Why not assume other?

you know as well as anyone that ML is a cop-out to avoid taking responsibility for your own failure to moderate well

the designers of the algorithm are responsible for its errors if they don't understand how it works either, they're responsible for that, like it or not

It does happen relating to specific tweets being said, and more likely in accounts with a certain viewpoint. Equally, how many blue tick accounts have suffered this fate?

All fascinating but I want corn facts.

(3) think that twitter people (and other IT people) just are too overconfident with their abilities to solve these problems quickly or even think about these problems in advance.

We believe our failures are bad luck and others’ are a lack of skill

I mean, to be entirely fair, it's also 100% in-character for twitter to fuck people over for no reason and then fail to communicate about it

Your second reason is especially true for IT.. I mean, "it's a machine so it can't possibly fail randomly.."

Over the last 25 years, I've seen so many ways of random, illogical, unbelievable, mind-boggling machine failure. And it's getting worse year by year with machines getting cheaper, networks getting faster, software getting bloateder and systems getting complexerer.

The “eventual” in “eventual consistency” means that for a while, users do not always get consistent responses to requests.

Company I used to work for hosted site for org which had a massive sales spike one day a year. Pre-cloud, so Infra was sized for regular traffic. Meaning on this day, site became difficult to use. Users had all sorts of conspiracy theories. Reality was mundane tech limits.

So they sold those little country flags and the spike was the day before the Last Night of the Proms? Or did they make Guy Fawkes masks and the spike was the day before Bonfire Night?

Something similar happens 2 or 3 times a year with the Steam PC gaming platform. When Winter or Summer sale happens, the store tends to become unusable for the first hour or so because millions of people want to check it out at the same time.

were their initials “I.R.S.”?

Nope ... it was selling something

Yup. Don't work at Twitter, but do work on large distributed stuff. Every conspiracy theory is born out of, "I think I am important, and I don't understand why this is hard." In general, nobody wants to thwart you, and stuff you don't understand is often very hard.

In addition, I seem to be seeing the same tweets over and over in my TL. I assume each tweet has a unique tag so why show it to me more than once when there are many tweets I am not seeing from the few thousand accounts I follow?

We are all animists at heart.

Except people were getting notifications that their account was locked immediately after tweeting certain phrases. I figured it was a breakdown of a system, just an overly aggressive anti-bot algorithm than a more innocent database or indexing error.

Other "shadow ban" issues without notifications or a clear trigger are absolutely likely to be an infrastructure problem, though

This is also true for complex organizations. As my much larger company acquires very small ones, the newly acquired see every bad thing as a deliberate decision by some anonymous power, and I have to explain, no... we are just imperfect and we messed up.

why are u nerdy losers shilling so hard for a bunch of dumb ass risk averse algorithms that reinforce this hell?

A past employer got taken over by a larger Texas based company. After a few weeks of listening in on their morning call and its list of horrors, we were told we would join their domain. Was proud that our head of IT refused point blank.

That’s less impressive than stepping in to help. Acquisitions resisting and wearing it as a badge of honor is deeply dysfunctional - a rising tide lifts all boats.

Actually, the outcome was that we designed a new Forest from scratch and migrated the whole company to it. By refusing to accept being subsumed into something worse than what we had, we got something better for everyone.

Do you know how to contact a real person at Twitter? I have an index rebuild issue (that's precisely the problem) and using dms for help or tweeting @TwitterSupport isn't working. For 2 months, my analytics stats are zeroes on retweets, likes, replies.

Have you tried reaching out in the dev forums or someone on the api team? Maybe @andypiper

So, block chain would fix it all!

I'm not a database engineer but I am database adjacent. I could definitely see how distributed index/distributed cache issues could look like shadowbans.

Oh sure. Now I remember CAP theorem. Thanks a lot, brain. en.m.wikipedia.org/wiki/CAP_theor…

I read this as database accident and i was nodding all me too oh me too

"I am database adjacent"

i built postgres text search for a small healthcare company and it's also REALLY easy to shadowban people with poorly tested weights/scoring and relevance thresholds. it's so easy to just... not return important resources in search

@imogennnmnnnnnn perhaps microtargeted individuals comprising majority would have attempts to research, document and publish delayed, frustrated and confused their entire lives, never knowing bots/browser/search engines were automatically preventing them from success, even in mundane pursuits.

this is absolutely true and in many cases actively harmful

What else did people tell you in private?

Pokemon balls are reverse tesseracts. When you throw a Pokeball, you're catching yourself.

Aaah the silph road sounded familliar

Also like to add didn't Pokemon movie play under miramax. And why is the 8th gym leader a mafia boss named giovanni?

I thought everyone knew this.

Holy FUCK

Can you please tell me more while I scrape my mind off the walls

(No comment)

That explains why some tweets I found on my phone disappeared sometimes when I tried looking them up later on on PC

I used to see a lot of "event replay" crap from Facebook (2010-2011) that was easily attributed to failure recovery. I still see this kind of thing with Twitter sometimes; eg a "someone liked tweet" message re-fires when I already saw it hours ago.

Oh so that's what's going on...!

About 2/3 of my feed on the web or iPhone app is "someone liked" or "someone retweeted" from yesterday, usually things I've already seen. I have to use Tweetdeck because it shows everything chronologically.

Happens on Slack too. Would still (so far) choose it over Skype for Business and Teams.

Idk. @housemaidfear when they still around was shadowbanned for 3 months. There is no war in Bah Singh Sae.

this is absolutely correct

Somewhere, kestrel is still backed up

I felt a tremor in the force when you said that. Flickers of @jpd waking me up in the middle of the night.

That one time I had to set an alarm to wake up every hour and restart kestrel felt that way.

This is absolutely wrong. @mrdonut confirmed there was a bug that looked like shadowbans, which @TwitterSupport *chose* to ignore for years.

I don't think this helps making them look/feel any less insidious. It just feels like a layer of plausible deniability on top.