What is twister?
twister is a microblogging peer-to-peer platform, that is, it is a distributed system like bittorrent or similar file sharing technologies. Being completely decentralized means that no one is able to shut it down, as there is no single point to attack. The system is also designed so it cannot be censored, freedom of speech cannot be taken from you. And because the cryptography is employed end-to-end, no entity is able to spy on your communications.
Is it open/free?
Yes. The protocol is open and community is invited to help to extend it with new features. The reference implementation is free software, based on Bitcoin and libtorrent sources, which are released under the terms of the MIT and BSD licenses, respectively.
How does it work?
For the complete description you should refer to the white paper. But in short: twister is comprised of three mostly independent overlay networks. The first provides distributed user registration and authentication and is based on the Bitcoin protocol. The second one is a Distributed Hash Table (DHT) overlay network providing key/value storage for user resources and tracker location for the third network. The last network is a collection of possibly disjoint “swarms” of followers, based on the Bittorrent protocol, which can be used for efficient near-instant notification delivery to many users.
Does it scale?
I hope so, but only time will tell. DHT network should distribute resources evenly to be stored by every node, including the posts themselves and profile information. Those resources are produced on a relatively slow volume, in average (add the total number of posts produced every day and then divide by the number of users). Registration database is duplicated on every node, with about a hundred bytes or so per user.
The registration database may actually grow larger if we have a million of users, but still nothing comparable to the size of Bitcoin transaction database. Some strategies are proposed to allow very low lightweight clients which wouldn’t need to store the entire registration database.
What does it do? (twister features)
The first prototype (or proof-of-concept implementation) is meant to replicate the basic feature set of a microblogging platform. That means: finding users, browsing profiles, follow/unfollow, send text posts limited to 140 characters, retransmiting and replying posts, navigate through post threads, mentions, hashtags and direct (private) messages. Private messages requires the recipient to be follower of the sender, which is a common requirement in other platforms as well.
Some other features may be difficult to implement in a completely decentralized system and may require more thought. This includes arbitrary search of words in all posts and collecting the hashtags to find out the top trends.
How is Bitcoin used here?
Bitcoin, in the sense of the digital currency, is not used at all. However, the Bitcoin protocol and the implementation of the neat idea of block chain is on the basis of twister. The block chain provides a sort of distributed notary service, certifying who owns a given nickname. The name is associated with a specific key pair, which is used for authentication and cryptography.
Can I mine Bitcoins with twister?
Not exactly. The same mechanism used in Bitcoin for mining is also used in twister but for a different purpose, ensuring the order in which user registrations took place (the nickname belongs to whoever registered it first). twister network must incentive users to mine, so block chain may keep advancing. However, unlike Bitcoin, there is no monetary value involved. The twister incentive is: whoever finds the hash collision to validate a new block of transactions will be awarded with the right to send a promoted message. Promoted messages have a certain probability of being displayed by twister client.
Promoted messages? Am I going to be flooded with SPAM?
No, I hope not. I don’t like promoted message any more than you do, but I believe that a fair balance between the allowed volume of promoted messages will not upset the users while providing a good incentive for people to run the twister infrastructure.
Currently there is a maximum of one promoted message to be shown every 8 hours for every client, but the exact policy to be used is meant to be decided by the community.
The mechanism is actually quite democratic. Anyone can start generating blocks to send promoted messages, so this is effectively an advertising mechanism reaching the entire population of twister users. While an entrepreneur may invest in a mining rig to announce his product, a non-profit organization may ask his supporters to use their own personal computers to increase the probability of spreading their message.
How do you make money out of this?
I don’t. But, on the other hand, my costs are pretty low (except for my own time). Because the resources to store data are distributed among the clients I have only to pay to host this very simple web page here.
Why should I trust you?
You shouldn’t. You may read the paper yourself to reach your own conclusions (feedback is appreciated). You may download the source code, you may inspect it and then you can recompile it to create your own executable. You need to trust no one.
How secure is twister?
twister uses the same elliptic curve parameters of Bitcoin, namely secp256k1. This is not the curve that was supposedly compromised by NSA, which is called sec256r1. A (non-compromised) 256-bit ECC public key should provide comparable security to a 3072-bit RSA public key (at least that is what we are told by the experts).
While I’m not a cryptographer myself, I would reason that nothing would provide more incentive for people to try breaking some encryption than money. There are millions of $USD laying on the table, encrypted with secp256k1 Bitcoin keys. I believe that if someone out there is able to break this technology, we would have already heard about it by now.
The direct message encryption uses ECIS (Elliptic Curve Integrated Encryption Scheme) of the SECG SEC1 standard. The implementation is based on a sample code posted on the internet by Ladar Levison of Lavabit (that email provider that was shut down in 2013 after Ladar refused to cooperate with the US government to permit system-wide monitoring of his clients). The cypher used in ECIS is AES-256 with SHA-512 HMAC.
Why peer-to-peer?
I believe that a microblogging platform based on peer-to-peer was the logical thing to do, an obvious solution urging to be created in order to address the concerns of free speech and privacy. The microblogging concept itself is such a basic communication tool that it makes no sense to handle this power to any single entity.
Is any peer notified when I get online?
The architecture is designed so that other users can’t know if you are online or not, what your IP address is, or which users’ posts you might be reading. This information is recorded nowhere. That is not a hard guarantee though: what I’m trying to say is that I personally have no means of doing it, nor any other normal user. However if one entity is capable of recording the entire internet traffic, he will probably be able to sort out exactly where you are connecting from (your IP address). If this bothers you then you probably already know the solution: use Tor.
What is the threat model for twister?
The basic threat model for twister is that any other peer is assumed to be individually untrustworthy. Therefore this other peer may try to deceive you by providing forged posts from other genuine users or to refuse to store or forward your own posts. In other to prevent these two scenarios, your twister client must (1) always check if each post is properly signed by the sender and (2) propagate your own posts to multiple random clients. Issue (1) also demands security from the underlying cryptography to ensure the authenticity of posts and the inviolability of private messages, as briefly discussed in question ”How secure is twister?” above.
Regarding the assumption that your twister client is able to connect to other truly random peers (2) who would not collude to undermine the privacy of your IP address (online presence), as discussed in question ”Is any peer notified when I get online?”, or even attack and isolate you into a separate network, the current threat model clearly assumes that the attacker is not a government but rather an user just like you. While being a clear limitation of twister, this also allow us to focus on solving the other (also quite complex) aspects of the decentralized microblogging technology, like resource storage and efficient post propagation. The assumption is that we should leave the problem of secure anonymizing technology implementation to the experts of the field, namely the Tor project, and then make twister to operate on top of this.
There is one remaining problem, however, preventing twister from operating on top of Tor and thus allowing a more interesting threat model where the adversary could be a censoring government. While both the registration network (Bitcoin-like) and post storage/propagation network (BitTorrent-like) can easily be run on top of a Tor proxy, the DHT overlay network uses UDP protocol which is not supported by Tor. One possible idea thought is to connect to other twister nodes which have full connectivity by TCP and have them to relay messages to the DHT network.
What is the problem with Twitter?
I like Twitter, I use it to read news and I have no complains or whatsoever. Twitter has a strong reputation of not being cooperative with PRISM wiretapping program of NSA, and of fighting to protect user’s private data on courts. I believe Twitter deserves the public praise and recognition for this.
We don’t know, however, for how long the company will be able to withstand such pressure. The so-called “gag orders” in US may prevent the recipient from ever saying anything about the request they receive, so if there is any piece missing in that relationship Twitter x government the truth is that we might never know.
So, without any specific complaint against Twitter, I’d say it is conceptually unjustifiable why everybody must trust any single company to ensure that a given post really came from the user it claims. There is no reason to trust that this entity will never hand out your data, or that none of his employees would spy on you (recently a Google employee was fired for spying on teen users). If there is a technical way to prevent this, why would we do it differently? Besides, I must admit that the technical challenge of implementing twister is a nice motivating factor by itself.
I have nothing to hide, why do I need privacy?
It is a common misconception that only wrongdoers need strong cryptography to protect their communications. Quoting a report published by the House of Lords, UK, 2009, “Mass surveillance has the potential to erode privacy. As privacy is an essential pre-requisite to the exercise of individual freedom, its erosion weakens the constitutional foundations on which democracy and good governance have traditionally been based in this country.“
It is therefore a consensus that when a government has the ability to convey mass recording of everybody’s communications, and exercises this ability without going through the due process of law, this opens possibility for a totalitarian state. Even citizens who are not wrongdoers may have personal things they feel embarrassed of, like, for example, their sexual option or the fact they may have gone through an alcoholism treatment. One in possession of such private information can use it against their opponents, including political adversaries, by harassing and blackmailing. This is a big strike against democracy.
Can't mass surveillance be stopped by law?
That’s a debatable topic. It is important to realize, however, two fundamental differences of these new digital forms of surveillance:
- Because of the technology, the costs for a government to record everybody’s communication just got ridiculously low.
- “Code is law”, professor Lawrence Lessig brilliantly realized. That means that it doesn’t matter, for example, whether is it legal or not to have your Skype conversations wiretapped: if the code is changed to include an eavesdropping backdoor then it will be so. The law is not the determinant, the code is.
So just read the news and take your own conclusions about it.
I've lost my private key, how do I get my nickname back?
You can’t, sorry.
Please! I would pay you to recover my key!
There is no way. You can’t post messages, update your profile etc if you don’t have your secret key. If I lost mine I would not be able to do anything either. The profile will be kind of “locked” forever and you will need to create a new one (with a different name).
It is highly recommended that you do a copy of your secret key and keep it in a safe place. Because the secret key is just a string of 52 characters you can actually print it, take a picture of the screen or even write it down to a piece of paper. (Sending the secret key by email would not be a wise idea though)
What's the difference between the DHT network and the torrent/swarm?
DHT is primarily used to securely store resources like avatar and profile. The fact that posts are also replicated there is just a convenience so you don’t need to subscribe to one’s torrent/swarm just to see his last posts. Of course, the multi-valued DHT resources also provide a handy, scalable and low-overhead mechanism to implement mentions, hashtags and stuff.
Multi-valued DHT resources have no guarantees though and they are not automatically refreshed by nodes. As the network grew from 3 to 150 nodes in the first days of 2014, some of my first multi-valued resources became inaccessible since the nodes storing them are no longer neighbors of the original DHT target ID. Refreshed keys like profiles and avatars have no such problem.
Anyway DHT is NOT the intended mechanism for updating your timeline. That would be a really bad idea (polling everybody’s DHT last posts every few seconds!). It doesn’t scale.
Swarm is used for posts only. Both public and private (Direct Messages).
Is the block chain used to distribute user's messages?
Of course not. That would have obvious scalability implications.
Why can't I see my followers?
Short answer: nobody can. Long answer: twister’s posts are propagated by bittorrent mechanism so essentially anyone that joins your torrent will be able to anonymously follow your posts. Apart of that, twister has the concept of ”publicly following” where each user will make the list of everybody he follows public. So, in theory, to find the followers one would need to download each other’s following list just to check if their name is in there. Of course, this is not a viable alternative.
The number of followers that is currently shown is obtained from libtorrent as the number of active peers in that torrent swarm. This value, however, was found to be a quite unreliable estimate (supposedly it would have the meaning of ”online followers”). Contributions/improvements here are welcomed.
What about twister for Windows? Mac? iPhone?
The current twister’s proof-of-concept implementation was entirely developed in Linux and I have successfully ported it to Android. Because the UI is just HTML5/Javascript, porting it to other platforms is only a matter of recompiling the daemon. Windows, Mac and iPhone are certainly possible but I have no resources myself to do it.
Another more interesting long-term goal though is to move all cryptography code into browser’s Javascript UI. This way users would be able to access twister from any client platform they use, choosing any third party server they want, while still keeping their private keys safe.
Update: twister now compiles in OSX Mavericks! Check
doc/build-osx.md for instructions.
love it..let me know when you get it going for windows.
re: Can’t mass surveillance be stopped by law?
the law can not prevent what technology permits.
Many questions. I’d like to see a table with a comparison with other services. So, is there a wiki somewhere already? (:
wiki is here: https://github.com/miguelfreitas/twister-core/wiki
but it is currently empty. looking for volunteers
completely busted my battery on the note II.
Installed it, went from day and a half to 10 hours of battery.
Uninstalled it didn’t solve it. Did a factory reset, solved it. Reinstalled, same thing happened. Some process must be left hanging around there.
Not worth using yet on Android
It does use more battery, but “factory reset” is definitely not needed. twister sample client installs no service on boot, so just rebooting your phone will stop it for good.
A new UI option to stop the deamon is planed (see issues on github pages).
Created issue for reducing power usage on Android.
The direct message encryption implementation is based on a sample code which was posted on the internet by Ladar Levison of Lavabit. We all know that Ladar has shut his site down because he refused to cooperate with the US government allowing system-wide monitoring of his clients. We can only reason that if Lavabit encryption were fundamentally flawed the authorities probably wouldn’t need to go through all of this hassle just to break into a single user’s account.
Research ‘at rest’ encryption. That’s what Lavabit did. Not only that, but he was providing unencrypted data to the Feds prior to the SSL Key request. The SSL Key request came after the Feds wanted a real-time wiretap, and Levison requested $2k to implement it (and another $1.5k after 60 days). The REAL reason the feds wanted the SSLKey wasn’t because Lavabit was ‘hard to crack’, but because the data was in plain text, trivially available, and Levison was trying to rape the government for the data.
So who do you trust more with your data? A lying businessman, or the government?
You should be using public/private key encryption. Not some half-assed effort by an amateur.
We do use public/private key encryption. Implementation is here:
https://github.com/miguelfreitas/twister-core/blob/master/src/key.cpp
Look for function
bool Encrypt(std::string const &vchText, ecies_secure_t &cryptex).If you have any security concern about the current implementation please let us know!
Nice project!!
It would be nice if you could make it easy to use this system on top of I2P and Tor.
(that would require you to be able to use crypto hashes instead of ip’s for the transport)
Hey, check out the design work some of us did on a Google Hackathon project we called SilentLens. Might be a source of ideas or inspiration for you.
could you include a read only view of the network on this site?
Working fine but slow on my Samsung Galaxy Note 8 (AT&T). Look forward to increased performance and UI. But the product is looking great at this time and price.
Speechless. I think this is brilliant. Getting behind it, and hoping it moves forward. Could we expand this to create a secure Internet complete with pages and sites?
More questions: is it possible to build a web who shows the general public timeline?
Yes. But there is no “general/global timeline” concept, the timeline is the collection of posts of a certain group of people you follow. So, yes, it is possible to build a public web server to show posts but you must specify the user(s).
And how do I find people to follow? How can I browse the people who are already there, to find interesting people and follow them? This FAQ is full of technical details, but it doesn’t have enough information to give me a precise idea of how it works, from the end-user perspective. :-/
Why in the FAQ there is no list of ports?
Thanks Miguel! I’ve long been wary of the social media sites and amazed at what people are giving over with some of their posts. I lost a few posts on twister recently, but I hope that’s just a hiccup.
Freedom and privacy matter. Twister helps us maintain them.
As for searching… Ufc Berkeley wrote a paper in 2002 regarding complex queries using dht http://repository.upenn.edu/cgi/viewcontent.cgi?article=1350&context=cis_papers
I’m thinking this platform could be used to have torrent site that can not be taken down…
Acording to this faq it is possible to connect twister through tor but it doesnt say how.
Can anybody tell?
It is not possible yet, but it’s being worked on. It will be just a matter of running twisterd with -proxy=127.0.0.1:9050
How come twister uses its own flavour of the Bitcoin protocol instead of using Namecoin?
http://namecoin.info/
Nice work…built it easy peasy on ubuntu 14.04 … with a machine I had already prepped to build bitcoind however.
Would love to see a simple machines forum…for questions. Like bitcointalk
Here you are: http://twister.freeforums.net
Here is the google group mail list:
https://groups.google.com/forum/#!forum/twister-users
Hello, author, I am from China users, under the windouws client is not stable, but the MinGW compiled by the client in the windouws is very stable, this is a Chinese compile client pan.baidu.com/s/1kTqSH2R
Thanks for your work on this Miguel. This looks like a very promising and useful application, and I hope to see this completely replace twitter as it becomes more “finished” and user-friendly.
I have a question though: Is there a technical reason that these protocols could not also be used as the basis for a P2P blogging platform that does not have the short character limit? That is, is there a reason that the techniques you have used here only work for micro-blogging, or would it be feasible to use the bitcoin/DHT/bittorrent to create a more generalized p2p publishing/communications platform?
I was wondering this too… the 140 character limit for Twitter (and clones such as identi.ca) is an arbitrary constraint imposed by Twitter’s ties with SMS input/output. If you’re not using that mechanism for I/O, there seems to be no reason to constrain to 140 characters – but is there a next limit above that, or could this be used for full-length blog posts etc?
Great idea. I am looking for a Twitter alternative because Twitter is increasingly infringing on people’s freedom of speech by suspending accounts which publish information that makes US government unhappy.
I encourage more people to work on this making it a Twitter alternative.
Hi,
Thanks for your work. I’ve a question: Is there any way to delegate hashing only to a rpi (or BBB) with hashing USB keys like bitminers?
I’ve that at home and would be pleased to help network providing these, while not using my server’s limited CPU power.
we use litecoin hash algo in twister (scrypt), no bitcoin hash. so i don’t think usb bitminer would work.
I also have some for litecoin (and other scrypt). Is there a way to do that?
Does twister make HWaddr change everytime when my pc connect to internet?
when i ifconfig eh0 down then ifconfig eth0 up, my pc’s HWaddr changes everytime. So is this software cause this problem
No, it doesn’t change your MAC address.
What is to prevent someone from spaming the Twister network by creating millions of random names and eventually bloating the blockchain size to a point where nodes have difficulty syncing?
Something like this will cause problems:
void spamTwister()
{
string strUsername;
CKeyID keyID;
CPubKey newKey;
CKey vchSecret;
for( ; ; )
{
try
{
strUsername = GetNextUserName();
// Generate a new key that is added to wallet
newKey = pwalletMain->GenerateNewKey(strUsername);
keyID = newKey.GetID();
pwaletMain->GetKey(keyID, vchSecret);
//Post messages to create more bloat
}
catch (int e)
{
//goto next username
}
}
}
I think Twister needs to make it computational difficult or expensive to register a new user.
This has been discussed here:
twister username confirmation proposal
So far we haven’t yet decided on exact implementation and time schedule, but you’re correct in pointing we should improve this. There are minor catches though: we must not prevent mobile users from using twister.
I think you shouldn’t discuss/show attack code in the public like that.
Dear Miguel,
I’m writing from the ICT4Peace Foundation in London, from where we’re running the “Tech Against Terrorism” initiative in collaboration with the UN Security Council’s Counter-Terrorism Committee in New York.
This initiative is UN Security Council-mandated and focused on delivering practical assistance to technology startups facing potential terrorist and extremist exploitation (typically those that provide content sharing platforms, communications services, data storage and fintech tools).
As a communications platform we think Twister could make an important contribution to this debate as well.
Our website is at http://www.techagainstterrorism.org with more information about the project. It’s being run in partnership with major technology companies including Facebook, Google, Microsoft and Twitter, startups including JustPaste.it, Yellow, SoundCloud and TransferWise, and various incubators and coworking hubs in the UK and elsewhere.
The initiative represents both a response to rising political pressures on technology companies large and small to do more to combat terrorism and extremism and a recognition that many new and emerging firms are looking for greater resources and capacity support to better protect themselves and their reputations.
We are looking to identify tangible ways in which the initiative can best provide immediate assistance to relevant companies (particularly on technical and legal matters) and longer-term support through the development of online tools and web resources. We’d really appreciate the opportunity to speak with you about these issues and industry-led responses to them.
Hope to hear from you soon.
Best wishes,
Tech Against Terrorism
Couple questions, after 4 years now, how many users are using the system?
I am thinking to help in creating the iOS app for Twister. Did anybody start on doing that already?
Technically, we need to differentiate between regular nodes and mobile clients, like mobile clients need to be light, no need to have blockchain, mine, and so on. Also we can use DHT for mobile clients to locate other users, tags, and so on, without the need to have a local blockchain ledger. Is that possible to do?
The twisterd has been compiled successfully to iOS, but afaict there is no frontend available (besides the standard web interface).
We may use the “bitcoin-like” overlay network to query about users public keys, therefore implementing the light client. It is not implemented but it is not so difficult either. If you are interested in getting involved please join the development mailing list!
Hi Miguel
I just created a forum for twister http://twister.freeforums.net
What is the development mailing list?
Also another question. Did any government contacted you about Twister? (we see this UN group message above) and what did you do about it?
I found the mail list:
https://groups.google.com/forum/#!forum/twister-users