I just want to bring attention to the latest set of cool new features in twister-html, most of those have been contributed by @erkanyuksel and our friends of the Calm client over the last weeks:

• Ability to split long text in multiple posts.
• Selectable theme between Original and Calm.
• Choose between enter or control-enter to send post.
• Preserve/ignore line feeds displaying.
• Automatic conversion of ascii sequences to unicode, for example type “:-)” and get ☺
• Inline image previewing.
• Use proxies to access external links.
• Hiding posts starting with mention to another users.

So if you haven’t updated your client for a while, give it a try! It is just a matter of doing git pull and then refreshing your browser page. Don’t forget to check the “Options” page.

Miguel

Short story: have you noticed some of your posts missing from other user’s mentions, RT’s, replies etc? Upgrade to 0.9.20 now!

Long story:

DHT stands for Distributed Hash Table. twister uses DHT to store user resources like profiles, avatars, but also standard posts. This is not the primary mechanism of posts propagation to your followers (which is bittorrent) but it is the mechanism used to send mentions, to reference (link) replies and stuff.

The thing with DHT is that data persistence is only statistically guaranteed. All peers are each contributing a small bit of storage capability to the twister network as they hold copies of some people’s posts. If, due to an extremely unlucky event, all peers go offline at the same time it would be like losing memory of posts. So, because peers come and go, we periodically perform a refresh on stored keys, that is, each peer tries to re-store the keys he has to the other 8 closest nodes.

Then another bad thing could happen: if the network was temporarily down at the exact time you hit “send” button, your mention wouldn’t be stored at DHT. ever.

Of course this is not good. So with 0.9.20 I’ve made a very small improvement to this logic: all DHT resources that are locally produced will also enter the local list of items to be refreshed. That means that even if your network was down when you hit “send”, twisterd will still get a chance to send this content to the right peers later (given that you leave your twisterd running).

So give it a try!

Miguel

Hi people,

Some twister-related news you might have missed:

• Techpresident running the article Making ‘NSA-Proof’ Social Networking Mainstream on twister and other alternative P2P networks. Note that, contrary to what some may believe, I’ve never claimed twister to be “NSA-Proof”. If you want to know more about it, I greatly recommend reading the ‘threat model’ topic in our FAQ page
• The very cool twister-html fork twister-calm theme by @hedgehog. This is a fork of standard twister’s default html client, implementing a new look and some new features. We’re looking into incorporating those changes – stay tuned!

While 0.9.18 introduced rotating of torrents, to ensure they would never “starve” by not receiving any posts, newer twister 0.9.19 makes a further distinction: torrents of the people you follow are exempted from this rotation. So instead of periodically pausing and resuming these torrents they will be always ready to receive new posts, thus leading to faster timeline updates and quicker delivery of direct messages.

Of course, there’s no such thing as a free lunch. Following hundreds of people will cause more resources on your machine to be used. This trade-off is discussed here:

twister-dev thread

In order to reduce the impact of this change, the size of network buffers used by each connection has been reduced. I believe that for most people the impact on memory should be negligible, but we may need smarter strategies for big twister users.

Try it out and let us know how it works for you!

Android APK and win32 builds are also updated here:

Download page

Debugging the messages (posts) propagation issues during the last few days is being a great learning experience and insightful about the best twister strategies.

twister basically has the problem of distributing torrents (where user’s posts are stored) among peers in a way that is both fair and secure. fair: each user will contribute their small share of processing and storage (keep your twister client running!). secure: prevent denial-of-services as using too much resources.

libtorrent introduces a nice concept for this purpose called “auto managed torrents” (*). Auto managed torrent have limits on the number of simultaneous active torrents, thus keeping resource usage limited, and may decide when to pause/resume each torrent.

Unfortunately twister was not rotating torrents to alternate between active and paused ones. Last torrents to be added where kept queued during the entire execution time of twisterd. Thanks @stark and @kseistrup for the help with debugging which lead to understanding this.

I don’t need to explain that this twister bug caused it to be secure but not fair. So I’ve just committed the code to rotate torrents in queue.

A lot of posts should start to properly propagate now. Welcome to twister-core 0.9.18.

Please download and update (git pull && make).

thanks and have fun.

* btw: libtorrent’s author – Arvid Norberg – deserves a lot of twister credit as it wouldn’t be possible to develop this project in such short time without libtorrent.

new twister-core 0.9.16. please upgrade. fix torrent tracker problem which prevented some users from propagating their posts.

issue #159

Update (Feb 20): the bug is not fixed yet. i reopened the above issue. another update will be required in the next few days.

Update (Feb 21): 0.9.17 is out. the logic for trackers changed a little bit, so they will now announce and reply DHT requests even if their torrents happen to be in a paused state. we need more users on twister network to upgrade their clients to 0.9.17 to properly evaluate how this fix will improve the torrent initialization and therefore the post propagation in general.

Feeling bored?

Ever wondered what people have been saying in promoted posts?

Do you think they might be trying to sell you a product or rather spread out some message about the Ukrainian police?

Now you may find out

Updated twister-core (0.9.15) + twister-html adds a new option “Switch to Promoted posts” so the timeline can be used to browse latest promoted posts, instead of letting them simply get lost, buried into an obscure corner of the blockchain…

Experimental top trending hashtag code was added to twister-core (0.9.14) and twister-html.

It seems to work, it is nice, helps finding about what is going on and everything… but it’s very naive implementation so i guess it will take no time until someone tries to abuse it…

i’m already thinking in improving it by incorporating the notion of “degrees of separation”. We all know we are friend of everybody in the world with less than 6 degrees of separation, right?

Six degrees of separation

The idea is having a crawler thread in twister-core which would periodically go recursively into the following lists (starting with local users) until a certain deep of separation (2? 3? 4?). Then it would collect the list of these usernames as “trusted” users.

The thing with top trends would be that only posts from users in this list would be considered. This way, fake users or spammers (who are not followed by anyone) would not be able to send thousands of posts just to make a top trend. Their posts would be simply ignored.

Discussion of Top Trends and degrees of separation here

Miguel

I know some parts of twister-core might benefit from optimization, but as a follower of Knuth’s optimization mantra i usually try to favor consistency over “premature optimization”.

So it happened that twister started performing very slowly recently. I noticed it and others have noticed it as well. Performance on low-end Android phones was so bad to the point of being unusable.

twisterd went to it’s first profiling session this afternoon. After a few attempts to use kernel perf and oprofile tools, I couldn’t get good call graphs to show me the hot spots (just knowing that CPU is spent inside OpenSSL doesn’t help at all). Finally it was Google’s gperftools that really gave me some useful information.

It turns that an almost one-liner patch is responsible for the most drastic CPU usage reduction in twister ever. Android devices may now run longer on battery. And since this part of the code competes with DHT processing, the quicker response times from other nodes may actually improve the twister’s interactivity as a whole. I’m optimistic. Lets update all the clients and see what happens!

APK and Windows precompiled are already updated to version 0.9.13. Have fun!

New twister-core 0.9.12 is available for download from github repository, win32 bundle and android apk. (you may check the version from network page)

The most notably bugfixes are:

- Check pieces before sending to other peers in swarm #15

This is a long outstanding bug which cause some users to not synchronize their torrent swarms properly. Because torrent swarms is used to update the timeline and to process Direct Messages, a lot of messages were noticed to be missing.

- Stack smashing issues and intermittent openssl errors

Possible Stack smashing in dhtTargetHash #27

stack smashing detected (because of stack protector) #105

core dumped when block chain is finaly updated #97

openssl’s thread id callback is needed for stable operation.

We are not sure if the fix #15 solves all torrent sync issues. There might still be other corner cases to be fixed (Not all posts appear on Postboard #82).

On the other news, soon we will have a new, better and revisited website and wiki pages. Thanks to the new editor @jpfox for taking the job!