Sending ACKs from hidden messages

[chan] bitmessage
Oct 12 16:27

Hi Peter, all, I'd like to start a discussion around PR1067 https://github.com/Bitmessage/PyBitmessage/pull/1067, as I'm not 100% convinced by the arguments that have been put forward. Currently, PyBitmessage ignores the ACK objects enclosed in type zero messages, breaking the standard protocol behaviour. Specifically, even if the destination address advertises that will send ACKs in its pubkey bitfield, and the message contains an ACK, if the message encoding is set to zero ("invisible") the ACK will not be sent, which is contrary to the protocol definition and the senders' expectation. It results in rebroadcasts and resource usage on sender side (because each rebroadcast requires new POW). PR1067 enables standard ACK processing for hidden messages. The reason for rejection was that it can be abused as an attack vector for deanonymisation, which is true. I would like to discuss and understand the risks and benefits a little more: 1) At a network-wide level, the ability to send ACKs from invisible messages has documented privacy benefits. For example, a use case from the original whitepaper (chapter 7 "passive operating mode") is increasing ACK anonymity by wrapping the original ACK in a message to another network user. In that case, the privacy is improved if the wrapper message is hidden, as Charlie is (legally) unaware that he is being used by Bradley as a reflector for Alice's ACK. And there are more interesting uses of the ACK mechanism for privacy enhancement purposes (remailers etc) that would benefit from network-wide protocol consistency. 2) At an individual level, honoring ACKs in hidden messages does not create a new attack vector. What it does is change a well-known, overt attack vector into a covert one. What could be achieved with hidden message ACKs can already be achieved today with visible messages “testing bitmessage, please ignore” or “unknown encoding” or “hack your spouse” and the only difference is that right now the attack could be noticed, but still not BLOCKED (other than by going fully offline). At the end of the day, the “dontsendack” flag is, and will remain the main control switch for any user concerned about ACK privacy. What I want to understand is: is the small privacy loss from point 2 more important than the small privacy gain from point 1? And is all this significant enough to justify a protocol inconsistency in pyBitmessage? And if yes, perhaps making this behaviour optional (via a “don't ack hidden messages” boolean) would be a better solution to address the different needs of different users? Ultimately, I respect the experience of the senior developers and will defer to their decision. I just want to make sure we're not missing out on a potential privacy-enhancing feature. Let's discuss.

BM-2cX62WCeFcUwzXWqxTBfaAzNy4j1y8yZVm
Oct 12 17:53

Hi, thank you for the feedback. > 1) At a network-wide level, the ability to send ACKs from invisible > messages has documented privacy benefits. For example, a use case > from the original whitepaper (chapter 7 "passive operating mode") is > increasing ACK anonymity by wrapping the original ACK in a message > to another network user. Yes, onion routing (although the whitepaper doesn't use that name). I want to move onion routing into a separate "thing". I'll probably just repurpose encoding 0. However, this will be opt-in, indicated by a bitfield. If you look at the protocol specification page in the wiki, it says bit 27 = onion_router (proposal). BTW, when I present about bitmessage, I do mention onion routing capability, and also that it's currently disabled for security reasons. Onion routing can benefit from some further tweaks, like TTL optimisation, online signaling and so on, and they should also have a separate interface (e.g. ordering in the "Messages" tab, or putting them into a separate tab). The pubkey objects for onion routers could be unencrypted so that anyone can use them, and have a shorter TTL to indicate when the router is online. There could also be further anonymisation increase by fragmentation and/or padding capabilities but I haven't thought about the design yet. They also could have the "normal" functionality disabled so that they only act as onion routers but not as normal senders/recipients. Onion routing will have further benefits when streams are implemented. > 2) At an individual level, honoring ACKs in hidden messages does not > create a new attack vector. What it does is change a well-known, > overt attack vector into a covert one. Correct. This would allow things like sending short-TTL-ed messages 24/7, and result in a fine-grained detection of when the target is online. Or if two addresses are hosted by the same node. If it's stealthy, you can do it over days/weeks without the target being aware of anything. This data can be used as additional input together with other deanonymisation techniques. > What I want to understand is: is the small privacy loss from point 2 > more important than the small privacy gain from point 1? There are other alternatives, like the one I described above. If you want, you can submit a pull requests that implements bitfield 27 and nothing else. The optimisation can be done incrementally. > And if yes, perhaps making this behaviour optional > (via a “don't ack hidden messages” boolean) would be a better > solution to address the different needs of different users? Or the converse, "do ack hidden messages" boolean, which is what my proposal is. > Ultimately, I respect the experience of the senior developers and > will defer to their decision. I just want to make sure we're not > missing out on a potential privacy-enhancing feature. I don't think I have some uber security experience (although I've been programming for 30 years and did work for a credit card payments company among other things). I prefer heuristics and other experts giving me feedback. One thing that I do is "default to more anonymity for myself" (i.e. the person running the software), which is why UPnP is disabled by default, and if you're on tor, by default you aren't accepting incoming connections and local peer discovery is disabled. It's also why I requested public feedback on peer discovery before implementing it. > Let's discuss. Ok. Let's make it opt-in, and incrementally optimise. Later, there could also be some wizard which will ask if you want to add an onion router address on first launch, or there could be heuristics which when detect you're online all the time, would recommend activating it. Peter Surda Bitmessage core developer

[chan] bitmessage
Oct 12 22:57

Peter, thanks for the clarifications. I should have asked the question before going ahead and submitting the PR. :) >> increasing ACK anonymity by wrapping the original ACK in a message to another network user. > Yes, onion routing (although the whitepaper doesn't use that name). I want to move onion routing into a separate "thing". I'll probably just repurpose encoding 0. However, this will be opt-in, indicated by a bitfield. If you look at the protocol specification page in the wiki, it says bit 27 = onion_router (proposal). > Onion routing can benefit from some further tweaks In brief, you are considering conflating hidden messages under onion routing, using bit 27 and encoding type zero. This makes sense from a certain perspective (there are significant overlaps between the two), but there is also potential to cause confusion and possibly break some existing edge cases: 1) I think, and will detail later, that there are use cases for type zero hidden messages outside of onion routing, and possibly even onion routing methods that don't use hidden messages; also, as you said, onion routing has a different set of optimizations and warnings. 2) I think we should not conflate technical capability (sending ACKs) with advertised intention (acting as onion router). Even if technically they're mostly the same under the hood, in the non-technical world they can have different connotations, perhaps even opposite: to some people, standards compliance is commendable, onion routing is subversive. >Correct. This would allow things like sending short-TTL-ed messages 24/7, and result in a fine-grained detection of when the target is online. Or if two addresses are hosted by the same node. If it's stealthy, you can do it over days/weeks without the target being aware of anything. Yes, the covert factor is indeed a significant issue, however note that there's nothing (except going whitelist-only) to stop a would-be attacker from doing the above OVERTLY right now. That it doesn't happen tells me that, from an attacker's perspective, the outcome is probably not worth the effort compared to other methods currently in use. > If you want, you can submit a pull requests that implements bitfield 27 and nothing else... > ... "do ack hidden messages" boolean, which is what my proposal is. This makes a lot of sense. I have an idea that may cover the onion routing feature, the edge cases and everything in between, but uses two bits from the bitfield. I'll send as a separate message and would like to know your thoughts.

[chan] bitmessage
Oct 12 23:14

Hi again, As I said, I think the current roadmap to onion routing and hidden message acknowledgements may cause some confusion and risks breaking some legitimate use cases. I suggest that a healthy balance could be achieved by using two separate bitfield bits (existing onion_router and a new bit send_hidden_ack) to control hidden ACK sending and onion routing independently from each other. To illustrate, I'll use some back-alley pseudocode: 1) Original (pre-mitigations) behaviour: “If message decrypted correctly, then if my address does_ack, then send the ACK” 2) Current behaviour: “If message decrypted correctly, then if my address does_ack, then send the ACK, except if the message is hidden, in which case drop the ACK” 3) Current roadmap behaviour: “If message decrypted correctly, then if my address does_ack, then send the ACK, except if the message is hidden, in which case drop the ACK, except if my address is also onion_router, in which case DO send the ACK, but also route onions, and optimize operation for onion routing etc, as an all-or-nothing bundle” # notice it's a bit hard to follow and what if I just want to send the ACK? 3b) Proposed behaviour using a second bit “does_hidden_ack”, 0 by default: “If message decrypted correctly, then if my address does_ack, then send the ACK, except if the message is hidden, then if my address does_hidden_ack, then send the ACK, in all other cases drop the ACK”, and independently from all of the above: “If my address is onion_router, then optimize for onion routing and do other cool things that onions do but regular addresses don't – promiscuous uptime protocol etc”, and finally: “Warn if onion_router is used without does_ack” Cons: uses two bits from the bitfield, adds complexity. Pros: mainly deriving from having independent controls for hidden acks and the full onion router behaviour: - cleaner and easier to understand - can perform stealth onion routing as a sub-channel of regular text messages, not just type zero (deniability is realized in this case via PR1062 that puts realistic message-like objects in the ack slot of regular non-onion messages, further blurring the distinction between ACKs and text messages) - can enable hidden message ACK functionality without advertising as an onion router, for non-onion use cases such as targeted public key distribution etc. Basically, it puts full control in the hands of the user, without cutting corners or forced bundling, and enables modular development and testing of clip-on solutions based on the ACK mechanism (like onion routing) without requiring changes in the object layer, or breaking compatibility with other clients. What do you think?

[chan] bitmessage
Oct 12 23:49

First, PyBitmessage is not violating the spec by ignoring ACKs in message encoded 0 messages. The spec says "Any data with this number may be ignored" so the pubkey, message, ack, and/or signature can be ignored (although technically it doesn't make sense to verify the signature if the pubkey is ignored). Second, the spec also says that "The sending node might simply be sharing its public key with you" which implies it is acceptable to use message encoding 0 to privately convey a pubkey. This is quite a neat feature allowing users to not publish pubkeys publicly and/or customize pubkeys for individual contacts by specifying different PoW difficulties and behavor bitfields. eg a user could specify that they will send ACKs for, and/or accept multi-part messages from, specific contacts. Implementations could easily generate a template message for message encoding 0 (similar to what PyBitmessage does currently) indicating the senders changed preferences for communicating with them. Message encoding 0 messages are only "hidden" in the sense that the spec only has 2 sentences about them thus implementors don't know what to do with them.

[chan] bitmessage
Oct 13 00:25

"In general engineering design contexts, the principle can be taken to mean that a component of a system should behave in a manner consistent with how users of that component are likely to expect it to behave; that is, users should not be astonished at the way it behaves" https://en.wikipedia.org/wiki/Principle_of_least_astonishment

[chan] bitmessage
Oct 13 01:14

So which is least astonishing, that something that may be ignored: a) is ignored, or b) is not ignored?

[chan] bitmessage
Oct 13 01:56

Tough question. Both answers are equally unastonishing. Perhaps the more relevant "hidden" question is which is least astonishing, that the encoding field effects: a) only how the message field is processed, or b) how any field in the msg object is processed? Since only 1 out of the 4 message encodings effects a field other than the message field then clearly (a) is least astonishing and therefore it is astonishing that ACKs are handled differently depending on the encoding of the message field.

BM-2cX62WCeFcUwzXWqxTBfaAzNy4j1y8yZVm
Oct 13 11:31

Well, since Bitmessage has a special security purpose, perhaps the correct "least astonishing" option should be if none of the adversaries show up at your doorstep because they deanonymised you. https://www.youtube.com/watch?v=_bSEfx6D8mA Peter Surda Bitmessage core developer

[chan] bitmessage
Oct 13 13:59

You make some very good points. The more I think about the attack that Peter described, I realize that the one-line patch from PR1067 was not the right way to do it. So thanks Peter for rejecting it. :) To summarise my previous stream-of-consciousness message, I think there should be several different levels on the privacy / functionality scale of an address object: - 0: send no ACKs - the equivalent of "dontsendack=True" in the current version, or setting the send_ack - 1: send ACKs from visible messages only - this is the default setting in the current version - 2: send ACKs from all message objects, visible and invisible - there is no such option currently; I suggested using a separate bit "send_hidden_acks" or similar - 3: act as an onion router, advertise this publicly and enable optimizations - the planned "onion_router" bit 27. Retaining option 1 as default seems like the safest option going forward. Really interesting discussion about the POLA, however in this case the proverbial balance between security and convenience appears pretty compelling to me. The ensuing astonishment can be smoothed a bit by documenting the hows and whys, and by providing an option to enable (at own risk) the "expected" behaviour. I also strongly believe that there is a technical need for the option 2 between 1 and 3. As long as this defaults to disabled and requires manually activation as an "advanced setting", it shouldn't introduce an extra risk to the average user. My only concern is at a protocol design level, about the sub-optimal use of the bitfield space. Long-term, the ability that you mentioned below, to customize and push these settings individually for each addressbook entry shows serious promise from a "choose what to share and with whom" angle, considering that currently users only have an all-or-nothing choice. Separate topic for some other time. Any thoughts or suggestions?

[chan] bitmessage
Oct 13 14:02

You make some very good points. The proverbial balance between and messages only concern is the POLA, however in message encoded messages only concern is a bit send no Acks in the right way to do it. To summarise my previous stream of the POLA, however in the more I realize that you mentioned below, to disabled and As this case the ensuing astonishment can be several different levels on the send no acks in the more I suggested using a separate bit send no Acks from all message, objects, visible messages only have an advanced setting the bitfield space. Any thoughts or similar act as this is a technical need for the spec only have an all message I think about the enable optimizations the one line patch from all message objects, visible messages only have an all message encoding to me; in the expected behaviour. My previous stream of the average user; choose what to enable optimizations the average user. Any thoughts or suggestions?

[chan] bitmessage
Oct 13 16:25

Thought: this Markov messages are gettin better, AI behind them? Suggestion: Never mind, motherfucka, just fuck ya brain off!

[chan] bitmessage
BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY

Subject Last Count
BM slow suddenly? Oct 20 10:19 3
How long until streams work? Oct 19 20:35 1
demanded difficulty of messages Oct 19 13:41 11
Python host to run bitmessage Oct 19 11:21 5
chanbot Oct 17 18:01 3
Ukraine Kiev Now Oct 17 13:47 2
how to resend pubkey Oct 17 08:02 4
What's a status of DevTalk pseudo-mailing list? Oct 17 00:54 10
Question on message decryption Oct 16 11:43 22
How they are trying to derail Bitmessage development Oct 16 07:32 4
Question on message decryption ( Edge Case Best config ) Oct 15 16:21 2
new BM mod via 0install Oct 14 21:53 4
pyBM eats too many CPU cycles Oct 14 18:14 43
Sending ACKs from hidden messages Oct 13 16:25 12
Using whatsapp, anybody? Oct 13 13:52 1
Message to Julian Assange. Oct 13 00:47 18
Future, philosophy and guns (was: Re: Message by Julian Assange) Oct 12 09:04 1
Message by Julian Assange. Oct 12 02:24 2
bitmessage implementation in any other programming language Oct 10 03:45 29
Message to a friend, with copy to Julian Assange. Oct 8 17:20 1
Advantage of using namecoin? Oct 8 01:16 3
ISS Space Station - Augmented Virtual Reality Oct 7 20:54 8
shadowchat vs. BM comparison Oct 7 13:54 14
why not use Lazarus to make a BM GUI ? Oct 5 16:02 5
unicode support in bitmessage (total msg size) Oct 4 17:33 5
Node list fuckery, mitigations? Oct 3 18:14 23
unicode support in bitmessage Oct 3 17:50 11
PGP integration with bitmessage Oct 2 19:08 23
Why getopt? Oct 2 14:42 5
monkeys with typewriters Sep 30 23:26 7
RFC: HTTP/S-based secondary peer wire protocol (proposal) Sep 30 23:07 4
Secret ANTIFA Handbook Exposes Real Agenda of AntiFa Sep 29 15:57 9
Building instructions for Windows Sep 29 07:29 25
tor browser sandbox 0.0.20 r48240 Sep 29 02:16 1
BM GUI via API using monkey studio Sep 29 01:22 19
addon to pyBM : otp crypter -- soon to be integrated into BM for extra safety ! Sep 28 22:39 2
I stick with pyBM 0.6.0 Sep 28 12:52 4
( WAS ..addon to pyBM ) Announce Sep 28 08:23 3
alternative to Qt-Designer ? none it seems Sep 27 23:44 2
(no subject) Sep 27 23:37 1
pyBM eats too many CPU cycles Sep 27 23:33 4
Why Bitmessage has waves of online activity? Sep 27 23:16 5
beamstat.com is being cached by google Sep 27 23:15 7
turn MiNode into a full client - simple task ! Sep 27 00:32 2
BitText XHKhFPCDzj: ultimate bitmessage forum Sep 26 21:53 2
GET XHKhFPCDzj Sep 26 21:44 1
28 days Sep 25 07:18 3
peer rating Sep 25 06:44 5
TTL Tweaks Sep 25 06:34 13
How to.. Sep 24 16:43 2
BitText Error Sep 24 16:32 1
BitText LIST Sep 24 16:16 1
HELP Sep 24 15:49 2
BitText ADD confirmation Sep 24 15:34 1
use MiNode py3 app to route a BM via "stream 7" Sep 24 14:07 1
BM GUI via API using KDevelop Sep 23 18:21 1
Email client integration Sep 23 16:34 10
got green light Sep 23 10:59 2
Git pull error Sep 23 08:49 13
Kdevelop + qt-designer for python Sep 22 21:47 1
NATO member Turkey boast that Russian S-400 SAMs can take out American B-52s, F-22s and Tomahawks Sep 22 12:24 1
bug? pyBM eats 17% of my 6core CPU cycles SOLVED :-) Sep 22 11:41 1
bug? pyBM eats 17% of my 6core CPU cycles Sep 22 11:37 1
monkey studio Sep 22 11:06 1
MX.forum.cool Sep 22 10:56 1
so stream 7 cannot work ? Then what did I do ? Sep 22 10:44 1