Latest git pull: inbox doesn't update

[chan] bitmessage
Jun 14 10:37 [raw]

I noticed that in the latest git pull the inbox doesn't update when new messages arrive. If I select another folder then back to the inbox it's updated with the new messages.

BM-NBctD2fu6pS7M21x5CrkFkkQ25kSvy6M
Jun 15 00:50 [raw]

Yes. This is indicative of an exploit. The vulnerability being exploited is that PyBitmessage does not accept inv messages with exactly 50,000 entries despite generating such messages itself. This results in high connection churn, increased transfers (hundreds of megabytes an hour), increased CPU and memory usage, message propagation delays with the possibility of messages not being propagated at all. This bug has been present ever since the network subsystem was rewritten. The following patch fixes the bug that causes rejection of inv message with exactly 50,000 items and also adds a compatibility fix so that 50,000 entry inv messages are not generated thus avoiding triggering the bug in remote nodes that haven't applied the patch. diff --git a/src/network/bmproto.py b/src/network/bmproto.py index aff6cd0..1f1c67b 100644 --- a/src/network/bmproto.py +++ b/src/network/bmproto.py @@ -300,7 +300,7 @@ class BMProto(AdvancedDispatcher, ObjectTracker): def _command_inv(self, dandelion=False): items = self.decode_payload_content("l32s") - if len(items) >= BMProto.maxObjectCount: + if len(items) > BMProto.maxObjectCount: logger.error("Too many items in %sinv message!", "d" if dandelion else "") raise BMProtoExcessiveDataError() else: diff --git a/src/network/tcp.py b/src/network/tcp.py index 163cbd8..c543ac4 100644 --- a/src/network/tcp.py +++ b/src/network/tcp.py @@ -180,7 +180,10 @@ class TCPConnection(BMProto, TLSDispatcher): for hash, storedValue in bigInvList.items(): payload += hash objectCount += 1 - if objectCount >= BMProto.maxObjectCount: + # Remove -1 below when sufficient time has passed for users to + # upgrade to versions of PyBitmessage that accept inv with 50,000 + # items + if objectCount >= BMProto.maxObjectCount - 1: sendChunk() payload = b'' objectCount = 0

BM-NBctD2fu6pS7M21x5CrkFkkQ25kSvy6M
Jun 15 03:44 [raw]

This patch undoes the conversion to any()/all() conditionals, reverting to the much faster (and IMO prettier format-wise) lazily-evaluated conditionals which should resolve any bugs caused by the premature optimization attempt. diff --git a/src/bitmessageqt/__init__.py b/src/bitmessageqt/__init__.py index 940a0dc..6cce6c3 100644 --- a/src/bitmessageqt/__init__.py +++ b/src/bitmessageqt/__init__.py @@ -2509,18 +2509,14 @@ class MyForm(settingsmixin.SMainWindow): # pylint: disable=too-many-public-meth treeWidget = self.widgetConvert(sent) if self.getCurrentFolder(treeWidget) != "sent": continue - if all( - [ - treeWidget == self.ui.treeWidgetYourIdentities, - self.getCurrentAccount(treeWidget) not in (fromAddress, None, False), - ] + if ( + treeWidget == self.ui.treeWidgetYourIdentities + and self.getCurrentAccount(treeWidget) not in (fromAddress, None, False) ): continue - elif all( - [ - treeWidget in [self.ui.treeWidgetSubscriptions, self.ui.treeWidgetChans], - self.getCurrentAccount(treeWidget) != toAddress, - ] + elif ( + treeWidget in [self.ui.treeWidgetSubscriptions, self.ui.treeWidgetChans] + and self.getCurrentAccount(treeWidget) != toAddress ): continue elif not helper_search.check_match( @@ -2565,12 +2561,10 @@ class MyForm(settingsmixin.SMainWindow): # pylint: disable=too-many-public-meth self.getCurrentSearchLine(tab), ): continue - if all( - [ - tableWidget == inbox, - self.getCurrentAccount(treeWidget) == acct.address, - self.getCurrentFolder(treeWidget) in ["inbox", None], - ] + if ( + tableWidget == inbox + and self.getCurrentAccount(treeWidget) == acct.address + and self.getCurrentFolder(treeWidget) in ["inbox", None] ): ret = self.addMessageListItemInbox( inbox, @@ -2582,12 +2576,10 @@ class MyForm(settingsmixin.SMainWindow): # pylint: disable=too-many-public-meth time.time(), 0, ) - elif treeWidget == all( - [ - self.ui.treeWidgetYourIdentities, - self.getCurrentAccount(treeWidget) is None, - self.getCurrentFolder(treeWidget) in ["inbox", "new", None] - ] + elif ( + treeWidget == self.ui.treeWidgetYourIdentities + and self.getCurrentAccount(treeWidget) is None + and self.getCurrentFolder(treeWidget) in ["inbox", "new", None] ): ret = self.addMessageListItemInbox( tableWidget, @@ -2612,17 +2604,12 @@ class MyForm(settingsmixin.SMainWindow): # pylint: disable=too-many-public-meth unicode(acct.fromLabel, 'utf-8')), sound.SOUND_UNKNOWN ) - if any( - [ - all( - [ - self.getCurrentAccount() is not None, - self.getCurrentFolder(treeWidget) != "inbox", # pylint: disable=undefined-loop-variable - self.getCurrentFolder(treeWidget) is not None, # pylint: disable=undefined-loop-variable - ] - ), - self.getCurrentAccount(treeWidget) != acct.address # pylint: disable=undefined-loop-variable - ] + if ( + self.getCurrentAccount() is not None + and ( + self.getCurrentFolder(treeWidget) not in ("inbox", None) # pylint: disable=undefined-loop-variable + or self.getCurrentAccount(treeWidget) != acct.address # pylint: disable=undefined-loop-variable + ) ): # Ubuntu should notify of new message irespective of # whether it's in current message list or not @@ -2789,13 +2776,9 @@ class MyForm(settingsmixin.SMainWindow): # pylint: disable=too-many-public-meth upnpThread = upnp.uPnPThread() upnpThread.start() - if all( - [ - BMConfigParser().get( - 'bitmessagesettings', - 'socksproxytype') == 'none', - self.settingsDialogInstance.ui.comboBoxProxyType.currentText()[0:5] == 'SOCKS', - ] + if ( + BMConfigParser().get('bitmessagesettings', 'socksproxytype') == 'none' + and self.settingsDialogInstance.ui.comboBoxProxyType.currentText()[0:5] == 'SOCKS' ): if shared.statusIconColor != 'red': QtGui.QMessageBox.about( @@ -2808,11 +2791,9 @@ class MyForm(settingsmixin.SMainWindow): # pylint: disable=too-many-public-meth ("Bitmessage will use your proxy from now on but you may want to manually restart " "Bitmessage now to close existing connections (if any)."))) - if all( - [ - BMConfigParser().get('bitmessagesettings', 'socksproxytype')[0:5] == 'SOCKS', - self.settingsDialogInstance.ui.comboBoxProxyType.currentText()[0:5] != 'SOCKS', - ] + if ( + BMConfigParser().get('bitmessagesettings', 'socksproxytype')[0:5] == 'SOCKS' + and self.settingsDialogInstance.ui.comboBoxProxyType.currentText()[0:5] != 'SOCKS' ): self.statusbar.clearMessage() state.resetNetworkProtocolAvailability() # just in case we changed something in the network connectivity @@ -2879,11 +2860,9 @@ class MyForm(settingsmixin.SMainWindow): # pylint: disable=too-many-public-meth acceptableDifficultyChanged = False - if any( - [ - float(self.settingsDialogInstance.ui.lineEditMaxAcceptableTotalDifficulty.text()) >= 1, - float(self.settingsDialogInstance.ui.lineEditMaxAcceptableTotalDifficulty.text()) == 0, - ] + if ( + float(self.settingsDialogInstance.ui.lineEditMaxAcceptableTotalDifficulty.text()) >= 1 + or float(self.settingsDialogInstance.ui.lineEditMaxAcceptableTotalDifficulty.text()) == 0 ): if BMConfigParser().get( 'bitmessagesettings', @@ -2901,11 +2880,9 @@ class MyForm(settingsmixin.SMainWindow): # pylint: disable=too-many-public-meth self.settingsDialogInstance.ui.lineEditMaxAcceptableTotalDifficulty.text() ) * defaults.networkDefaultProofOfWorkNonceTrialsPerByte))) - if any( - [ - float(self.settingsDialogInstance.ui.lineEditMaxAcceptableSmallMessageDifficulty.text()) >= 1, - float(self.settingsDialogInstance.ui.lineEditMaxAcceptableSmallMessageDifficulty.text()) == 0, - ] + if ( + float(self.settingsDialogInstance.ui.lineEditMaxAcceptableSmallMessageDifficulty.text()) >= 1 + or float(self.settingsDialogInstance.ui.lineEditMaxAcceptableSmallMessageDifficulty.text()) == 0 ): if BMConfigParser().get( 'bitmessagesettings', @@ -2932,11 +2909,9 @@ class MyForm(settingsmixin.SMainWindow): # pylint: disable=too-many-public-meth # start:UI setting to stop trying to send messages after X days/months # I'm open to changing this UI to something else if someone has a better idea. - if all( - [ - self.settingsDialogInstance.ui.lineEditDays.text() == '', - self.settingsDialogInstance.ui.lineEditMonths.text() == '', - ] + if ( + self.settingsDialogInstance.ui.lineEditDays.text() == '' + and self.settingsDialogInstance.ui.lineEditMonths.text() == '' ): # We need to handle this special case. Bitmessage has its default behavior. The input is blank/blank BMConfigParser().set('bitmessagesettings', 'stopresendingafterxdays', '') BMConfigParser().set('bitmessagesettings', 'stopresendingafterxmonths', '') @@ -2956,11 +2931,9 @@ class MyForm(settingsmixin.SMainWindow): # pylint: disable=too-many-public-meth if lineEditMonthsIsValidFloat and not lineEditDaysIsValidFloat: self.settingsDialogInstance.ui.lineEditDays.setText("0") if lineEditDaysIsValidFloat or lineEditMonthsIsValidFloat: - if all( - [ - float(self.settingsDialogInstance.ui.lineEditDays.text()) >= 0, - float(self.settingsDialogInstance.ui.lineEditMonths.text()) >= 0, - ] + if ( + float(self.settingsDialogInstance.ui.lineEditDays.text()) >= 0 + and float(self.settingsDialogInstance.ui.lineEditMonths.text()) >= 0 ): shared.maximumLengthOfTimeToBotherResendingMessages = sum( float(str(self.settingsDialogInstance.ui.lineEditDays.text())) * 24 * 60 * 60, @@ -4289,17 +4262,13 @@ class MyForm(settingsmixin.SMainWindow): # pylint: disable=too-many-public-meth myAddress = tableWidget.item(currentRow, 0).data(QtCore.Qt.UserRole) otherAddress = tableWidget.item(currentRow, 1).data(QtCore.Qt.UserRole) account = accountClass(myAddress) - if all( - [ - isinstance(account, GatewayAccount), - otherAddress == account.relayAddress, - any( - [ - currentColumn in [0, 2] and self.getCurrentFolder() == "sent", - currentColumn in [1, 2] and self.getCurrentFolder() != "sent", - ] - ), - ] + if ( + isinstance(account, GatewayAccount) + and otherAddress == account.relayAddress + and ( + currentColumn in [0, 2] and self.getCurrentFolder() == "sent" + or currentColumn in [1, 2] and self.getCurrentFolder() != "sent" + ) ): text = str(tableWidget.item(currentRow, currentColumn).label) @@ -4644,12 +4613,10 @@ class MyForm(settingsmixin.SMainWindow): # pylint: disable=too-many-public-meth if column != 0: return # only account names of normal addresses (no chans/mailinglists) - if any( - [ - not isinstance(item, Ui_AddressWidget), - not self.getCurrentTreeWidget(), - self.getCurrentTreeWidget().currentItem() is None, - ] + if ( + not isinstance(item, Ui_AddressWidget) + or not self.getCurrentTreeWidget() + or self.getCurrentTreeWidget().currentItem() is None ): return @@ -4719,11 +4686,9 @@ class MyForm(settingsmixin.SMainWindow): # pylint: disable=too-many-public-meth if tableWidget.item(currentRow, 0).unread is True: self.updateUnreadStatus(tableWidget, currentRow, msgid) # propagate - if all( - [ - folder != 'sent', - sqlExecute('''UPDATE inbox SET read=1 WHERE msgid=? AND read=0''', msgid) > 0, - ] + if ( + folder != 'sent' + and sqlExecute('''UPDATE inbox SET read=1 WHERE msgid=? AND read=0''', msgid) > 0 ): self.propagateUnreadCount()

BM-NBTqk3386Tx8sXJp7UmVZKLLoGZ6eX1B
Jun 15 03:55 [raw]

Uhh... what?

[chan] bitmessage
BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY

Subject Last Count
Why did all my messages vanish? Jun 19 21:19 5
asyncronous data Jun 19 19:52 5
PyBitmessage Security Scan on Branch v0.6 Jun 19 12:02 59
onionscan update Jun 19 11:06 8
Feature request: delete all messages from user Jun 19 05:52 3
ERROR - Too many items in inv message! Jun 19 05:45 15
Feature request: delete all messages from user Jun 18 23:40 1
test Jun 18 23:24 5
Inbox bug Jun 18 22:20 5
Integration with GPG (GnuPG) Jun 18 22:20 4
attack? Jun 18 22:10 3
a GOOD implementation of 2fa for conventional email please! Jun 18 21:03 1
unpickling knownnodes to a readable format Jun 18 04:43 27
WARNING - Probably wrong category number in playSound() Jun 17 09:41 1
I don't receive any BMs when I have only one peer. Jun 16 17:13 6
identicon code bug? Jun 16 06:35 1
Free Git Replacement Jun 15 17:31 8
github Jun 15 04:35 1
Latest git pull: inbox doesn't update Jun 15 03:55 4
IPFS Jun 13 21:48 8
latest in the spy world Jun 13 19:14 2
(no subject) Jun 13 19:12 1
TIMESERVICE Jun 13 19:05 1
Questions about BM nodes Jun 12 22:53 7
D2A41B229F7BCE6F9B429D3E33A47598 Jun 12 22:26 1
Why not reject old clients from connection to the network? Jun 12 19:18 10
Add an option to connect only to onions Jun 12 00:42 2
Help Improving Algorithm Jun 11 23:48 3
hey - why not make pyBM as shitty as "Signal-App" by Marlinspike ? Jun 11 21:53 1
Silence debug.log foe less disk-write Jun 11 14:44 4
Questions about "Max acceptable difficulty" Jun 11 04:24 2
"Post to BM" API Jun 10 12:11 5
"Configuration NOT changed" Jun 10 09:41 2
Error/Warnings in debug log: Should I worry? Jun 10 09:34 1
Bitmessage Security Test: ZWD attempt Jun 10 08:05 1
bitmessage inaccessible Jun 10 08:04 1
mailchuck.com email gateway Jun 10 07:47 3
Microsoft owns GitHub Jun 9 15:23 1
NIST key management guidelines suggest that 15360-bit RSA keys are equivalent in strength to 256-bit symmetric keys… Jun 9 11:25 12
Cloudflare MITM blocker Jun 9 11:21 4
GitHub Jun 9 11:16 15
Improvement of Trustedpeer setting Jun 6 06:26 2
blank blank blank Jun 6 06:26 5
is multiple trustedpeers possible? Jun 6 01:00 7
Bitmessage Documentation Bug Jun 2 10:09 4
REAL security experts endorse "security by obscurity" May 31 13:22 7
TRUE LOVE May 31 06:38 1
PyBitmessage Security ... Security Levels May 30 04:56 2
How to force BM to use only .onion nodes? May 30 04:56 15
Dread May 29 16:31 1
6F3F2CF9891928A25B71BBC4707B8753 May 29 10:56 1
SMTP and IMAP integration in the client May 29 06:21 5
Bitmessage Wiki Blocked May 29 00:51 12
Desiderata May 28 20:07 2
Bitmessage Bug May 28 17:15 2
I solved the Bitmessage Captcha Puzzle! May 28 08:56 2
setup trusted peer question May 28 08:05 6
bitmessagemain from pyinstaller executable won't run May 28 07:41 4
help with messages.dat May 28 07:36 7
Roll Your Own Crypto! May 28 07:06 6
look closely May 27 18:33 5
How to use chan alt.anonymous.messages May 27 08:21 2
feature request May 27 01:34 10
YOU WANNA HIRE A LEGIT HACKER????? May 26 04:39 5
Security Test on PyBitmessage Branch Master May 26 00:11 1
#2 May 25 22:41 6
minimum difficulty for chans May 25 16:45 16
BM-2cWkFSxB4cyeNVr99tgJdkMA2nfivbXLiH May 25 07:07 2
ein kleines pyBM Nebenproblem in KDE LiquidShell May 24 18:14 1
PyBitmessage 0.6.3.2 blacklist whitelist May 24 06:29 6
Test DML May 24 02:18 1
Now, following my own advice, adding channel bitmessage and general to the blacklist May 23 15:50 1
hyperboria node [fc5b:acf7:9762:439c:394d:02bb:d603:05de]:8444 May 23 01:34 1