pickle puzzle

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 11 18:49 [raw]

I tried to unpickle knownnodes.dat, remove all the non-onion entries, then repickle it so that bitmessage will load it properly. I tried with standard pickle commands, edited the json output, but when I repickle it for some reason bitmessage does not recognize it and just erases the file and starts over on startup with the default bootstrap entries. Bitmessage is not able to load the edited pickle. Does any python guru care to lend a hand at what I need to investigate to get the desired result?

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 11 18:59 [raw]

for what?

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 11 19:17 [raw]

the original post already answered this question

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 11 19:29 [raw]

It didn't answer WHY you want to remove the non-onion entries. Especially since BitMessage is just going to get them from the connected nodes again anyway.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 11 19:36 [raw]

Exactly. Why someone wants to delete non-onion entries?

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 11 23:34 [raw]

Add to torrc: SocksPort 127.0.0.1:9151 IsolateDestAddr OnionTrafficOnly Then configure bitmessage to use that new port 9151 and it will only connect to onion nodes. Optionally you can delete knownnodes.dat and add to keys.dat: [knownnodes] maxnodes = 1000 This will reduce the number of nodes in knownnodes to 1000, retreive the freshest onion nodes, and connect to those onion nodes them faster. Don't set maxnodes too low or you may never get more than the default boostrap onion node. If you lower maxnodes and don't delete knownnodes.dat, you may end up with 1000 non-onion nodes and never connect at all.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 00:12 [raw]

import argparse import imp import sys try: import cPickle as pickle except ImportError: import pickle # Dynamically generate and import stripped down 'state' module containing just # the Peer namedtuple to unpickle knownnodes successfully and output a pickle # data stream PyBitmessage understands module_name = 'state' module_code = '''from collections import namedtuple Peer = namedtuple('Peer', ('host', 'port')) ''' module = imp.new_module(module_name) exec module_code in module.__dict__ sys.modules[module_name] = module parser = argparse.ArgumentParser() parser.add_argument('knownnodes', type=argparse.FileType('rb')) args = parser.parse_args() streams = {} for stream, nodes in pickle.load(args.knownnodes).iteritems(): streams[stream] = {peer: attrs for peer, attrs in nodes.iteritems() if peer.host[-6:] == '.onion'} pickle.dump(streams, sys.stdout)

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 04:19 [raw]

In much older threads, some have worried that malware could overwrite knownnodes.dat with malicious bootstrap entries. I don't recall the results of the threads, but erasing the file and starting over with the default bootstrap entries would be a good precaution for bitmessage to perform in that situation.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 05:16 [raw]

If malware is able to modify knownnodes.dat, I'd be more worried about an exploit than malicious bootstrap nodes.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 06:29 [raw]

WHY? none of your business, that's why.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 08:16 [raw]

Add to torrc: SocksPort 127.0.0.1:9151 IsolateDestAddr OnionTrafficOnly Then configure bitmessage to use that new port 9151 and it will only connect to onion nodes. Optionally you can delete keys.dat and add to knownnodes.dat: [knownnodes] maxnodes = 1000 This will reduce the number of nodes in knownnodes to 1000, retreive the freshest onion nodes, and connect to those onion nodes them faster. Don't set maxnodes too low or you may never get more than the default boostrap onion node. If you lower maxnodes and don't delete keys.dat, you may end up with 1000 non-onion nodes and never connect at all.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 11:02 [raw]

I rate it an eight, mate.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 11:03 [raw]

gr8 b8, m8, I r8 8/8

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 16:15 [raw]

You think you have superior rationale. The subject is not malware. The subject is how to load the pickle with onion only nodes. It doesn't matter why.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 16:15 [raw]

If I were malware I would send me your keys.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 16:15 [raw]

this is a clever workaround

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 16:18 [raw]

> exec module_code in module.__dict__ dangerous vector for injection

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 21:23 [raw]

Not really. An attacker has to be able to modify memory or the file. exec or no exec, an attacker can do whatever they want if they can modify memory or the file.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 13 00:04 [raw]

from argparse import ArgumentParser, FileType from collections import namedtuple from copy_reg import _reconstructor Peer = namedtuple('Peer', ('host', 'port')) safe_class_map = { 'copy_reg._reconstructor': _reconstructor, '__builtin__.tuple': tuple, 'state.Peer': Peer, } def get_class(module, name): fullname = '%s.%s' % (module, name) klass = safe_class_map.get(fullname, None) if not klass: raise UnpicklingError('Unpickling %s not supported' % fullname) return klass parser = ArgumentParser(description='Remove non-onion entries from knownnodes') parser.add_argument('input', nargs='?', default='-', type=FileType('rb'), help='Path of knownnodes.dat to filter or - for stdin (default: stdin)', metavar='in_knownnodes') parser.add_argument('output', nargs='?', default='-', type=FileType('wb'), help='Path to save filtered knownnodes or - for stdout (default: stdout)', metavar='out_knownnodes') args = parser.parse_args() try: import cPickle as pickle unpickler = pickle.Unpickler(args.input) unpickler.find_global = get_class except ImportError: import pickle class Unpickler(pickle.Unpickler): def find_class(self, module, name): return get_class(module, name) unpickler = Unpickler(args.input) streams = {} for stream, nodes in unpickler.load().iteritems(): streams[stream] = {peer: attrs for peer, attrs in nodes.iteritems() if peer.host[-6:] == '.onion'} args.output.write(pickle.dumps(streams).replace('__main__', 'state', 1))

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 13 01:03 [raw]

slick work

[chan] bitmessage
BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY

Subject Last Count
Vuvuzela - anonymous messaging that scales to millions of users Nov 12 16:07 6
forwarding in BM Nov 12 15:04 5
ending the waffle Nov 12 10:55 6
Dear Freemasons Nov 12 07:13 2
CSS3 in Bitmessage interface Nov 12 06:56 1
Pastwatch & Aqua Distributed Version Control Nov 11 11:56 1
SOLUTION for spam Nov 11 11:56 23
Vuvuzela - Metadata-private messaging Nov 11 11:56 1
tes Nov 9 11:19 2
I'm back Nov 9 03:35 8
Bitmessage Network Health Report Nov 7 23:48 15
nodejs clientr KEWLIO Nov 7 07:26 4
Scalability Idea Nov 7 07:24 7
Do NOT spam Nov 7 03:09 8
here is the trick to run pyBM on a server without trouble Nov 5 18:41 8
Scalability Question?? Nov 5 09:09 3
re Re: Scalability Question?? Nov 5 08:21 1
aaa Nov 5 02:48 1
Bitmessage Plugins Nov 3 21:33 3
Any nodejs interface to the bitmessage api yet? Nov 3 19:12 2
Recent API status bug Nov 2 12:38 9
zero bundle -- 0net Nov 2 10:41 4
zero git on 0net Nov 1 12:43 6
(no subject) Nov 1 02:48 6
greetings Oct 31 23:05 3
Re: Oct 31 22:25 1
{ ^ } break { ^ } Oct 31 22:11 1
(no subject) Oct 31 14:33 5
INVALID FORMAT Oct 31 12:12 6
hello world Oct 31 07:40 1
Is there anybody out there? Oct 30 08:03 12
join the darknet - be badass at leakswldjpesnuvn.onion Oct 29 20:33 5
more cores, slower pyBM Oct 29 01:36 15
new bitboard thread Oct 27 17:17 3
http://leakswldjpesnuvn.onion seems stable Oct 27 16:36 1
spot the spammer Oct 27 09:37 3
oniontkryve46opu.onion Oct 27 09:01 2
3 BM websites and all fucked Oct 26 21:00 12
Newcomer Oct 26 18:36 10
135453 Oct 25 22:06 1
Stay in touch Oct 25 13:06 1
new BM site online Oct 25 10:39 3
134730 Oct 25 09:59 1
BM is flatlining : https://beamstat.com/obj Oct 25 08:13 9
a new bitboard went online Oct 25 02:10 4
BM is flatlining : https://beamstat.com/obj Oct 25 00:23 1
sql Oct 24 22:44 1
how I hacked BM Oct 24 22:11 3
--curses mode with bitboard crashy Oct 24 21:30 5
BMF bug Oct 24 04:21 1
onion4442sx7tvvk.onion ONION 444 new website for BM ! hot shit ! Oct 24 04:21 5
post with \ backskash Oct 24 04:21 1
running pyBM as daemon on a remote server Oct 24 04:21 11
how I hacked BM Oct 24 04:17 3
BM is flatlining : https://beamstat.com/obj 1200 bytes the average object Oct 24 04:17 2
secret bin for Bitmessage people Oct 24 04:16 33
post with \ backskash Oct 24 04:11 1
anti-crash loop for BM Oct 22 06:53 2
actually, Oct 22 03:45 1
onion4442sx7tvvk.onion ONION 444 new website for BM ! hot shit ! Oct 21 21:49 1
magnet link publishing Oct 21 19:11 4
wanna hack a webserver ? free link here : http://nybarox.pythonanywhere.com Oct 21 07:16 17
cypherpunk Oct 21 06:54 5
leakswldjpesnuvn.onion relaunched and works like a charm ! Oct 20 22:49 1
leakswldjpesnuvn.onion relaunched and works like a charm ! Oct 20 20:44 1
new chan for BM site: http://leakswldjpesnuvn.onion/board/?chan=BM-2cVDWbAj3oftfGD1saBukfgGHDeUFKzNHc Oct 20 19:08 1
http://leakswldjpesnuvn.onion hot !!!! Oct 20 18:49 5
feature request Oct 20 08:04 3
http://leakswldjpesnuvn.onion Oct 20 04:36 1
new beamstat-like BM site online! read + write ! http://leakswldjpesnuvn.onion Oct 20 04:29 5
broadcast ===> BM-2cSmA3nNy2CnKN2Jmcexg6Eytgn9vLiDJg Oct 20 02:13 2
broadcast ===> BM-2cWPwaFc4LecJgQRfa4HHbC88yKxiUMKdv Oct 20 01:18 1
more badassy shit Oct 20 00:51 1
feature request Oct 19 23:28 1
badass shit Oct 19 21:14 1
bitboard thread Oct 19 21:14 17
working pic converter Oct 19 21:11 1
working pic converter Oct 19 19:20 5
badass shit Oct 19 17:43 1
wanna hack a webserver ? free link here : http://nybarox.pythonanywhere.com Oct 19 15:38 1
total badass shit Oct 19 14:56 2
working pic converer Oct 19 11:41 2
help make bm list Oct 19 11:24 4
justice being served , after all Oct 18 20:26 1
paste videos safely ! no install necessary Oct 18 18:44 1
I rented a 1 € v-server Oct 18 17:58 1
secret bin for Bitmessage people Oct 18 17:19 1
boring news Oct 18 17:10 1
bitmessage-address-lowercase.py Oct 18 16:35 13
babe Oct 18 16:29 1
bitboard as webserver like beamstat : super simple ! Oct 18 09:07 1
alex.grey-gaia-painting.10.jpg Oct 17 11:07 1
secret bin , no spam ! Oct 17 10:28 16
Wehrmacht: Trade weapons on OpenBazaar Oct 17 05:55 2
alchi - git via tor is easy ! Oct 17 05:04 2
boring news Oct 17 03:53 2
Wondering Oct 17 03:46 2
remove onion gateways Oct 17 03:35 4
is this satanic world worth living in ? Oct 17 02:53 1
Better than BM Oct 17 02:46 8