pickle puzzle

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 11 18:49 [raw]

I tried to unpickle knownnodes.dat, remove all the non-onion entries, then repickle it so that bitmessage will load it properly. I tried with standard pickle commands, edited the json output, but when I repickle it for some reason bitmessage does not recognize it and just erases the file and starts over on startup with the default bootstrap entries. Bitmessage is not able to load the edited pickle. Does any python guru care to lend a hand at what I need to investigate to get the desired result?

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 11 18:59 [raw]

for what?

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 11 19:17 [raw]

the original post already answered this question

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 11 19:29 [raw]

It didn't answer WHY you want to remove the non-onion entries. Especially since BitMessage is just going to get them from the connected nodes again anyway.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 11 19:36 [raw]

Exactly. Why someone wants to delete non-onion entries?

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 11 23:34 [raw]

Add to torrc: SocksPort 127.0.0.1:9151 IsolateDestAddr OnionTrafficOnly Then configure bitmessage to use that new port 9151 and it will only connect to onion nodes. Optionally you can delete knownnodes.dat and add to keys.dat: [knownnodes] maxnodes = 1000 This will reduce the number of nodes in knownnodes to 1000, retreive the freshest onion nodes, and connect to those onion nodes them faster. Don't set maxnodes too low or you may never get more than the default boostrap onion node. If you lower maxnodes and don't delete knownnodes.dat, you may end up with 1000 non-onion nodes and never connect at all.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 00:12 [raw]

import argparse import imp import sys try: import cPickle as pickle except ImportError: import pickle # Dynamically generate and import stripped down 'state' module containing just # the Peer namedtuple to unpickle knownnodes successfully and output a pickle # data stream PyBitmessage understands module_name = 'state' module_code = '''from collections import namedtuple Peer = namedtuple('Peer', ('host', 'port')) ''' module = imp.new_module(module_name) exec module_code in module.__dict__ sys.modules[module_name] = module parser = argparse.ArgumentParser() parser.add_argument('knownnodes', type=argparse.FileType('rb')) args = parser.parse_args() streams = {} for stream, nodes in pickle.load(args.knownnodes).iteritems(): streams[stream] = {peer: attrs for peer, attrs in nodes.iteritems() if peer.host[-6:] == '.onion'} pickle.dump(streams, sys.stdout)

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 04:19 [raw]

In much older threads, some have worried that malware could overwrite knownnodes.dat with malicious bootstrap entries. I don't recall the results of the threads, but erasing the file and starting over with the default bootstrap entries would be a good precaution for bitmessage to perform in that situation.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 05:16 [raw]

If malware is able to modify knownnodes.dat, I'd be more worried about an exploit than malicious bootstrap nodes.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 06:29 [raw]

WHY? none of your business, that's why.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 08:16 [raw]

Add to torrc: SocksPort 127.0.0.1:9151 IsolateDestAddr OnionTrafficOnly Then configure bitmessage to use that new port 9151 and it will only connect to onion nodes. Optionally you can delete keys.dat and add to knownnodes.dat: [knownnodes] maxnodes = 1000 This will reduce the number of nodes in knownnodes to 1000, retreive the freshest onion nodes, and connect to those onion nodes them faster. Don't set maxnodes too low or you may never get more than the default boostrap onion node. If you lower maxnodes and don't delete keys.dat, you may end up with 1000 non-onion nodes and never connect at all.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 11:02 [raw]

I rate it an eight, mate.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 11:03 [raw]

gr8 b8, m8, I r8 8/8

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 16:15 [raw]

You think you have superior rationale. The subject is not malware. The subject is how to load the pickle with onion only nodes. It doesn't matter why.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 16:15 [raw]

If I were malware I would send me your keys.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 16:15 [raw]

this is a clever workaround

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 16:18 [raw]

> exec module_code in module.__dict__ dangerous vector for injection

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 12 21:23 [raw]

Not really. An attacker has to be able to modify memory or the file. exec or no exec, an attacker can do whatever they want if they can modify memory or the file.

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 13 00:04 [raw]

from argparse import ArgumentParser, FileType from collections import namedtuple from copy_reg import _reconstructor Peer = namedtuple('Peer', ('host', 'port')) safe_class_map = { 'copy_reg._reconstructor': _reconstructor, '__builtin__.tuple': tuple, 'state.Peer': Peer, } def get_class(module, name): fullname = '%s.%s' % (module, name) klass = safe_class_map.get(fullname, None) if not klass: raise UnpicklingError('Unpickling %s not supported' % fullname) return klass parser = ArgumentParser(description='Remove non-onion entries from knownnodes') parser.add_argument('input', nargs='?', default='-', type=FileType('rb'), help='Path of knownnodes.dat to filter or - for stdin (default: stdin)', metavar='in_knownnodes') parser.add_argument('output', nargs='?', default='-', type=FileType('wb'), help='Path to save filtered knownnodes or - for stdout (default: stdout)', metavar='out_knownnodes') args = parser.parse_args() try: import cPickle as pickle unpickler = pickle.Unpickler(args.input) unpickler.find_global = get_class except ImportError: import pickle class Unpickler(pickle.Unpickler): def find_class(self, module, name): return get_class(module, name) unpickler = Unpickler(args.input) streams = {} for stream, nodes in unpickler.load().iteritems(): streams[stream] = {peer: attrs for peer, attrs in nodes.iteritems() if peer.host[-6:] == '.onion'} args.output.write(pickle.dumps(streams).replace('__main__', 'state', 1))

BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY
Feb 13 01:03 [raw]

slick work

[chan] bitmessage
BM-2cWy7cvHoq3f1rYMerRJp8PT653jjSuEdY

Subject Last Count
End of support for Windows XP for binary builds Feb 15 17:46 1
how to use mailing list...? Feb 13 09:39 3
UK Column News - 11 February 2019 Feb 10 11:07 5
come on guys, leak some more shitwarez Feb 10 07:28 14
DJ Bernstein sightings on Bitmessage Feb 10 06:57 3
UK Column News - February 12 2019 Feb 9 21:19 1
UK Column News - February 12th 2019 Feb 9 21:19 1
UK Column News - 12th February 2019 Feb 9 21:16 1
UK Column News - 11th February 2019 Feb 9 21:14 1
UK Column News - 9th February 2019 Feb 9 21:13 1
UK Column News - February 2019 7th Feb 7 07:45 2
UK Column News - 7 2019 February Feb 7 07:42 1
UK Column News - 2019 February 7th Feb 7 07:40 2
UK Column News - February 7th 2019 Feb 7 07:37 2
UK Column News - 2019 February 7 Feb 7 07:35 2
UK Column News - February 7 2019 Feb 7 07:29 1
UK Column News - 7th February 2019 Feb 7 07:26 3
UK Column News - 7 February 2019 Feb 7 07:25 1
UK Column News - 6th February 2019 Feb 2 15:57 3
UK Column News - 5th February 2019 Feb 2 15:57 4
UK Column News - 4th February 2019 Feb 2 15:57 5
what does dandelion: 90 do? Feb 1 11:42 7
stop test penis, please. it's OK Jan 30 09:39 4
Call to murder Angela Merkel, Emmanuel Macron, Petro Poroshenko, Jens Stoltenberg etc. Jan 27 21:49 1
dammit ! dang nigger pranked Dr. David Duke Jan 27 19:37 2
djurlite enacting Jan 27 00:00 1
Reversed shot upper value Jan 26 23:59 1
Normal drilling mud circulation buffer gas Jan 26 22:18 1
Power monitor homotopy boundary Jan 26 21:25 1
Pelerine point subtract counter Jan 26 21:25 1
Teeth misalignment country setting Jan 26 21:24 1
Crankous jam radio station Jan 26 21:23 1
Older the hyperarial Jan 26 21:23 1
Defects survey positive muon Jan 26 21:23 1
extrusion nozzle methanol treatment Jan 26 21:23 1
Townships hearth gas Jan 26 21:23 1
Transversal equalizer on pentalpha Jan 26 21:18 1
Salmoncoloured obtain circuit Jan 26 21:18 1
serializer firm support Jan 26 21:18 1
depredation for petroleum series Jan 26 21:11 1
Plotting camera the reeving system Jan 26 21:06 1
Conventional weapons for jack bar assembly Jan 26 20:59 1
operationally ready well sinking Jan 26 20:59 1
Tympan franzise Jan 26 20:58 1
Equipment status chart with frequency sounding Jan 26 20:58 1
Difference construction the alette Jan 26 20:52 1
Vitality rotten Jan 26 20:51 1
Multiloquence progressive fracture Jan 26 20:50 1
automatic backspace assemble editing continuous decomposition Jan 26 20:47 1
Summer oil level platy Jan 26 20:43 1
Approximative limit paramour Jan 26 20:43 1
Card file beddable Jan 26 20:38 1
Damage accumulation then hot leveling Jan 26 20:38 1
Frequency analysis method headless resistor Jan 26 20:38 1
Trustor with grounded sea ice Jan 26 20:38 1
Roundsman the outweigh a disadvantage Jan 26 20:38 1
Military law forest shelter belt Jan 26 20:38 1
tunnel cathode bring in evidence Jan 26 20:27 1
Vacuum melted alloy job control program Jan 26 20:19 1
Duplicate insulator string nuclear magnetic resonance log Jan 26 20:19 1
Linear parameter the underinvoicing Jan 26 20:19 1
Namesake oxygenated oil Jan 26 20:19 1
Echo chamber positive function Jan 26 20:19 1
Plasma belt amoebosis Jan 26 20:18 1
Local optimization the equicontinuous group Jan 26 20:18 1
Film cartridge resign management Jan 26 20:18 1
Approximate root hereditaments Jan 26 20:11 1
Peppering loop body Jan 26 20:05 1
Winged hollow reamer limiting formation factor Jan 26 20:01 1
Bottom cut on activated fins Jan 26 19:59 1
Paradox of thrift impenetrable Jan 26 19:58 1
delay decision fluidized bed Jan 26 19:58 1
Wall bushing hygienic enamel Jan 26 19:57 1
Wellmannered the mesic Jan 26 19:56 1
Incommunicative the waste rock Jan 26 19:56 1
Unloading operation the upper girth Jan 26 19:55 1
Shopwindow marlstone limestone Jan 26 19:55 1
Rotary bed the noncyclic trajectory Jan 26 19:55 1
Release labour the finance plan Jan 26 19:55 1
Dunst flange groove Jan 26 19:55 1
detrucking point radicals Jan 26 19:55 1
Razor obligation to notify Jan 26 19:50 1
Aberrant behaviour nearshore current Jan 26 19:49 1
Apprehend integration operator Jan 26 19:49 1
Chase all fear correct the compass Jan 26 19:45 1
initiating terminal the deck slab Jan 26 19:42 1
trailed extended graph Jan 26 19:41 1
Boom tip general appraiser Jan 26 19:41 1
Decimal representation reduction room Jan 26 19:36 1
Passband edge true amplitude section stack Jan 26 19:32 1
Cross correlation commove Jan 26 19:29 1
Observing robot diseconomies of scale Jan 26 19:27 1
Offset ink graphics formatter Jan 26 19:27 1
Sound rocket dielectric oil Jan 26 19:24 1
Storm rail for aphasic Jan 26 19:21 1
Top dressing algebra manifold Jan 26 19:20 1
Open crankcase compressor the interfacial slip Jan 26 19:18 1
underground survey for asphalting Jan 26 19:02 1
Wegotism multilinear Jan 26 19:02 1
cock a pistol squeezability Jan 26 19:02 1