Additional services #772

New Issue

2015-02-01T16:37:40+01:00

Kagami commented

2015-02-01 16:37:40 +01:00

(Migrated from github.com)

Hi.

I'd like to introduce several new node types compatible with the Bitmessage protocol: NODE_MOBILE and NODE_GATEWAY. They are almost identical to the nodes described here: https://bitmessage.org/wiki/Mobile_Protocol_specification

But several PyBitmessage quirks stop me:

PyBitmessage doesn't check services bitfield at all. This means that if I advertise node which uses non-TCP transport with services = 4 (BE value), PyBitmessage will constantly try to connect to it despite the fact the node hasn't NODE_NETWORK bit.
~~I can't advertise NODE_NETWORK and NODE_GATEWAY node as two different addr entries with the same IP because knownNodes structure is a host-ip/host-details map.~~ Oops, I missed the fact knownNodes is a host+port/host-details map now.
Even if I advertise NODE_GATEWAY as a separate addr entry, PyBitmessage won't be able to connect to it anyway so NODE_GATEWAY addresses will propogate badly through the network because of the node alive checkings: https://github.com/Bitmessage/PyBitmessage/blob/master/src/class_receiveDataThread.py#L666
PyBitmessage currently doesn't take into account services field when it packs the addr message: https://github.com/Bitmessage/PyBitmessage/blob/master/src/class_receiveDataThread.py#L672, https://github.com/Bitmessage/PyBitmessage/blob/master/src/class_receiveDataThread.py#L275

So far I know one rather awkward solution: advertise NODE_NETWORK and NODE_GATEWAY fields (i.e. services = 5) as single addr entry and use hardcoded port for non-TCP transports (e.g. 18444) but this is rather tricky.

Hi. I'd like to introduce several new node types compatible with the Bitmessage protocol: `NODE_MOBILE` and `NODE_GATEWAY`. They are almost identical to the nodes described here: https://bitmessage.org/wiki/Mobile_Protocol_specification But several `PyBitmessage` quirks stop me: - `PyBitmessage` doesn't check `services` bitfield at all. This means that if I advertise node which uses non-TCP transport with `services = 4` (BE value), `PyBitmessage` will constantly try to connect to it despite the fact the node hasn't `NODE_NETWORK` bit. - ~~I can't advertise `NODE_NETWORK` and `NODE_GATEWAY` node as two different `addr` entries with the same IP because `knownNodes` structure is a host-ip/host-details map.~~ Oops, I missed the fact `knownNodes` is a host+port/host-details map now. - Even if I advertise `NODE_GATEWAY` as a separate `addr` entry, PyBitmessage won't be able to connect to it anyway so `NODE_GATEWAY` addresses will propogate badly through the network because of the node alive checkings: https://github.com/Bitmessage/PyBitmessage/blob/master/src/class_receiveDataThread.py#L666 - `PyBitmessage` currently doesn't take into account `services` field when it packs the `addr` message: https://github.com/Bitmessage/PyBitmessage/blob/master/src/class_receiveDataThread.py#L672, https://github.com/Bitmessage/PyBitmessage/blob/master/src/class_receiveDataThread.py#L275 So far I know one rather awkward solution: advertise `NODE_NETWORK` and `NODE_GATEWAY` fields (i.e. `services = 5`) as single `addr` entry and use hardcoded port for non-TCP transports (e.g. 18444) but this is rather tricky.

Kagami commented

2015-02-01 16:53:12 +01:00

(Migrated from github.com)

Sorry, I've updated the issue with several details.

Kagami commented

2015-02-18 16:36:41 +01:00

(Migrated from github.com)

Also, node services data isn't saved to the disk. It was saved before inside the knownnodes[1] table records but not anymore with knownnodes.dat.

  CREATE TABLE knownnodes (
    timelastseen int,
    stream int,
    services blob,
    host blob,
    port blob,
    UNIQUE(host, stream, port) ON CONFLICT REPLACE
)

Also, node services data isn't saved to the disk. It was saved before inside the `knownnodes`[1] table records but not anymore with `knownnodes.dat`. 1. ``` sql CREATE TABLE knownnodes ( timelastseen int, stream int, services blob, host blob, port blob, UNIQUE(host, stream, port) ON CONFLICT REPLACE ) ```

Kagami commented

2015-02-23 14:09:16 +01:00

(Migrated from github.com)

So, I have two variants how it can be fixed:

PyBitmessage should store net_addr even if it hasn't NODE_NETWORK bit. And it shouldn't connect to the nodes without NODE_NETWORK bit.
PyBitmessage shouldn't store net_addrs without NODE_NETWORK bit.

Each variant has it's own disadvantages:

In the second case, we need to hardcode NODE_GATEWAY port because there is no other place where it can be stored (except the higher bits of the services bitfield itself but it's too hackish).
But in the first case it's very easy to spam knownnodes because common node cannot verify availability of the node without NODE_NETWORK bit.

So this is the plan how the second variant can be implemented:

PyBitmessage should store services field of the net_addr (it would require to change knownnodes format, but it's not too hard)
PyBitmessage should also properly advertise services field loaded from knownnodes.dat
PyBitmessage should discard net_addrs without NODE_NETWORK bit

@Atheros1, do you have any objections/suggestions? I think I can make PR for this in 1-2 months.

So, I have two variants how it can be fixed: - PyBitmessage should store net_addr even if it hasn't NODE_NETWORK bit. And it shouldn't connect to the nodes without NODE_NETWORK bit. - PyBitmessage shouldn't store net_addrs without NODE_NETWORK bit. Each variant has it's own disadvantages: - In the second case, we need to hardcode NODE_GATEWAY port because there is no other place where it can be stored (except the higher bits of the services bitfield itself but it's too hackish). - But in the first case it's very easy to spam knownnodes because common node cannot verify availability of the node without NODE_NETWORK bit. So this is the plan how the second variant can be implemented: - PyBitmessage should store services field of the net_addr (it would require to change knownnodes format, but it's not too hard) - PyBitmessage should also properly advertise services field loaded from knownnodes.dat - PyBitmessage should discard net_addrs without NODE_NETWORK bit @Atheros1, do you have any objections/suggestions? I think I can make PR for this in 1-2 months.

Kagami commented

2015-02-23 14:24:19 +01:00

(Migrated from github.com)

Though, the sad part is: nodes with special services still won't be propogated fast because PyBitmessage won't give advantage to them when sending big addr message. So the alternative approaches are:

Use own set of bootstrap nodes which would propogate only nodes with NODE_GATEWAY services (equivalent to forking the Bitmessage network)
Broadcast objects encrypted with well-known keys which contain NODE_GATEWAY net_addrs (even more hackish)
Somehow say to the PyBitmessage via version exchange that we are only interested in NODE_GATEWAY nodes

Though, the sad part is: nodes with special services still won't be propogated fast because PyBitmessage won't give advantage to them when sending big addr message. So the alternative approaches are: - Use own set of bootstrap nodes which would propogate only nodes with NODE_GATEWAY services (equivalent to forking the Bitmessage network) - Broadcast objects encrypted with well-known keys which contain NODE_GATEWAY net_addrs (even more hackish) - Somehow say to the PyBitmessage via version exchange that we are only interested in NODE_GATEWAY nodes

Kagami commented

2015-03-03 19:30:48 +01:00

(Migrated from github.com)

Also, the interesting question: what if some node will lie about net_addr's services and constantly rebroadcast wrong values? How are we going to detect and ban such node?

This repo is archived. You cannot comment on issues.

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Bitmessage/PyBitmessage-2025-02-26#772