Resend TTL and timing issues, misleading description #847

New Issue

2016-03-10T11:46:39+01:00

PeterSurda commented

2016-03-10 11:46:39 +01:00

(Migrated from github.com)

Sent over BM:

TTL is stored as the expiration date so it will have to be set to a new value which will affect the message payload checksum which will need a new PoW.

However, it is a lie what the client says, i.e. "By default, if you send a message to someone and he is offline for more than two days, Bitmessage will send the message again after an additional two days. This will be continued with exponential backoff forever; messages will be resent after 5, 10, 20 days ect. until the receiver acknowledges them.", let alone a sick delusional nonsense exhibited in the first quoted sentence.

In reality, the first resend is done once the original message expires. This first resend (and all the consecutive resends as well) will have TTL set to a maximum value, i.e. 28 days, (resulting in a more difficult PoW) and the next resend after that will be scheduled in 28 days * 2^currentresendnumber.

original send: time T, original TTL, resend scheduled in 28 days
first resend: time T+28 days, TTL=28 days, resend scheduled in 28 * 2^1 days (=56 days)
second resend: time T+28+56 days (=84 days), TTL=28 days, resend scheduled in 28 * 2^2 days (=112 days)
third resend: time T+28+56+112 days (=196 days), TTL=28 days, resend scheduled in 28 * 2^3 days (=224 days)
fourth resend: time T+28+56+112+224 days (=420 days), TTL=28 days, resend scheduled in 28 * 2^4 days (=448 days)
fifth resend: time T+28+56+112+224+448 days (=868 days), ...

So the message is not resent after 5, 10, 20 days as the client lies to you, but after 28, 84, 196 days. In other words, if a message is sent with the default 4 days TTL and the recipient happens to not run their BM client within this short period of time, they will have to wait for another 24 days to have a chance to receive your message. By that time the message may not be relevant to them anymore and if you deleted the message in the meantime it will never get resent at all.

So this feature is basically non-functional and useless for most of the users.

In the client it's stated that if the recipient isn't online for 2 days the client automatically resends the message. Will it have to calculate the PoW again? How is the TTL going to be accounted? From the new resend datetime?

Sent over BM: TTL is stored as the expiration date so it will have to be set to a new value which will affect the message payload checksum which will need a new PoW. However, it is a lie what the client says, i.e. "By default, if you send a message to someone and he is offline for more than two days, Bitmessage will send the message again after an additional two days. This will be continued with exponential backoff forever; messages will be resent after 5, 10, 20 days ect. until the receiver acknowledges them.", let alone a sick delusional nonsense exhibited in the first quoted sentence. In reality, the first resend is done once the original message expires. This first resend (and all the consecutive resends as well) will have TTL set to a maximum value, i.e. 28 days, (resulting in a more difficult PoW) and the next resend after that will be scheduled in 28 days \* 2^currentresendnumber. - original send: time T, original TTL, resend scheduled in 28 days - first resend: time T+28 days, TTL=28 days, resend scheduled in 28 \* 2^1 days (=56 days) - second resend: time T+28+56 days (=84 days), TTL=28 days, resend scheduled in 28 \* 2^2 days (=112 days) - third resend: time T+28+56+112 days (=196 days), TTL=28 days, resend scheduled in 28 \* 2^3 days (=224 days) - fourth resend: time T+28+56+112+224 days (=420 days), TTL=28 days, resend scheduled in 28 \* 2^4 days (=448 days) - fifth resend: time T+28+56+112+224+448 days (=868 days), ... So the message is not resent after 5, 10, 20 days as the client lies to you, but after 28, 84, 196 days. In other words, if a message is sent with the default 4 days TTL and the recipient happens to not run their BM client within this short period of time, they will have to wait for another 24 days to have a chance to receive your message. By that time the message may not be relevant to them anymore and if you deleted the message in the meantime it will never get resent at all. So this feature is basically non-functional and useless for most of the users. --- In the client it's stated that if the recipient isn't online for 2 days the client automatically resends the message. Will it have to calculate the PoW again? How is the TTL going to be accounted? From the new resend datetime?

infinity0 commented

2016-04-14 21:05:57 +02:00

(Migrated from github.com)

There's some more improvements to be made here:

it should be possible for the sender to start a manual resend, if he notices that a message is not yet acknowledged.
The algorithm described is not actually exponential backoff. In actual exponential backoff, one does not wait exactly (k * a ^ n) days for n = {0,1,2...} but a random number between 0 and (k * a ^ n). In other words on the 3rd retry, it is still possible to resend after a delay of 0. Furthermore, it is debatable whether exponential backoff is even appropriate here - perhaps simply resending every 2 days would be better. Exponential backoff is for congestion avoidance, and it's not clear that this applies to the bitmessage delivery model.
It should be possible for the recipient to notice that he hasn't received some messages. At the very least, some UI warning saying "You were offline for X days, and might have missed messages. To be sure you received all messages, you need to be continually online every 2 days for the next Y days".

There's some more improvements to be made here: 1. it should be possible for the sender to start a manual resend, if he notices that a message is not yet acknowledged. 2. The algorithm described is not actually [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff). In actual exponential backoff, one does not wait exactly (k \* a ^ n) days for n = {0,1,2...} but a random number between 0 and (k \* a ^ n). In other words on the 3rd retry, it is still possible to resend after a delay of 0. Furthermore, it is debatable whether exponential backoff is even appropriate here - perhaps simply resending every 2 days would be better. Exponential backoff is for congestion avoidance, and it's not clear that this applies to the bitmessage delivery model. 3. It should be possible for the recipient to notice that he hasn't received some messages. At the very least, some UI warning saying "You were offline for X days, and might have missed messages. To be sure you received all messages, you need to be continually online every 2 days for the next Y days".

👍 1

PeterSurda commented

2016-04-15 14:29:58 +02:00

(Migrated from github.com)

There appear to be several separate isuses here:

the description in the UI does not match the behaviour of PyBitmessage
the behaviour of PyBitmessage retransmits can be changed
the behaviour of PyBitmessage reception can be changed

I personally think that:

the description should be changed
the retransmits can also be changed, but there shouldn't be gaps like they are now
notification of the recipient is also a good idea, because messages that do not include an ACK (chans or recipient has ACKs disabled) are not retransmitted at all

There appear to be several separate isuses here: - the description in the UI does not match the behaviour of PyBitmessage - the behaviour of PyBitmessage retransmits can be changed - the behaviour of PyBitmessage reception can be changed I personally think that: - the description should be changed - the retransmits can also be changed, but there shouldn't be gaps like they are now - notification of the recipient is also a good idea, because messages that do not include an ACK (chans or recipient has ACKs disabled) are not retransmitted at all

PeterSurda commented

2017-02-25 23:43:09 +01:00

(Migrated from github.com)

Now the TTL grows exponentially, and it waits 10% of the TTL past the expiration before retransmitting. So it's still not exactly "exponential backoff", but it's more exponential than before, and it matches the description more accurately.

PeterSurda commented

2020-09-28 09:51:36 +02:00

(Migrated from github.com)

I think it's good enough now, closing, if anyone needs more changes, you can reopen.

This repo is archived. You cannot comment on issues.