Moved config directory and now receiving duplicates #801

New Issue

2015-06-18T08:44:16+02:00

sigmike commented

2015-06-18 08:44:16 +02:00

(Migrated from github.com)

Hello,

A few weeks ago I moved my .config/PyBitmessage to another computer, ran bitmessage from there for a few days and moved it back to its original place. Everything works fine except since then I receive many messages I've already received.

I think I only receive duplicates of messages that first arrived before I made this config switch, but I'm not sure.

It's different from #611 because the duplicates are still there after I restart bitmesssage.

I'm running current master.

Hello, A few weeks ago I moved my `.config/PyBitmessage` to another computer, ran bitmessage from there for a few days and moved it back to its original place. Everything works fine except since then I receive many messages I've already received. I think I only receive duplicates of messages that first arrived before I made this config switch, but I'm not sure. It's different from #611 because the duplicates are still there after I restart bitmesssage. I'm running current master.

Atheros1 commented

2015-06-19 19:50:28 +02:00

(Migrated from github.com)

You have received the duplicates on the same computer? (As opposed to receiving the same message on two different computers?)

If the sender resends the message then this can happen with the current code. The current code no longer removes duplicates based on the [sender, receiver, subject, message] combination but rather using a hash of the signature (so it is basically a random ID). If the sender resends a message before the previous message expires for some reason, or if you change your client to not send acknowledgements, then you can receive duplicates. I'm not really sure why this would happen with an unmodified client.

You have received the duplicates on the same computer? (As opposed to receiving the same message on two different computers?) If the sender resends the message then this can happen with the current code. The current code no longer removes duplicates based on the [sender, receiver, subject, message] combination but rather using a hash of the signature (so it is basically a random ID). If the sender resends a message before the previous message expires for some reason, or if you change your client to not send acknowledgements, then you can receive duplicates. I'm not really sure why this would happen with an unmodified client.

sigmike commented

2015-06-19 22:41:48 +02:00

(Migrated from github.com)

You have received the duplicates on the same computer?

Yes. I received some of them 3 or 4 times.

The current code no longer removes duplicates based on the [sender, receiver, subject, message] combination but rather using a hash of the signature (so it is basically a random ID).

I guess it's the sighash in the inbox table. The duplicate messages do have different sighash. Actually, one of them has an empty sighash. Is that normal? (It's only on one case though. There are duplicates that all have a non blank sighash)

If the sender resends a message before the previous message expires for some reason, or if you change your client to not send acknowledgements, then you can receive duplicates. I'm not really sure why this would happen with an unmodified client.

So these duplicates could mean the sender didn't receive my acknowledgments for some reason?

> You have received the duplicates on the same computer? Yes. I received some of them 3 or 4 times. > The current code no longer removes duplicates based on the [sender, receiver, subject, message] combination but rather using a hash of the signature (so it is basically a random ID). I guess it's the sighash in the inbox table. The duplicate messages do have different sighash. Actually, one of them has an empty sighash. Is that normal? (It's only on one case though. There are duplicates that all have a non blank sighash) > If the sender resends a message before the previous message expires for some reason, or if you change your client to not send acknowledgements, then you can receive duplicates. I'm not really sure why this would happen with an unmodified client. So these duplicates could mean the sender didn't receive my acknowledgments for some reason?

Atheros1 commented

2015-06-22 21:06:46 +02:00

(Migrated from github.com)

Yes, it could mean that the sender did not receive the acknowledgements.

The empty sighash could be a message that you received before you upgraded your software.

Yes, it could mean that the sender did not receive the acknowledgements. The empty sighash could be a message that you received before you upgraded your software.

sigmike commented

2015-07-03 08:18:48 +02:00

(Migrated from github.com)

I got the confirmation the other end is using a standard PyBitMessage and has some messages that haven't received the acknowledgment. I keep receiving duplicates, including on very recent messages. I also received one duplicate from someone else, so I guess it's not a problem with the other end. Is it possible my acknowledgments are invalid somehow? How can I check that?

PeterSurda commented

2015-11-11 09:35:08 +01:00

(Migrated from github.com)

@sigmike I will try to reproduce it. Are you available for helping me debug it?

sigmike commented

2015-11-11 09:42:24 +01:00

(Migrated from github.com)

Yes. The issue still occurs.

PeterSurda commented

2015-11-11 09:45:02 +01:00

(Migrated from github.com)

Based on the config file location, it looks like you're using a Unix-like system other than OSX (probably linux), correct?

sigmike commented

2015-11-11 09:46:58 +01:00

(Migrated from github.com)

Yes, Linux.

PeterSurda commented

2015-11-11 09:53:22 +01:00

(Migrated from github.com)

Which version are you using: the latest stable (0.4.4), the latest master, or perhaps my fork (https://github.com/mailchuck/PyBitmessage)?

sigmike commented

2015-11-11 09:57:45 +01:00

(Migrated from github.com)

Master a little before the time I created the issue. I'm at b3afbb4308.

Master a little before the time I created the issue. I'm at b3afbb43086a1c96b620677fc9f15ed35322becb.

PeterSurda commented

2015-11-11 09:59:46 +01:00

(Migrated from github.com)

@sigmike Also, what version is the other side (the person who says they haven't received the acknowledgement) use? You write they use "standard PyBitMessage", is that 0.4.4 or latest master? The duplication fixes @Atheros1 made were done post-0.4.4, but I always thought that it is the recipient that should upgrade, the sender doesn't have to.

sigmike commented

2015-11-11 10:01:28 +01:00

(Migrated from github.com)

The other side is using 0.4.4.

PeterSurda commented

2015-11-11 11:01:40 +01:00

(Migrated from github.com)

I looked at the code and I have a theory about what happened. If you received the messages and shut down PyBitmessage before the acknowledgements were actually broadcasted (or, the recipient was offline for too long), it looks like it could have the effect you describe. Your client would think that it already sent the acks, and would not send them again if it received the message again.

To test this theory, what you can do is shutdown PyBitmessage (your, not the senders) and clean the inventory table:

sqlite3 ~/.config/PyBitmessage/messages.dat
delete from inventory;
.quit

That deletes the local copy of the objects on the P2P network and fetches them again. It would also cause PyBitmessage to create new acks when it receives messages which have you as a recipient. You still may receive one final duplicate, but this time you'll generate the ack and the recipient should receive it.

Other than causing you to redownload stuff and requiring a bit more disk space until the next vacuum, I don't think this has adverse effects.

I looked at the code and I have a theory about what happened. If you received the messages and shut down PyBitmessage before the acknowledgements were actually broadcasted (or, the recipient was offline for too long), it looks like it could have the effect you describe. Your client would think that it already sent the acks, and would not send them again if it received the message again. To test this theory, what you can do is shutdown PyBitmessage (your, not the senders) and clean the inventory table: ``` sqlite3 ~/.config/PyBitmessage/messages.dat delete from inventory; .quit ``` That deletes the local copy of the objects on the P2P network and fetches them again. It would also cause PyBitmessage to create new acks when it receives messages which have you as a recipient. You still may receive one final duplicate, but this time you'll generate the ack and the recipient should receive it. Other than causing you to redownload stuff and requiring a bit more disk space until the next vacuum, I don't think this has adverse effects.

sigmike commented

2015-11-11 11:23:32 +01:00

(Migrated from github.com)

Ok I'll try that.

But note that this is happening on multiple messages sent at varying interval. So it seems unlikely I closed the client between the message reception and ack broadcast that often.

I'm also pretty confident the other side very rarely lets the software closed for more than one day.

Ok I'll try that. But note that this is happening on multiple messages sent at varying interval. So it seems unlikely I closed the client between the message reception and ack broadcast that often. I'm also pretty confident the other side very rarely lets the software closed for more than one day.

PeterSurda commented

2015-11-11 12:53:40 +01:00

(Migrated from github.com)

I tried it just to make sure it doesn't break anything, and I could not notice any adverse effect.

PeterSurda commented

2015-11-11 13:08:52 +01:00

(Migrated from github.com)

How about you ask them to send messages to me? This is the address I use for testing: BM-2cWuABX2d7XEPzxNmFb6wBedZ6zbW6Evgh

PeterSurda commented

2015-11-21 10:36:14 +01:00

(Migrated from github.com)

It was pointed out to me that ACK messages have a TTL of 1 hour (I haven't checked it out myself yet though). This would explain the type of behaviour you're experiencing.

Try to make sure both of you are online at the same time (not in order to fix it, but in order to seei if you can still reproduce it).

It was pointed out to me that ACK messages have a TTL of 1 hour (I haven't checked it out myself yet though). This would explain the type of behaviour you're experiencing. Try to make sure both of you are online at the same time (not in order to fix it, but in order to seei if you can still reproduce it).

PeterSurda commented

2016-03-01 03:25:27 +01:00

(Migrated from github.com)

The minimum ACK TTL is not one hour, rather 1 day. In 0.4.4, it was fixed at 2.5 days.

I can't reproduce this.

I recommend that you all upgrade to a version of post-TTL (e.g. latest official master or the mailchuck fork) instead of 0.4.4 and make sure that the client syncs at least once a day. The mailchuck fork has faster message downloading and a status indicator showing if it's synced up.

The minimum ACK TTL is not one hour, rather 1 day. In 0.4.4, it was fixed at 2.5 days. I can't reproduce this. I recommend that you all upgrade to a version of post-TTL (e.g. latest official master or the mailchuck fork) instead of 0.4.4 and make sure that the client syncs at least once a day. The mailchuck fork has faster message downloading and a status indicator showing if it's synced up.

This repo is archived. You cannot comment on issues.