- rearranged code to reduce cyclic dependencies
- doCleanShutdown is separated in shutdown.py
- shared queues are separated in queues.py
- some default values were moved to defaults.py
- knownnodes partially moved to knownnodes.py
- complete the version and SSL handshake first, and only then feed
errors into the stream and close connection
- this allows more accurate error handling on both sides
- also the timeOffset error trigger is now more accurate, but requires
more nodes to upgrade
- version command sends list of all participating streams
- biginv sends lists of hosts for all streams the peer wants (plus
immediate children)
- objects will spread to all peers that advertise the associated stream
- please note these are just network subsystem adjustments, streams
aren't actually usable yet
- queues were too short
- some error handling was missing
- remove nonblocking repeats in receive data thread
- singleCleaner shouldn't wait unnecessarily
- Missing renamed to PendingDownload
- PendingDownload now only retries 3 times rather than 6 to dowload an
object
- Added PendingUpload, replacing invQueueSize
- PendingUpload has both the "len" method (number of objects not
uploaded) as well as "progress" method, which is a float from 0
(nothing done) to 1 (all uploaded) which considers not only objects
but also how many nodes they are uploaded to
- PendingUpload tracks when the object is successfully uploaded to the
remote node instead of just adding an arbitrary time after they have
been send the corresponding "inv"
- Network status tab's "Objects to be synced" shows the sum of
PendingUpload and PendingDownload sizes
- remember what was requested from which node
- remember if it was received
- re-request object if we haven't received any new object for more than
a minute
- tries to avoid calling senddata it it would block receiveDataThread,
allowing fore more asynchronous operation
- request objects in chunks of 100 (CPU performance optimisation)
- moved logic into a Missing singleton
- shouldn't try to download duplicates anymore, only requests a hash
once every 5 minutes and not from the same host
- removed obsoleted variables
- the "Objects to be synced" in the Network tab should now be correct
- removed some checks which aren't necessary anymore in my opinion
- fix missing self in Throttle (thanks landscape.io)
- send buffer to send multiple commands in one TCP packet
- recv/send operation size now based on bandwith limit
- send queue limited to 100 entries
- buffer getdata commands to fill send queue, instead of waiting for the
data packet to arrive first (i.e. allow getdata to work asynchronously)
- SSL handshake would often fail, because verack packet was being sent
at the same time as the do_handshake was executed in a different
thread. This makes it so that do_handshake waits until verack is done
sending.
- also minor modifications in SSLContext initialisation
- fixes errors introduced in the earlier refactoring
- more variables moved to state.py
- path finding functions moved to paths.py
- remembers IPv6 network unreachable (in the future can be used to skip
IPv6 for a while)
- got rid of shared config parser and made it into a singleton
- refactored safeConfigGetBoolean as a method of the config singleton
- refactored safeConfigGet as a method of the config singleton
- moved softwareVersion from shared.py into version.py
- moved some global variables from shared.py into state.py
- moved some protocol-specific functions from shared.py into protocol.py
- minor refactoring, made it into singleton instead of a shared global
variable. This makes it a little bit cleaner and moves the class into
a separate file
- removed duplicate inventory locking
- renamed singleton.py to singleinstance.py (this is the code that
ensures only one instance of PyBitmessage runs at the same time)
- TLS handshake in python is apparently always asynchronous, so it needs
proper handling of SSLWantReadError and SSLWantWriteError
- also adds a timeout and a proper shutdown if handshake fails
- sometimes SSL connections unnecessarily disconnected on non-fatal
errors. This should fix that. This is however a short term solution
because of migrating to asyncore which has its own error handling
- if your time is off by more than an hour, you won't be able to
establish a connection to the network. This patch adds a UI
notification so that the user can understand why he can't connect.
- will send the correct combination of hostname and port
- if proxyhostname is a hostname and an IP address, it will now allow
multiple parallel connections for hidden service
- PyBitmessage can now run as a hidden service on Tor
- three new variables in keys.dat: onionhostname, onionport, onionbindip
- you need to manually add a hidden service to tor
Attackers injected node addresses with port 0 into the network. Port 0
is unusable on many OSes and can't be listened on. PyBitmessage won't
accept nodes that have port 0 anymore.
- postpone initial sleep until the first getdata is received
- also sleep when received a getdata request for an object that hasn't
been advertised to the other node yet
There was a report that by quickly asking a large number of nodes if
they have an ACK object (which the attacker knows but it is injected
into the network by the recipient of the message), it can estimate how
an object propagates through the network, and eventually pinpoint an
originating IP address of the injection, i.e. the IP address of the
message recipient.
This patch mitigates against it by stalling when asked for a nonexisting
object (so that the attacker can't spam requests), and also upon
connection before sending its own inventory list (so that reconnecting
won't help the attacker). It estimates how long a short message takes to
propagate through the network based on how many nodes are in a stream
and bases the stalling time on that. Currently that is about 15 seconds.
Initial connection delay takes into account the time that already passed
since the connection was established.
This basically gives the attacker one shot per a combination of his own
nodes and the nodes he can connect to, and thus makes the attack much
more difficult to succeed.
When advertising nodes and when establishing connections, private IP
range checks were not done. This could cause private IPs to be
advertised across the network. Also, some of the checks weren't
IPv6-aware.
Fixes Bitmessage#768
Flood mitigation was done both in the ObjectProcessorQueue as well as
receiveData threads. This patch removes the mitigation in receiveData
threads and cleans up the one in the ObjectProcessorQueue
