Commit Graph

42 Commits

Author SHA1 Message Date
lakshyacis
81872c7f2f
network code quality fixes 2020-01-08 12:53:04 +05:30
a69732f060
Addrthread finish
- addrthread is supposed to spread addresses as they appear. This was never
  finished during migration to asyncore
- conservative to prevent flood and loops
- randomises order
- move protocol constants into a separate file
- move addr packet creation into a separate file
- see #1575
2019-11-30 13:47:24 +01:00
2a165380bb
Restrict outbound connections on network groups
Logic borrowed from bitcoin, see CNetAddr::GetGroup() in src/netaddress.cpp
Simplified, so may not work fully identically but for our purposes it's good
enough. Won't connect to more than one host from a /16 subnet on IPv4 and a /32
subnet on IPv6.
2019-11-18 12:20:29 +01:00
d6c1845b71
Moved Peer from state to network.node
and trustedPeer to network.connectionpool.BMConnectionPool attribute
2019-11-11 17:13:12 +02:00
bbdbca253b
Added warnings about changing port settings in api and network.tcp 2019-10-18 09:35:31 +03:00
7a89109fc9
New logging approach in order to reduce imports from submodules
and use logging without risk of circular import. Only subpackage
that imports from debug is bitmessageqt - because it also uses
debug.resetLogging().
Instead of from debug import logger is now recommended to use:

import logging

logger = logging.getLogger('default')

All subclasses of StoppableThread now have a logger attribute.
All threading related stuff except for set_thread_name()
was moved from helper_threading to network.threads.

Fixed two my mistakes from previous edit of debug in a1a8d3a:

 - logger.handlers is not dict but iterable
 - sys.excepthook should be set unconditionally
2019-10-18 09:35:24 +03:00
4825c5a136
Universal bootstrap procedure for any connection type 2019-09-25 18:55:01 +03:00
0a06567071
Connect to bootstrap nodes by name 2019-09-25 18:55:01 +03:00
bbab0010e6
Moved a test script from network.tcp to tests.core 2019-07-26 12:08:42 +03:00
996e71ae6f
flake8 in network package 2019-07-26 12:07:57 +03:00
453e045ae5
Minimal implementation of onionpeer object 2019-05-10 12:24:41 +03:00
e25fb857cb
Slightly reduced TCPConnection.sendAddr() and changed
in order to send only nodes with non-negative rating
2019-02-07 18:52:08 +02:00
coffeedogs
0bceeca524
Changes based on style and lint checks. (final_code_quality_5) 2018-10-31 14:04:35 +00:00
coffeedogs
02f40f5036
Fixed: ERROR - Too many items in inv message! Closes 1274 2018-06-14 11:57:04 +01:00
0c9cb4824d
Memory leak (kind of) fix
- objectsKnownToThem is supposed to track if it's necessary to send inv
  commands to a peer
- it is supposed to enter garbage collection after 1 hour
  (ObjectTracker.trackingExpires)
- due to peer not announcing all objects, or announcing them before we
  announce, this contains excessive number of entries after connection
  is fully established.
- profiling revealed that this creates unnecessary memory to be kept
  allocated
- this patch will prevent tracking of entries sent during bigInv,
  reducing the memory "leak"
- it is possible, in theory, that this does have some negative effect,
  like increased bandwidth or neglecting to announce some invs. It
  probably doesn't though as my review of objectsKnownToThem occurrences
  didn't reveal any such case, and since the dict didn't track fully
  accurately anyway (so it would have already been broken if it was a
  problem), I consider it an acceptable risk at the moment. If it indeed
  causes problems, they can be solved separately
- I tested this on one of the bootstrap servers with little memory, and
  it increased the number of connections than can be handled by a factor
  of about 3.5
2018-04-01 11:39:02 +02:00
Mahendra
1a36361ef9
replace all random.xxx with helper random function 2018-03-21 19:26:27 +05:30
fd1a6c1fa1
Dandelion update
- dandelion fixes
- try to wait as long as possible before expiration if there are no
outbound connections
- expire in invThread rather than singleCleaner thread
- deduplication of code in inv and dinv command methods
- turn on by default, seems to work correctly now
- turn off dandelion if outbound connections are disabled
- start tracking downloads earlier, and faster download loop
- remove some obsolete lines
- minor PEP8 updates
2018-02-03 11:46:39 +01:00
8788f2d349
Server full and duplicate handling
- will try to report "Server full" over protocol for 10 extra
connections over limit, instead of simply dropping them
- if connected to the same host inbound and outbound, handle as server
full (prevents duplicate connections)
2018-01-02 14:29:21 +01:00
02490e3286
Don't break if over 50k messages
- typo if there were over 50k messages in inventory caused PyBM to stall
2017-12-29 08:41:15 +01:00
2d34e73648
Dandelion updates
- fixes and feedback from @gfanti and @amiller
- addresses #1049
- minor refactoring
- two global child stems with fixed mapping between parent and
child stem
- allow child stems which don't support dandelion
- only allow outbound connections to be stems
- adjust stems if opening/closing outbound connections (should
allow partial dandelion functionality when not enough outbound
connections are available instead of breaking)
2017-10-20 01:21:49 +02:00
391d40d78b
Socket closing changes
- closing reason moved to a variable
- actual closing now done in asyncore thread instead of
receivedata thread
2017-10-19 09:08:05 +02:00
a013814c6b
Network tab updates
- handle add/remove entry instead of recreating the whole connection
list
- update processed object counts after each object
2017-10-19 08:39:09 +02:00
6ce86b1d0a
Dandelion++ implementation
- untested, some functionality may be missing, don't turn on
- also, it randomises upload of requested objects
- affects #1049
2017-09-25 01:17:04 +02:00
cc955cd69d
Try new ports of binding fails
- API and BM protocol will try random ports for binding if those
configured are occupied
2017-08-09 17:29:23 +02:00
3941b39136
Randomise node id
- in order to detect if it's connected to to itself, PyBitmessage now
uses a per-connection id rather than a global one
2017-07-10 07:10:05 +02:00
2d7d9c2f92
Asyncore update
- request downloads in bigger chunks
- don't put whole objects into the receiveDataQueue
2017-07-08 06:54:25 +02:00
0f3a69adf4
Asyncore update: remove references to deleted files 2017-07-08 06:53:20 +02:00
ba4162d7fe
Asyncore update
- get rid of per-connection writeQueue/receiveQueue, and instead use
strings and locking
- minor code cleanup
- all state handlers now should set expectBytes
- almost all data processing happens in ReceiveDataThread, and
AsyncoreThread is almost only I/O (plus TLS). AsyncoreThread simply
puts the connection object into the queue when it has some data for
processing
- allow poll, epoll and kqueue handlers. kqueue is untested and
unoptimised, poll and epoll seem to work ok (linux)
- stack depth threshold handler  in decode_payload_content, this is
recursive and I think was causing occasional RuntimeErrors. Fixes #964
- longer asyncore loops, as now data is handled in ReceiveDataThread
- randomise node order when deciding what to download. Should prevent
retries being stuck to the same node
- socks cleanup (socks5 works ok, socks4a untested but should work too)
2017-07-06 19:45:36 +02:00
fe0664640e
Migrate antiIntersectionDelay to asyncore
- implemented by ignoring getdata during the delay rather than sleeping
as it was in the threaded model
- it can happen that a valid getdata request is received during the
delay. A node should be implemented in a way that retries to download,
that may not be the case with older PyBitmessage versions or other
implementations
2017-07-05 09:27:52 +02:00
a0bbd21efc
Add ratings to peers
- outbound peers now have a rating
- it's also shown in the network status tab
- currently it's between -1 to +1, changes by 0.1 steps and uses a
hyperbolic function 0.05/(1.0 - rating) to convert rating to
probability with which we should connect to that node when randomly
chosen
- it increases when we successfully establish a full outbound connection
to a node, and decreases when we fail to do that
- onion nodes have priority when using SOCKS
2017-07-05 09:17:01 +02:00
189578cba3
Asyncore proxy fixes
- SOCKS5 now seems to work, SOCKS4a untested
2017-06-24 12:23:56 +02:00
0cc8589b27
Asyncore updates
- should prevent the same object being re-requested indefinitely
- locking for object tracking
- move SSL-specific error handling to TLSDispatcher
- observe maximum connection limit when accepting a new connection
- stack depth test (for debugging purposes)
- separate download thread
- connection pool init moved to main thread
2017-06-21 12:16:33 +02:00
cba749088a
Asyncore updates
- mainly work on proxy support, but it's still not fully working
- minor bugfixes
2017-06-10 10:13:49 +02:00
e8d9a7f183
Asyncore connect handling
- minor improvements in handling of connect events so that it's not
processed twice
2017-06-03 16:30:05 +02:00
d75d920a68
Asyncore updates
- clean object tracking dictionaries in the cleaner thread
- clean up close / handle_close
- add locking to tracking dictionaries
2017-06-02 07:09:35 +02:00
18988ae2e6
Asyncore updates
- performance optimisation, reduce number of loops when waiting
for protocol headers / commands
2017-05-31 10:17:36 +02:00
f23c169eec
Don't connect to myself
- track local IP+port of incoming connections and don't connect
to them in the future
2017-05-31 00:04:21 +02:00
fa9811f426
Asyncore update
- duplicate checking implemented
- connection pool vs. socket closing cleanup
2017-05-30 23:53:43 +02:00
c85d52b8e8
Asyncore updates
- asyncore is now on by default
- inv announcements implemented
- bandwidth limit implemented / fixed
- stats on download / upload speed now work
- make prints into logger
- limit knownNodes to 20k as it was before
- green light fixed
- other minor fixes
2017-05-29 00:24:07 +02:00
5d4e1e2007
asyncore fixes
- bm headers and commands are only read up to expected length.
On a very fast connection (e.g. local VM), reading verack
also read a part of the TLS handshake
- some debugging info moved from print to logger.debug
- tls handshake cleanup
2017-05-27 22:30:30 +02:00
21f6d38ec2
Asyncore fixes
- TCP fixes
2017-05-27 21:52:56 +02:00
f8b4b427fc
Asyncore update
- bugfixes
- UDP socket for local peer discovery
- new function assembleAddr to unify creating address command
- open port checker functionality (inactive)
- sendBigInv is done in a thread separate from the network IO
thread
2017-05-27 19:09:21 +02:00