Commit Graph

29 Commits

Author SHA1 Message Date
Network fixes 2020-01-10 16:51:17 +05:30
network.BMConnectionPool: added shortcuts connections()
and establishedConnections(), some formatting fixes
2019-11-11 12:03:04 +02:00
New logging approach in order to reduce imports from submodules
and use logging without risk of circular import. Only subpackage
that imports from debug is bitmessageqt - because it also uses
Instead of from debug import logger is now recommended to use:

import logging

logger = logging.getLogger('default')

All subclasses of StoppableThread now have a logger attribute.
All threading related stuff except for set_thread_name()
was moved from helper_threading to network.threads.

Fixed two my mistakes from previous edit of debug in a1a8d3a:

 - logger.handlers is not dict but iterable
 - sys.excepthook should be set unconditionally
2019-10-18 09:35:24 +03:00
d8ea0afe40 Pylint fixes 2019-09-02 19:15:15 +05:30
8589f01d3d flake8 fixes 2019-09-02 19:02:50 +05:30
Inherit helper_threading.StoppableThread from threading.Thread
and do random.seed() in its __init__
2019-08-06 18:07:19 +03:00
Moved missingObjects dict from state to network.objectracker 2018-12-16 17:44:26 +02:00
Download fixes
- in corner cases, download request could have contained an incorrect
  request length. I haven't actually checked if this can be triggered
- wait for downloading until anti intersection delay expires. Doesn't
  necessarily mean that it will always avoid peer's anti intersection
  delay, but it's close enough
- tracks last time an object was received. If it was too long time ago,
  reset the download request queue. This avoid situations like when a
  request gets ignored during the anti intersection delay, but it will
  keep thinking there are still pending requests as long as not all
  missing objects have been requested. This caused staggered download
  (request 1000 items, wait 1 minute, request 1000 more, wait another
  minute, ...)
- with these fixes, you should end up downloading as fast as your
  network and CPU allow
- best tested with trustedpeer
2018-04-03 19:24:07 +02:00
replace all with helper random function 2018-03-21 19:26:27 +05:30
Dandelion update
- dandelion fixes
- try to wait as long as possible before expiration if there are no
outbound connections
- expire in invThread rather than singleCleaner thread
- deduplication of code in inv and dinv command methods
- turn on by default, seems to work correctly now
- turn off dandelion if outbound connections are disabled
- start tracking downloads earlier, and faster download loop
- remove some obsolete lines
- minor PEP8 updates
2018-02-03 11:46:39 +01:00
Download fixes
- don't make empty requests
- don't make requests if all objects are pending already
2018-02-02 12:44:43 +01:00
Download thread exception handling 2018-02-01 23:18:08 +01:00
Download fixes
- check if already have object before requesting
2018-02-01 22:58:04 +01:00
Download fixes
- don't expire too quickly
- ignore connections that haven't been fully established yet
2018-02-01 12:48:14 +01:00
Typo 2018-02-01 12:26:54 +01:00
Download optimisation
- new data structure to handle download tracking, uses less CPU
2018-02-01 12:19:39 +01:00
Download optimisations
- don't make empty download requests
- use smaller chunks when they can be spread across multiple
2018-01-01 12:51:35 +01:00
Don't crash download thread if no connections 2017-12-02 00:50:58 +01:00
Distribute downloads more evenly
- also increases expireation of missing objects from 10 minutes to an
2017-12-02 00:48:08 +01:00
Download optimisation
- pending download tracking now per-connection instead of globally
2017-11-17 19:49:51 +01:00
Download thread error handling 2017-10-22 15:28:30 +02:00
Download optimisation
- more accurate tracking
- randomise download order
- longer cycle
2017-10-22 11:32:37 +02:00
Asyncore performance optimisation
- use bytearray instead of strings for buffers
2017-10-16 08:07:32 +02:00
Change thread names
- not needed to have "Thread" in the name of a thread
2017-07-10 07:05:50 +02:00
Asyncore update
- request downloads in bigger chunks
- don't put whole objects into the receiveDataQueue
2017-07-08 06:54:25 +02:00
Asyncore update
- get rid of per-connection writeQueue/receiveQueue, and instead use
strings and locking
- minor code cleanup
- all state handlers now should set expectBytes
- almost all data processing happens in ReceiveDataThread, and
AsyncoreThread is almost only I/O (plus TLS). AsyncoreThread simply
puts the connection object into the queue when it has some data for
- allow poll, epoll and kqueue handlers. kqueue is untested and
unoptimised, poll and epoll seem to work ok (linux)
- stack depth threshold handler  in decode_payload_content, this is
recursive and I think was causing occasional RuntimeErrors. Fixes #964
- longer asyncore loops, as now data is handled in ReceiveDataThread
- randomise node order when deciding what to download. Should prevent
retries being stuck to the same node
- socks cleanup (socks5 works ok, socks4a untested but should work too)
2017-07-06 19:45:36 +02:00
Download thread updates
- now tracks downloads globally too, so it doesn't request the same
object from multiple peers at the same time
- retries at the earliest every minute
- stops trying to download an object after an hour
- minor fixes in retrying downloading invalid objects
2017-07-05 09:25:49 +02:00
Code quality improvements 2017-06-24 12:13:35 +02:00
Asyncore updates
- should prevent the same object being re-requested indefinitely
- locking for object tracking
- move SSL-specific error handling to TLSDispatcher
- observe maximum connection limit when accepting a new connection
- stack depth test (for debugging purposes)
- separate download thread
- connection pool init moved to main thread
2017-06-21 12:16:33 +02:00