Commit Graph

39 Commits

Author SHA1 Message Date
Dmitri Bogomolov 7a89109fc9
New logging approach in order to reduce imports from submodules
and use logging without risk of circular import. Only subpackage
that imports from debug is bitmessageqt - because it also uses
debug.resetLogging().
Instead of from debug import logger is now recommended to use:

import logging

logger = logging.getLogger('default')

All subclasses of StoppableThread now have a logger attribute.
All threading related stuff except for set_thread_name()
was moved from helper_threading to network.threads.

Fixed two my mistakes from previous edit of debug in a1a8d3a:

 - logger.handlers is not dict but iterable
 - sys.excepthook should be set unconditionally
2019-10-18 09:35:24 +03:00
coffeedogs c875769b15
Changes based on style and lint checks. (final_code_quality_3) 2018-10-31 12:58:46 +00:00
Peter Šurda 1184b23223
Separate protocol processing exceptions
- exceptions thrown by the state methods are separated from missing
  connections or states. This allows more accurate error reporting
2018-02-19 21:27:38 +01:00
Peter Šurda eb97face61
Show traceback in protocol parser error handler 2018-02-12 17:07:54 +01:00
Peter Šurda ba91d21261
CPU hogging fix
- handle _command functions that don't return anything
- fix udp command function
2018-01-22 22:18:01 +01:00
Peter Šurda 4086253730
Bandwidth limit optimisation
- should be slightly more accurate and use slightly fewer resources
2018-01-02 15:24:47 +01:00
Peter Šurda 5a787f41d2
Socket closing changes
- explicit close only through asyncore error handler
- implicit close through garbage collector
- avoid duplicate closing
2017-11-17 13:37:51 +01:00
Peter Šurda 15857e6551
Asyncore updates
- reduce buffer size to 128kB (was 2MB)
- IP address handling use str instead of buffer (the latter, even
though it should be faster, breaks the code on Windows)
- read up to full buffer after fully established (otherwise
downloads become too slow due to the loop time). This reverts
a change made in d28a7bfb86
2017-10-20 01:07:30 +02:00
Peter Šurda 7b470d4b66
handle shutdown in receivequeuethread
- sometimes during shutdown, the receivequeuethread would get
stuck
2017-10-19 09:11:34 +02:00
Peter Šurda 391d40d78b
Socket closing changes
- closing reason moved to a variable
- actual closing now done in asyncore thread instead of
receivedata thread
2017-10-19 09:08:05 +02:00
Peter Šurda d28a7bfb86
Asyncore performance optimisation
- don't transfer unnecessary amount of bytes from network buffers
- slice buffer more efficiently if it results in an empty buffer
2017-10-19 09:02:33 +02:00
Peter Šurda 7ec3fc7a5a
Prevent negative DownloadChunk in asyncore 2017-10-19 09:00:54 +02:00
Peter Šurda 4c9006a632
Asyncore performance optimisation
- use bytearray instead of strings for buffers
2017-10-16 08:07:32 +02:00
Peter Šurda 660997b8f4
Code Quality 2017-08-22 13:49:27 +02:00
Peter Šurda dcc181bf75
Asyncore processing thread synchronisation
- threre was a synchronisation problem where one thread could process
more data than another thread was expecting, leading to the thread
crashing
2017-07-10 23:18:58 +02:00
Peter Šurda f6d5d93bf2
Multiple receive queues fix
- forgot to commit busy handler
2017-07-10 20:52:11 +02:00
Peter Šurda bdf61489ae
Allow multiple ReceiveQueue threads
- defaults to 3
2017-07-10 07:08:10 +02:00
Peter Šurda a98b8690d3
Asyncore fixes
- fix broken loops
- optimise I/O tests
2017-07-07 07:55:29 +02:00
Peter Šurda ba4162d7fe
Asyncore update
- get rid of per-connection writeQueue/receiveQueue, and instead use
strings and locking
- minor code cleanup
- all state handlers now should set expectBytes
- almost all data processing happens in ReceiveDataThread, and
AsyncoreThread is almost only I/O (plus TLS). AsyncoreThread simply
puts the connection object into the queue when it has some data for
processing
- allow poll, epoll and kqueue handlers. kqueue is untested and
unoptimised, poll and epoll seem to work ok (linux)
- stack depth threshold handler  in decode_payload_content, this is
recursive and I think was causing occasional RuntimeErrors. Fixes #964
- longer asyncore loops, as now data is handled in ReceiveDataThread
- randomise node order when deciding what to download. Should prevent
retries being stuck to the same node
- socks cleanup (socks5 works ok, socks4a untested but should work too)
2017-07-06 19:45:36 +02:00
Peter Šurda 846fced0a2
Remove obsolete inactive code 2017-07-05 09:19:56 +02:00
Peter Šurda dc5a91f326
Remove stack depth warnings
- I was never able to trigger them
2017-06-24 12:19:19 +02:00
Peter Šurda 2685fe29b1
Code quality improvements 2017-06-24 12:13:35 +02:00
Peter Šurda 0cc8589b27
Asyncore updates
- should prevent the same object being re-requested indefinitely
- locking for object tracking
- move SSL-specific error handling to TLSDispatcher
- observe maximum connection limit when accepting a new connection
- stack depth test (for debugging purposes)
- separate download thread
- connection pool init moved to main thread
2017-06-21 12:16:33 +02:00
Peter Šurda e8d9a7f183
Asyncore connect handling
- minor improvements in handling of connect events so that it's not
processed twice
2017-06-03 16:30:05 +02:00
Peter Šurda d75d920a68
Asyncore updates
- clean object tracking dictionaries in the cleaner thread
- clean up close / handle_close
- add locking to tracking dictionaries
2017-06-02 07:09:35 +02:00
Peter Šurda c85d52b8e8
Asyncore updates
- asyncore is now on by default
- inv announcements implemented
- bandwidth limit implemented / fixed
- stats on download / upload speed now work
- make prints into logger
- limit knownNodes to 20k as it was before
- green light fixed
- other minor fixes
2017-05-29 00:24:07 +02:00
Peter Šurda 5d4e1e2007
asyncore fixes
- bm headers and commands are only read up to expected length.
On a very fast connection (e.g. local VM), reading verack
also read a part of the TLS handshake
- some debugging info moved from print to logger.debug
- tls handshake cleanup
2017-05-27 22:30:30 +02:00
Peter Šurda 21f6d38ec2
Asyncore fixes
- TCP fixes
2017-05-27 21:52:56 +02:00
Peter Šurda fa9ad537a5
Add task_done to asyncore-related queues 2017-05-27 19:39:19 +02:00
Peter Šurda f8b4b427fc
Asyncore update
- bugfixes
- UDP socket for local peer discovery
- new function assembleAddr to unify creating address command
- open port checker functionality (inactive)
- sendBigInv is done in a thread separate from the network IO
thread
2017-05-27 19:09:21 +02:00
Peter Šurda e309a1edb3
Asyncore update
- separate queue for processing blocking stuff on reception
- rewrote write buffer as a queue
- some addr handling
- number of half open connections correct
2017-05-25 23:04:33 +02:00
Peter Šurda 9683c879bc
Asyncore update
- Network status UI works but current speed isn't implemented yet
- Track per connection and global transferred bytes
- Add locking to write queue so that other threads can put stuff
there
- send ping on timeout (instead of closing the connection)
- implement open port checker (untested, never triggered yet)
- error handling on IO
2017-05-25 14:59:18 +02:00
Peter Šurda d635e515b9
Big Asyncore update
- most of the stuff is done so it partially works
- disabled pollers other than select (debugging necessary)
- can switch in the settings, section network, option asyncore (defaults
to False)
2017-05-24 16:51:49 +02:00
Peter Šurda 96d58f3c11
Asyncore update (WIP) 2017-04-16 18:27:15 +02:00
Peter Šurda 46c9ea9403
Async network updates (WIP)
- cleaner command handling
- separating into header and command handling
- incoming connection handler
- bugfixes and more debug information
2017-03-20 18:32:26 +01:00
Peter Šurda 49869d0b56
Networking subsystem updates
- version command struct for faster unpacking
- increase read buffer to 2MB to allow a full command to fit
- initial bitmessage protocol class (WIP)
- error handling
- remove duplicate method
2017-03-11 11:12:08 +01:00
Peter Šurda 998935be5f
New network subsystem, WIP
- finished proxy design
- socks4a and socks5 implemented
- authentication not tested
- resolver for both socks4a and socks5
- http client example using the proxy
2017-03-10 23:11:57 +01:00
Peter Šurda 2cc4435cfc
Imports, typos, syntax errors
- remove obsolete imports
- add missing imports
- fix typos
- fix syntax errors
- thanks to landscape.io for report
2017-02-08 14:19:02 +01:00
Peter Šurda 75090abaaf
Advanced dispatcher class
- generic class the new asyncore-based network subsystem that handles
  buffered data transfer
2017-01-10 21:20:49 +01:00