Mahendra
1a36361ef9
replace all random.xxx with helper random function
2018-03-21 19:26:27 +05:30
4b995e7996
Fix circular import
...
- previous commit broke
2018-03-13 11:42:43 +01:00
1e4400a207
Remove obsolete code
...
- PendingDownload and PendingUpload in inventory.py are obsolete
- the replacement functions only provide lengths and are in
network/stats.py
2018-03-13 07:32:23 +01:00
0da0c10995
Socks5 parsing fix
...
- Fixes #1134
- Came over BM, thank you kind stranger
2018-02-26 20:04:57 +01:00
1184b23223
Separate protocol processing exceptions
...
- exceptions thrown by the state methods are separated from missing
connections or states. This allows more accurate error reporting
2018-02-19 21:27:38 +01:00
757740369b
Add proxy authentication
...
- proxy module was missing initialisation of authentication
2018-02-18 20:53:16 +01:00
eb97face61
Show traceback in protocol parser error handler
2018-02-12 17:07:54 +01:00
066b419e16
Bugfixes
...
- typo in dandelion
- stealth ackdata fix for broadcasts and mailing lists
2018-02-09 00:49:08 +01:00
08bb85a952
Dandelion staticmethod fix
2018-02-07 17:22:26 +01:00
3d1fa473fb
Dandelion updates
...
- expiration now uses poisson distribution just like in the bitcoin
version
2018-02-06 22:28:56 +01:00
03f08b3cfd
Typo
2018-02-04 21:16:30 +01:00
a646ec4902
Allow separate proxy for onions
...
- new options in network section: onionsocksproxytype,
onionsockshostname and onionsocksport. These allow to separate
connectivity types for onion and non-onion addresses, e.g. connect to
clear nodes over clearnet and onions over tor
- also remove some obsolete imports
2018-02-04 21:03:54 +01:00
fd1a6c1fa1
Dandelion update
...
- dandelion fixes
- try to wait as long as possible before expiration if there are no
outbound connections
- expire in invThread rather than singleCleaner thread
- deduplication of code in inv and dinv command methods
- turn on by default, seems to work correctly now
- turn off dandelion if outbound connections are disabled
- start tracking downloads earlier, and faster download loop
- remove some obsolete lines
- minor PEP8 updates
2018-02-03 11:46:39 +01:00
053f434e04
Download fixes
...
- don't make empty requests
- don't make requests if all objects are pending already
2018-02-02 12:44:43 +01:00
57c8c7c07c
Download thread exception handling
2018-02-01 23:18:08 +01:00
8498143783
Download fixes
...
- check if already have object before requesting
2018-02-01 22:58:04 +01:00
290b87a49f
Download fixes
...
- don't expire too quickly
- ignore connections that haven't been fully established yet
2018-02-01 12:48:14 +01:00
c5dc7fc903
Typo
2018-02-01 12:26:54 +01:00
8d05742436
Typo
2018-02-01 12:20:41 +01:00
68b58ce0c5
Download optimisation
...
- new data structure to handle download tracking, uses less CPU
2018-02-01 12:19:39 +01:00
d223bfc6f2
Fix kqueue poller
...
- separate read and write filters
- make filters peristent for reduced syscall count
2018-01-31 22:25:23 +01:00
222e666a60
Network thread shutdown iterator fix
...
Thanks to @g1itch for pointing this out.
2018-01-31 21:09:36 +01:00
9e79386595
Set state to "close" and call handle_close() for all connections
2018-01-30 14:04:03 +02:00
d6df4470e1
No connection CPU hog fix
...
- the previous fix was incomplete, it shouldn't consume excessive
resources now when there are no connections
2018-01-23 15:59:58 +01:00
01c8f3b66d
Fix asyncore CPU usage on no connection
...
- if there are no connections, asyncore would hog CPU
- thanks to an anonymous contributor
2018-01-22 22:37:29 +01:00
ba91d21261
CPU hogging fix
...
- handle _command functions that don't return anything
- fix udp command function
2018-01-22 22:18:01 +01:00
c9851b9f41
Connection lookups invalid data handling
...
- shouldn't throw an exception if argument is a string rather than Peer
2018-01-02 22:23:03 +01:00
f74f82e54f
Start downloading earlier
2018-01-02 22:20:33 +01:00
4086253730
Bandwidth limit optimisation
...
- should be slightly more accurate and use slightly fewer resources
2018-01-02 15:24:47 +01:00
8788f2d349
Server full and duplicate handling
...
- will try to report "Server full" over protocol for 10 extra
connections over limit, instead of simply dropping them
- if connected to the same host inbound and outbound, handle as server
full (prevents duplicate connections)
2018-01-02 14:29:21 +01:00
36cc5b9cf5
Download optimisations
...
- don't make empty download requests
- use smaller chunks when they can be spread across multiple
connections
2018-01-01 12:51:35 +01:00
1864762a0a
Apply bandwidth limits without restart
...
- also minor style fixes
2018-01-01 12:49:08 +01:00
e9b1aa48a9
Protocol error handler fixes
...
- was broken if there was no error message in "raise"
- added default texts for network exceptions
2017-12-29 08:49:08 +01:00
02490e3286
Don't break if over 50k messages
...
- typo if there were over 50k messages in inventory caused PyBM to stall
2017-12-29 08:41:15 +01:00
a3398d6a17
Don't crash download thread if no connections
2017-12-02 00:50:58 +01:00
41ead2bfb5
Distribute downloads more evenly
...
- also increases expireation of missing objects from 10 minutes to an
hour
2017-12-02 00:48:08 +01:00
e558b1fb72
Error handling
...
- proxy connections didn't init fullyEstablished
2017-11-19 13:48:43 +01:00
fdfbb77ed2
Close filehandles on connection reaping
...
- I thought this is done automatically through garbage collection, but I
think as the channel is still assigned in the asyncore map, it needs
to be done manually. Basically filehandle limit exceeded and it
crashed
2017-11-19 00:05:55 +01:00
5e042b76e7
Typo
...
- missing brackets
2017-11-18 10:05:41 +01:00
3aa6f386db
Dandelion fixes
...
- dandelion would always think there is a cycle and trigger fluff
- cycle fluff trigger didn't correctly re-download and re-announce the
object. Now it remembers between (d)inv and object commands that it's
in a fluff trigger phase.
2017-11-18 09:47:17 +01:00
4690dd6f00
Copy object contents from buffers on instantiation
...
- this may fix some memory issues
2017-11-17 23:53:46 +01:00
3c3d69e5de
Reap closed connection fix
2017-11-17 19:50:39 +01:00
1e02d2b48a
Download optimisation
...
- pending download tracking now per-connection instead of globally
2017-11-17 19:49:51 +01:00
5a787f41d2
Socket closing changes
...
- explicit close only through asyncore error handler
- implicit close through garbage collector
- avoid duplicate closing
2017-11-17 13:37:51 +01:00
1792289e00
close handling fix
...
- don't close a connection twice
2017-11-14 23:43:05 +01:00
4b40d4bce1
Download thread error handling
2017-10-22 15:28:30 +02:00
75a6f605c1
Download optimisation
...
- more accurate tracking
- randomise download order
- longer cycle
2017-10-22 11:32:37 +02:00
b025624f2a
missingObjects fix
...
- didn't notice valid objects arriving (only invalid)
2017-10-20 23:21:25 +02:00
6655e99aa3
Pending download stats optimisations
...
- tracks separately a global list for a faster sum. Needs slightly
more memory
2017-10-20 23:11:33 +02:00
a746ba9da7
Stop downloading objects with insufficient PoW
...
- object with insufficient PoW weren't correctly detected and it
tried to download them over and over again
2017-10-20 13:21:39 +02:00
2d34e73648
Dandelion updates
...
- fixes and feedback from @gfanti and @amiller
- addresses #1049
- minor refactoring
- two global child stems with fixed mapping between parent and
child stem
- allow child stems which don't support dandelion
- only allow outbound connections to be stems
- adjust stems if opening/closing outbound connections (should
allow partial dandelion functionality when not enough outbound
connections are available instead of breaking)
2017-10-20 01:21:49 +02:00
15857e6551
Asyncore updates
...
- reduce buffer size to 128kB (was 2MB)
- IP address handling use str instead of buffer (the latter, even
though it should be faster, breaks the code on Windows)
- read up to full buffer after fully established (otherwise
downloads become too slow due to the loop time). This reverts
a change made in d28a7bfb86
2017-10-20 01:07:30 +02:00
ab458531e8
Changes in SOCKS and onion handling in connectionchooser
...
- onion addresses have a priority of 1 when SOCKS is on
- don't connect to private/loopback addresses when SOCKS is on
2017-10-19 09:16:29 +02:00
7b470d4b66
handle shutdown in receivequeuethread
...
- sometimes during shutdown, the receivequeuethread would get
stuck
2017-10-19 09:11:34 +02:00
391d40d78b
Socket closing changes
...
- closing reason moved to a variable
- actual closing now done in asyncore thread instead of
receivedata thread
2017-10-19 09:08:05 +02:00
d28a7bfb86
Asyncore performance optimisation
...
- don't transfer unnecessary amount of bytes from network buffers
- slice buffer more efficiently if it results in an empty buffer
2017-10-19 09:02:33 +02:00
7ec3fc7a5a
Prevent negative DownloadChunk in asyncore
2017-10-19 09:00:54 +02:00
f785558ca5
Don't close UDP socket on bad packet magic
2017-10-19 09:00:02 +02:00
a090eea9b0
Minor multiqueue updates
...
- add task_done to addrthread and invthread
- implement totalSize for multiqueue
- order in invThread changed
2017-10-19 08:56:48 +02:00
d44c6c6464
Forget known nodes with bad rating
2017-10-19 08:52:44 +02:00
a013814c6b
Network tab updates
...
- handle add/remove entry instead of recreating the whole connection
list
- update processed object counts after each object
2017-10-19 08:39:09 +02:00
7e0932815d
UDP socket closing fix
...
- invalid data or an incomplete read on UDP socket caused
it to close
2017-10-19 01:46:32 +02:00
4c9006a632
Asyncore performance optimisation
...
- use bytearray instead of strings for buffers
2017-10-16 08:07:32 +02:00
a49b3b5f84
Asyncore UDP fix
...
- when there was an error writing to a udp socket, it wasn't
handled correctly
2017-10-06 18:26:06 +02:00
333170b172
Dandelion fixes
...
- more exception handling
- only use outbound connections for stems
(thanks to @amillter for info)
- don't create stems if config disabled
- addresses #1049
2017-10-06 16:33:44 +02:00
6548999a49
Dandelion fix
...
- thanks to g1itch for reporting
- addresses #1049
2017-10-02 08:02:29 +02:00
b1442ecb0a
Dandelion fixes and updates
...
- also, randomise the item order in an inv/dinv command
2017-09-30 13:42:04 +02:00
9923d288e0
Dandelion fixes
...
- in route selector, some connections may not have the services
attribute (yet)
- Addresses #1049
2017-09-25 09:17:15 +02:00
d574b167d8
Dandelion updates & fixes
...
- Addresses #1049
- Add dandelion routes for locally generated objects
- Minor bugfixes
- Send dinv commands on stem objects (instead of always sending inv
command)
2017-09-25 08:49:21 +02:00
6ce86b1d0a
Dandelion++ implementation
...
- untested, some functionality may be missing, don't turn on
- also, it randomises upload of requested objects
- affects #1049
2017-09-25 01:17:04 +02:00
1881bcea68
Don't connect on first start until approved
2017-09-21 18:18:42 +02:00
660997b8f4
Code Quality
2017-08-22 13:49:27 +02:00
e7231f3aea
Fix multiple TCP bind address handling
2017-08-09 23:30:22 +02:00
6c695c8ac7
Remove non-asyncore network code (partial)
2017-08-09 17:36:52 +02:00
0b07b1c89a
Peer discovery updates
...
- allow loopback addresses, now you can bind different loopback IP
addresses on a single system and they will auto-cross-connect
- always listen for discovery on 0.0.0.0
- [network] - bind now also applies for the TCP socket as well as UDP
socket
- closing socket iterator fix
2017-08-09 17:34:47 +02:00
e071efac1a
Typo
2017-08-09 17:29:48 +02:00
cc955cd69d
Try new ports of binding fails
...
- API and BM protocol will try random ports for binding if those
configured are occupied
2017-08-09 17:29:23 +02:00
0324958e92
Peer discovery fixes
...
- incoming packets weren't correctly processed
2017-08-06 23:05:54 +02:00
f338c00f8e
Change peer discovery tracking from queue to a dict
...
- with a queue, a situation could occur when new entries are appended
but nothing is polling the queue
2017-08-06 21:29:54 +02:00
4564d37f5b
Typo in previous commits
2017-08-06 21:26:25 +02:00
38872159fb
Typo in previous commit
2017-08-06 20:40:35 +02:00
5895dc2f1f
Asyncore Windows error handling
...
- windows behaves somewhat differently when using select
2017-08-06 20:39:14 +02:00
5108d08ac9
Windows asyncore error handler fix
...
- WSAEWOULDBLOCK is now checked on connect and accept
2017-08-06 18:18:21 +02:00
8f14fb05a1
UDP socket setsockopt fix
2017-08-05 10:14:15 +02:00
a29f7534ee
Add EINTR handler for select and poll pollers
2017-07-21 09:06:02 +02:00
2530c62050
epoll throws IOError rather than select.error
2017-07-21 07:49:34 +02:00
aa059d6f2f
Handle TLS errors in receivequeuethread
...
- well at least EBADF, it seems to happen sometimes
2017-07-21 07:47:18 +02:00
4f19c37fdc
Parser fix for multi-level arrays
2017-07-11 10:29:29 +02:00
dcc181bf75
Asyncore processing thread synchronisation
...
- threre was a synchronisation problem where one thread could process
more data than another thread was expecting, leading to the thread
crashing
2017-07-10 23:18:58 +02:00
f6d5d93bf2
Multiple receive queues fix
...
- forgot to commit busy handler
2017-07-10 20:52:11 +02:00
db2d78c9b6
Make protocol decoder less recursive
...
- apparently, recursion has bad performance in Python, so the decoder is
now flat, except when parsing "version" command
2017-07-10 07:15:27 +02:00
853c8561ec
Per connection node id part 2
...
- forgot to include this in the previous commit
2017-07-10 07:12:52 +02:00
3941b39136
Randomise node id
...
- in order to detect if it's connected to to itself, PyBitmessage now
uses a per-connection id rather than a global one
2017-07-10 07:10:05 +02:00
bdf61489ae
Allow multiple ReceiveQueue threads
...
- defaults to 3
2017-07-10 07:08:10 +02:00
f088e0ae21
Change thread names
...
- not needed to have "Thread" in the name of a thread
2017-07-10 07:05:50 +02:00
4fce01e34a
Less data transferred in invThread and addrThread
2017-07-08 18:02:47 +02:00
2df9598774
Asyncore update: Fix incoming connections
...
- dereferencing wasn't done correctly for incoming connections
2017-07-08 07:33:29 +02:00
2d7d9c2f92
Asyncore update
...
- request downloads in bigger chunks
- don't put whole objects into the receiveDataQueue
2017-07-08 06:54:25 +02:00
0f3a69adf4
Asyncore update: remove references to deleted files
2017-07-08 06:53:20 +02:00
5ae1b6d865
Asyncore update: remove obsolete files
2017-07-08 06:52:17 +02:00
