Commit Graph

1831 Commits

Author SHA1 Message Date
aa059d6f2f
Handle TLS errors in receivequeuethread
- well at least EBADF, it seems to happen sometimes
2017-07-21 07:47:18 +02:00
4f19c37fdc
Parser fix for multi-level arrays 2017-07-11 10:29:29 +02:00
dcc181bf75
Asyncore processing thread synchronisation
- threre was a synchronisation problem where one thread could process
more data than another thread was expecting, leading to the thread
crashing
2017-07-10 23:18:58 +02:00
f6d5d93bf2
Multiple receive queues fix
- forgot to commit busy handler
2017-07-10 20:52:11 +02:00
db2d78c9b6
Make protocol decoder less recursive
- apparently, recursion has bad performance in Python, so the decoder is
now flat, except when parsing "version" command
2017-07-10 07:15:27 +02:00
853c8561ec
Per connection node id part 2
- forgot to include this in the previous commit
2017-07-10 07:12:52 +02:00
3941b39136
Randomise node id
- in order to detect if it's connected to to itself, PyBitmessage now
uses a per-connection id rather than a global one
2017-07-10 07:10:05 +02:00
bdf61489ae
Allow multiple ReceiveQueue threads
- defaults to 3
2017-07-10 07:08:10 +02:00
f088e0ae21
Change thread names
- not needed to have "Thread" in the name of a thread
2017-07-10 07:05:50 +02:00
4fce01e34a
Less data transferred in invThread and addrThread 2017-07-08 18:02:47 +02:00
2df9598774
Asyncore update: Fix incoming connections
- dereferencing wasn't done correctly for incoming connections
2017-07-08 07:33:29 +02:00
2d7d9c2f92
Asyncore update
- request downloads in bigger chunks
- don't put whole objects into the receiveDataQueue
2017-07-08 06:54:25 +02:00
0f3a69adf4
Asyncore update: remove references to deleted files 2017-07-08 06:53:20 +02:00
5ae1b6d865
Asyncore update: remove obsolete files 2017-07-08 06:52:17 +02:00
PyBitmessage Translations
1aa45d6eac
Auto-updated language eo from transifex 2017-07-07 16:32:25 +02:00
PyBitmessage Translations
6eb9e93575
Auto-updated language ja from transifex 2017-07-07 16:31:14 +02:00
PyBitmessage Translations
4f969088cf
Auto-updated language pl from transifex 2017-07-07 07:56:19 +02:00
a98b8690d3
Asyncore fixes
- fix broken loops
- optimise I/O tests
2017-07-07 07:55:29 +02:00
de22e547c5
Remove buggy log message if prctl is missing
- it's not that important that you need to be informed of it, and
importing logging may cause cyclic dependencies/other problems
2017-07-06 20:06:43 +02:00
ba4162d7fe
Asyncore update
- get rid of per-connection writeQueue/receiveQueue, and instead use
strings and locking
- minor code cleanup
- all state handlers now should set expectBytes
- almost all data processing happens in ReceiveDataThread, and
AsyncoreThread is almost only I/O (plus TLS). AsyncoreThread simply
puts the connection object into the queue when it has some data for
processing
- allow poll, epoll and kqueue handlers. kqueue is untested and
unoptimised, poll and epoll seem to work ok (linux)
- stack depth threshold handler  in decode_payload_content, this is
recursive and I think was causing occasional RuntimeErrors. Fixes #964
- longer asyncore loops, as now data is handled in ReceiveDataThread
- randomise node order when deciding what to download. Should prevent
retries being stuck to the same node
- socks cleanup (socks5 works ok, socks4a untested but should work too)
2017-07-06 19:45:36 +02:00
a2b8867c1a
Tooltips for network status columns 2017-07-06 19:36:04 +02:00
4536e44b8c
Thread names propagate to system
- the thread names should now show up in the monitoring tools of
operating systems (tested on linux)
2017-07-06 19:35:40 +02:00
PyBitmessage Translations
00a4558971
Auto-updated language eo from transifex 2017-07-06 04:53:29 +02:00
PyBitmessage Translations
83e60d231a
Auto-updated language pl from transifex 2017-07-06 04:52:26 +02:00
PyBitmessage Translations
1dbf98d7f7
Auto-updated language ja from transifex 2017-07-06 00:06:05 +02:00
PyBitmessage Translations
27f9fb0237
Auto-updated language ru from transifex 2017-07-05 17:09:30 +02:00
fe0664640e
Migrate antiIntersectionDelay to asyncore
- implemented by ignoring getdata during the delay rather than sleeping
as it was in the threaded model
- it can happen that a valid getdata request is received during the
delay. A node should be implemented in a way that retries to download,
that may not be the case with older PyBitmessage versions or other
implementations
2017-07-05 09:27:52 +02:00
fc19e4119a
Download thread updates
- now tracks downloads globally too, so it doesn't request the same
object from multiple peers at the same time
- retries at the earliest every minute
- stops trying to download an object after an hour
- minor fixes in retrying downloading invalid objects
2017-07-05 09:25:49 +02:00
846fced0a2
Remove obsolete inactive code 2017-07-05 09:19:56 +02:00
d086781ce8
AddrQueue fix
- forgot to commit import
2017-07-05 09:19:18 +02:00
a0bbd21efc
Add ratings to peers
- outbound peers now have a rating
- it's also shown in the network status tab
- currently it's between -1 to +1, changes by 0.1 steps and uses a
hyperbolic function 0.05/(1.0 - rating) to convert rating to
probability with which we should connect to that node when randomly
chosen
- it increases when we successfully establish a full outbound connection
to a node, and decreases when we fail to do that
- onion nodes have priority when using SOCKS
2017-07-05 09:17:01 +02:00
9d09f9f3ce
Reduce severity of socks connectivity errors
- Fixes #1024
- Fixes #1019
2017-07-05 09:07:00 +02:00
773d91bbe2
Unknown object log entry less severe
- unnecessarily classified as critical
- fixes #1023
2017-07-05 09:01:40 +02:00
e00a02206b
AddrThread
- this thread is for spreading new/updated addresses in active
connections, analogous to the InvThread
- it doesn't do anything yet, this is just a dummy queue at the moment
2017-07-05 08:57:44 +02:00
27f5de0f9c
Unified random number provider
- not used yet, just an inactive helper function
- I received feedback that OpenSSL.rand isn't more secure than
os.urandom. I read several debates/analyses about it and concur
2017-07-05 08:52:16 +02:00
PyBitmessage Translations
70c5929e92
Auto-updated language ru from transifex 2017-07-04 17:51:58 +02:00
cc3cf77759
New class multiqueue
- to be used for invthread and addthread
- updated invthread for multiqueue
2017-06-27 13:25:12 +02:00
f5a143d0b8
Config validator
- config options can have validators
- limit maxoutboundconnections to max 8
2017-06-27 13:19:12 +02:00
20e01860cf
Network status peer list shouldn't be editable 2017-06-27 13:16:41 +02:00
PyBitmessage Translations
8a3577aed7
Auto-updated language eo from transifex 2017-06-25 19:26:14 +02:00
PyBitmessage Translations
30b65aaefc
Auto-updated language pl from transifex 2017-06-25 19:25:47 +02:00
PyBitmessage Translations
ccfe58c2c0
Auto-updated language ja from transifex 2017-06-25 16:49:31 +02:00
aa203b23ee
Fix typo introduced by code quality patch 2017-06-24 23:09:08 +02:00
189578cba3
Asyncore proxy fixes
- SOCKS5 now seems to work, SOCKS4a untested
2017-06-24 12:23:56 +02:00
916b85c862
Connection pool cleanup
- minor code quality improvement
2017-06-24 12:23:16 +02:00
0dc0b22974
Expired / Stream mismatch / duplicate object error handling
- cleanup of the code
2017-06-24 12:22:41 +02:00
0a79490e2c
Known nodes maximum configurable part 2 2017-06-24 12:21:42 +02:00
d57b0c55ee
Object validator trigger moved
- from bmproto to bmobject
2017-06-24 12:21:06 +02:00
dc5a91f326
Remove stack depth warnings
- I was never able to trigger them
2017-06-24 12:19:19 +02:00
e9edf70d3a
TLS updates
- save TLS version
- minor TLS error handling updates
2017-06-24 12:18:15 +02:00
b9d60f8b41
Max known nodes configurable 2017-06-24 12:17:01 +02:00
26eb54a82e
Network status updates
- now lists each node with its info instead of a per-stream summary
2017-06-24 12:16:12 +02:00
bfbdd7e140
Chan address validator button feedback
- During validation, the button not only turns gray but also changes
label
2017-06-24 12:14:23 +02:00
2685fe29b1
Code quality improvements 2017-06-24 12:13:35 +02:00
243025a1aa
Leave __delitem__ unimplemented in filesystem storage
- rename the cleaning method
2017-06-21 12:17:40 +02:00
618f3865c1
Main thread end
- instead of sleeping, it just ends
2017-06-21 12:16:56 +02:00
0cc8589b27
Asyncore updates
- should prevent the same object being re-requested indefinitely
- locking for object tracking
- move SSL-specific error handling to TLSDispatcher
- observe maximum connection limit when accepting a new connection
- stack depth test (for debugging purposes)
- separate download thread
- connection pool init moved to main thread
2017-06-21 12:16:33 +02:00
76fed78211
Made it possible to use installed umsgpack 2017-06-12 13:54:44 +03:00
a3a55e53c4
UDP Socket dict address fix
- fixes #1008
2017-06-11 14:11:39 +02:00
cba749088a
Asyncore updates
- mainly work on proxy support, but it's still not fully working
- minor bugfixes
2017-06-10 10:13:49 +02:00
7deb7c3d4f
Typo 2017-06-10 10:11:50 +02:00
8d41d5fcf6
default known nodes update 2017-06-10 10:11:21 +02:00
7f06cb7c27
Inventory flush fix in GUI
- Fixes #1011
2017-06-10 10:10:59 +02:00
7e8ee51322
Fallback umsgpack import fix 2017-06-10 10:09:44 +02:00
f366447e94
Fix identicon imports 2017-06-10 10:09:14 +02:00
d34fdbb3f4
Fix network status in api 2017-06-10 10:08:40 +02:00
ae97f7abd8
Typo 2017-06-10 10:07:47 +02:00
009a215224
Fix api connected hosts lists 2017-06-09 10:07:51 +02:00
7bde4e9445
Missing module name in bmproto 2017-06-04 10:25:16 +02:00
e8d9a7f183
Asyncore connect handling
- minor improvements in handling of connect events so that it's not
processed twice
2017-06-03 16:30:05 +02:00
a9c0000c17
Treat some invalid objects as received
- update to 6044df5adf
- objects that are expired or in wrong stream are not re-requested
anymore, even if they aren't stored in the inventory
- the previous option "acceptmismatch" now only affects whether such
objects are stored in the inventory
2017-06-03 16:29:21 +02:00
Scott
f78f1a718b
Change api.py 2017-06-02 18:53:13 -06:00
6044df5adf
Optional storing of expired and off-stream objects
- a new config file option, network/acceptmismatch, allows the inventory
to store objects that expired or are from a stream we're not
interested in. Having this on will prevent re-requesting objects that
other nodes incorrectly advertise. It defaults to false
2017-06-02 15:43:35 +02:00
d75d920a68
Asyncore updates
- clean object tracking dictionaries in the cleaner thread
- clean up close / handle_close
- add locking to tracking dictionaries
2017-06-02 07:09:35 +02:00
4c17a18006
Don't send invs to unestablished connections 2017-05-31 23:34:06 +02:00
18988ae2e6
Asyncore updates
- performance optimisation, reduce number of loops when waiting
for protocol headers / commands
2017-05-31 10:17:36 +02:00
2555f692eb
Network status update part 2
- only update processed items every 2 seconds
2017-05-31 10:16:30 +02:00
e522f015a8
Network status updates
- only update processed numbers once every 2 seconds
- moved inventory lookups to the main inventory so now all
storage modules work with it
2017-05-31 10:15:47 +02:00
d9e3349eeb
Fix own IP detection
- minor bug in the previous commit
2017-05-31 00:22:07 +02:00
f23c169eec
Don't connect to myself
- track local IP+port of incoming connections and don't connect
to them in the future
2017-05-31 00:04:21 +02:00
fa9811f426
Asyncore update
- duplicate checking implemented
- connection pool vs. socket closing cleanup
2017-05-30 23:53:43 +02:00
abaa2c72e5
typo 2017-05-29 15:04:22 +02:00
11d02b1e41
typo 2017-05-29 14:59:42 +02:00
3a543efd83
Typo 2017-05-29 14:54:13 +02:00
7f381c0c25
Asyncore update
- incoming object handling fix
2017-05-29 14:52:31 +02:00
1ccfd41c3f
Asyncore updates
- fix connected to myself check
2017-05-29 14:41:02 +02:00
0aa5dbd958
Asyncore update
- shutdown fix
2017-05-29 14:35:08 +02:00
bdaa939e2c
ENOTSOCK fix 2017-05-29 13:39:26 +02:00
97c44b97f4
Asyncore update
- handle WSAENOTSOCK
2017-05-29 13:14:25 +02:00
a5c1b0c529
Asyncore fixes
- better handling of WSA* checks on non-windows systems
- handle EBADF on Windows/select
- better timeouts / loop lengths in main asyncore loop and
spawning new connections
- remove InvThread prints
2017-05-29 12:56:59 +02:00
74f1a74a8c
Make SO_REUSEPORT optional
- apparently not available on Windows
2017-05-29 11:30:56 +02:00
e7525d47be
Disable memory usage logging
- it looks like it's Unix specific and doesn't work on windows
2017-05-29 11:26:56 +02:00
02a07e5119
Asyncore update
- default to true (original attempt didn't work correctly)
2017-05-29 03:16:14 +02:00
73c41bff9d
typo
-BMConfigParser. instead of BMConfigParser().
2017-05-29 02:30:18 +02:00
65bb6648e7
Asyncore updates
- fix crash in inv thread
- more prints changed into logger
- minor fixes
2017-05-29 00:47:41 +02:00
c85d52b8e8
Asyncore updates
- asyncore is now on by default
- inv announcements implemented
- bandwidth limit implemented / fixed
- stats on download / upload speed now work
- make prints into logger
- limit knownNodes to 20k as it was before
- green light fixed
- other minor fixes
2017-05-29 00:24:07 +02:00
5d4e1e2007
asyncore fixes
- bm headers and commands are only read up to expected length.
On a very fast connection (e.g. local VM), reading verack
also read a part of the TLS handshake
- some debugging info moved from print to logger.debug
- tls handshake cleanup
2017-05-27 22:30:30 +02:00
21f6d38ec2
Asyncore fixes
- TCP fixes
2017-05-27 21:52:56 +02:00
99e714c432
UDP socket bugfixes 2017-05-27 20:43:27 +02:00
fa9ad537a5
Add task_done to asyncore-related queues 2017-05-27 19:39:19 +02:00
f8b4b427fc
Asyncore update
- bugfixes
- UDP socket for local peer discovery
- new function assembleAddr to unify creating address command
- open port checker functionality (inactive)
- sendBigInv is done in a thread separate from the network IO
thread
2017-05-27 19:09:21 +02:00
36b5e2c04f
Inventory storage abstraction
- can have multiple storage types for inventory
- sqlite is the old one, filesystem is a new available
2017-05-27 19:03:27 +02:00
1d87c63504
Traceback on Ctrl-C
- Ctrl-C will print a traceback of all threads instead of
complaining
2017-05-27 19:02:05 +02:00
7b9b7504ad
Don't clean right on startup 2017-05-27 19:01:14 +02:00
d699a28e49
Add variables to errno and socket
- to make sure they work cross platform without having to do
complicated tests
2017-05-27 19:00:19 +02:00
b37a05fd0a
Allow encoding 3 in broadcast API 2017-05-27 07:48:29 +02:00
e309a1edb3
Asyncore update
- separate queue for processing blocking stuff on reception
- rewrote write buffer as a queue
- some addr handling
- number of half open connections correct
2017-05-25 23:04:33 +02:00
51e52401fe
Windows plaform check pythonic
- moved to .startswith instead of 'in'
- thanks @Lvl4sword
2017-05-25 15:00:10 +02:00
9683c879bc
Asyncore update
- Network status UI works but current speed isn't implemented yet
- Track per connection and global transferred bytes
- Add locking to write queue so that other threads can put stuff
there
- send ping on timeout (instead of closing the connection)
- implement open port checker (untested, never triggered yet)
- error handling on IO
2017-05-25 14:59:18 +02:00
edcba9982b
Asyncore getdata processing performance improvement
- no need to query DB for existence of each entry
2017-05-24 21:35:50 +02:00
fa56ab3e6f
Asyncore update
- better error handling
- bug fixes
- remove some debug output
2017-05-24 21:15:36 +02:00
bafdd6a93a
Allow making outbound connections in asyncore 2017-05-24 16:54:33 +02:00
d635e515b9
Big Asyncore update
- most of the stuff is done so it partially works
- disabled pollers other than select (debugging necessary)
- can switch in the settings, section network, option asyncore (defaults
to False)
2017-05-24 16:51:49 +02:00
d498f1c0ae
Configparser update
- add default values for maxdownload/uploadrate, asyncore
- rework error handler slightly
2017-05-24 16:49:16 +02:00
198470f734
Revert parser/renderer max message size
- it's now dealt with during decoding
2017-05-15 12:25:30 +02:00
183f509f09
Decompression limit
- there is now a configurable decompression limit, default at
1MB. Oversize messages are trated as if they never arrived,
just a log entry
2017-05-15 12:23:16 +02:00
9f4a1fa0a4
Config file defaults and address unification
- bmconfigpaser.py now allows to put default values for a specific
option in the file
- addresses as sections are now detected by "BM-" rather than
just ignoring bitmessagesettings. There can now be other sections
with a cleaner config file
2017-05-15 12:18:07 +02:00
660997f8e7
Quick hack for excessively long messages
- only process the first MB of a message for GUI purposes
(parsing/rendering)
2017-05-14 15:40:35 +02:00
fd2603247d
Fix onionbindip for some systems with IPv6
- in some cases when IPv6 stack is available and onionbindip is an IPv4
  address, socket.bind doesn't change the bound address, ending up
  listening on everything
2017-05-10 20:01:23 +02:00
d9d3515905
Node class, WIP
- for new network subsystem
2017-05-07 20:16:49 +02:00
23b9555929
Add TLS version debug info 2017-05-07 20:15:57 +02:00
1c55bf7d4b
Add umsgpack as fallback
- if a "big" msgpack module isn't available, use bundled umsgpack
2017-05-07 20:15:16 +02:00
bf76c7f6ec
Allow extended encoding in API 2017-04-30 10:39:48 +02:00
96d58f3c11
Asyncore update (WIP) 2017-04-16 18:27:15 +02:00
e6f0b34f9b
Fixed some inconvenience on first run mainly in Ubuntu.
- immediately return from initCL() if numpy or pyopencl is unevailable
   (no ImportError because of resetPoW() call)
 - use glob to find C extension even if it named like
   `bitmsghash.x86_64-linux-gnu.so`

If user chooses to show the Settings dialog:

 - activate the "Network Settings" tab
 - remove option 'dontconnect' if settings have been saved
2017-04-05 14:25:02 +03:00
51aeb284ca
Async network IO updates
- WIP
2017-04-04 10:46:01 +02:00
fe93473fc5
getpubkey length handling
- don't try to process getpubkey that is too long
2017-04-04 10:44:53 +02:00
c1bdcc2aba
ACKdata handling changes
- any type of object can now serve as ACKdata
2017-04-04 10:43:29 +02:00
46c9ea9403
Async network updates (WIP)
- cleaner command handling
- separating into header and command handling
- incoming connection handler
- bugfixes and more debug information
2017-03-20 18:32:26 +01:00
913b401dd0
PendingDownloadQueue updates
- track pending hashId more accurately
- add timeout and a cleanup so that the download queues don't
get stuck and memory is freed
- randomise download order (only works for inv commands with
more than 1 entry)
2017-03-20 01:22:37 +01:00
1af49a0165
Download tracking refactoring
- replace PendingDownload singleton dict with a Queue
- total memory and CPU requirements should be reduced
- get rid of somObjectsOfWhichThisRemoteNodeIsAlearedyAware. It has very
little practicle effect and only uses memory
2017-03-19 22:08:00 +01:00
0529fe2313
Known node count updates
- if too many nodes, only delete oldest nodes in bootstrap provider
  mode, in normal mode ignore new nodes as it used to before
- in bootstrap provider mode, penalise nodes announced by others by 1
  day instead of 3 hours
2017-03-11 12:14:40 +01:00
49869d0b56
Networking subsystem updates
- version command struct for faster unpacking
- increase read buffer to 2MB to allow a full command to fit
- initial bitmessage protocol class (WIP)
- error handling
- remove duplicate method
2017-03-11 11:12:08 +01:00
a1d1114cb2
New network subsystem updates
- auto-select select/poll/epoll/kqueue depending on what's available
2017-03-10 23:56:38 +01:00
998935be5f
New network subsystem, WIP
- finished proxy design
- socks4a and socks5 implemented
- authentication not tested
- resolver for both socks4a and socks5
- http client example using the proxy
2017-03-10 23:11:57 +01:00
3ac67e5da7
Connection error reporting changes
- fewer tracebacks
- more standardised reports including errno
2017-03-09 11:26:44 +01:00
PyBitmessage Translations
4014b80b7c
Auto-updated language ja from transifex 2017-03-08 17:25:19 +01:00
PyBitmessage Translations
b28fe3a220
Auto-updated language fr from transifex 2017-03-07 18:38:10 +01:00
PyBitmessage Translations
964809bbd6
Auto-updated language ru from transifex 2017-03-07 10:23:54 +01:00
PyBitmessage Translations
5d545dbcab
Auto-updated language sk from transifex 2017-03-06 22:16:29 +01:00
PyBitmessage Translations
5c2af00306
Auto-updated language de from transifex 2017-03-06 22:06:42 +01:00
PyBitmessage Translations
f4caf5f6a4 Auto-updated language fr from transifex 2017-03-06 21:58:59 +01:00
PyBitmessage Translations
fd871fca85 Auto-updated language ru from transifex 2017-03-06 21:56:38 +01:00
PyBitmessage Translations
76ae62c1f0 Auto-updated language ja from transifex 2017-03-06 21:55:52 +01:00
PyBitmessage Translations
fe6f446d07 Auto-updated language pl from transifex 2017-03-06 21:44:06 +01:00
PyBitmessage Translations
bbf2264862
Auto-updated language eo from transifex 2017-03-06 20:54:39 +01:00
ee7e630694
Show a dialog with QR-code for selected bm-address 2017-03-06 18:00:23 +02:00
9836206402
Missed translations: namecoin and welcom message 2017-03-05 23:07:18 +02:00
bea675f9a6
Remove unnecessary "print" from previous commit 2017-03-02 15:09:01 +01:00
53657dba47
Phase 1 of SHA256 support
- new variable "digestalg" which defaults to "sha1", but allows "sha256"
  for those who want to sign using this
- Addresses #953
2017-03-02 15:03:08 +01:00