Network problems, memory, network size #986
Labels
No Label
bug
build
dependencies
developers
documentation
duplicate
enhancement
formatting
invalid
legal
mobile
obsolete
packaging
performance
protocol
question
refactoring
regression
security
test
translation
usability
wontfix
No Milestone
No project
No Assignees
1 Participants
No due date set.
Dependencies
No dependencies set.
Reference: Bitmessage/PyBitmessage-2024-08-21#986
Loading…
Reference in New Issue
Block a user
No description provided.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
I've been having some problems with PC slowing down / stalling (last week), these were fixed by closing BitMessage. There were a lot of big messages waiting to download 135), but setting the allowed transfer rate down seemed to fix it. Bitmessage had suddenly seemed to hog all my Internet bandwidth.
Today that got worse and I ended up taking up 13.6 GB of (32GB) ram. So i have made the issue.
It looks like there is a problem, with large files combined with some sort of DDoS on the network. Can do further checking / information, if you wish to investigate.
Thank you for the report, this is a known issue and there are several tickets open already about it (#964 #870). There are multiple factors causing this (that I know of). You can try the v0.6, there have been some changes that may make it easier for you (it has less memory and CPU usage).
You don't write if you're accepting incoming connections, if yes, then edit keys.dat and add a new variable, maxtotalconnections and set it to maybe 50 or even 30. You may also reduce the bandwidth in network settings, because incoming connections will hog all your bandwidth.
Thanks Peter, I'll try the keys.dat and check out the other reports I missed .. I've checked and I was on (latest?) 0.6.2 is 0.6.3 "out" soon?..
Just FYI :
The number of connections is 142.
I didn't seem to be able to edit the max difficulty down from 5.0 to 4.0. to test if that helped.
It is taking a long, long time to disconnect, The keys.dat was in ~/.config/pybitmessage in Kubuntu.
FYI : after adding maxtotalconnections = 25 to [bitmessagesettings] in keys.dat
(Shame I can't cut and paste the error text along the bottom information panel and had to type this)
Failed to build C PoW module. Please build manually.
I think max difficulty must be at least 1000, but that's for creating PoW.
This is also a related issue, similarly as with the memory hog should be fixed with the new network subsystem.
This means you're missing g++ and/or openssl headers. On Kubuntu, execute as root
apt-get install build-essential libssl-devPlease try the latest code (v0.6 branch) and let me know if you're seeing and improvement. It should be able to handle hundreds of connections on a normal computer now.
Thanks Peter, my BM 0.6.2 currently has 3 GB memory and still slow to close down.
As soon as it closes ... I will update BM and note here if there is any obvious change / improvement.
Thanks again.
Closed down, git pull, git checkout 0.6.2 (already on 0.6.2) -- started up : testing ...
Well on my workstation I now have 44 connections and PyBM uses 561MB RAM. I had up to 75 connections earlier, it needed a little bit more memory but still under 1GB.
Well, it (new 0.6.2 version) seem to be working with less memory and the shut down took seconds not minutes. I'll keep an eye on it, after it's had longer hours to build up transfers.
Thanks again.
Just checking next day. Everything tests better, thanks for the update.
Closed down : few seconds.
Memory running at 220 MB.
Earlier today I got 100 connections on linux with about 680MB RAM.
Same here, network bandwidth consuming looks excessive. I add my stats for this issue, lest create a new one.
Version: 0.6.3.2-529559d (latest I could get via 'git pull')
OS: Ubuntu 16.04.4 64-bit
Bandwidth usage (Up/Down): 697Mb/1124Mb after 7 hours of being online.
Network speed limits: (Up/Down): 48/96 kB/s
Some keys.dat parameters:
maxoutboundconnections = 8
Default ports (TCP/UDP 8444) are open, incoming connections accepted. At the moment of writing this, 16 incoming and 5 outgoing connections to other BM nodes exist.
I would appreciate developers' efforts to prevent bandwidth hogging, or at least providing instructions on how to see detailed stats (which data type consumes how much bandwidth). Thanks.
That was an "attack" on the network that was fixed over a year ago. Have you tried upgrading, my version is 0.6.3.2 and there is settings / network settings where you can also set a max bandwidth to allocate to bitmessage.
(I'm not a dev)
@wrapperband In fact, I am using the latest pullable version (I mentioned its exact ID), so if something was fixed, it's either a regression, or a new bug.
Also, I have set bandwidth limit (please read my comment), but even with it the bandwidth usage is quite unrealistic, unless clients are requesting public keys thousand times in a row, without actual necessity.
If you're accepting incoming connections, the current protocol design will cause a lot of bandwidth consumption. There may be some bugs that exacerbate bandwith usage but mainly it's the inv exchange that happens right after the protocol handshake that's consuming bandwidth. There isn't much that can be done in short term.
@PeterSurda I see. Perhaps this should be explicitly said in FAQ/wherever, otherwise there will be more users complaining at high bandwidth usage.
In any case, I would still suggest creating a stats feature that could show the percentage of different traffic type (AFAIK, that doesn't compromise anyone's security).
Thanks.
This is for practical purposes fixed, there are no more known urgent issues in the current code. There is still some performance penalty due to slicing write / read buffers, and the protocol overhead that requires to send all invs. This will be addressed separately.