bitmessage dont start up any more when database too large or related #524
Assignees
PeterSurda
Clear assignees
PeterSurda
No Assignees
No due date set.
Dependencies
No dependencies set.
Reference: Bitmessage/PyBitmessage-2025-02-27#524
Loading…
x
Reference in New Issue
Block a user
No description provided.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
memory overflow at 1.79gigabytes ram consumption on windows 32bit machine running provided latest win32 binary of bitmessage.exe
bitmessage.exe recently crashed during runtime with qt4...dll gui or similar, and now never starts up any more in the end, as it crashes with unknown memory error whenever it is started up now and taskmanager and other process related tools show that its memory consumption grows up til exactly right below 1.8gigs of ram then it gives this crash. reproducable.
some days ago I think there was a post on the general channel public shared address list talking exactly about this phenomenon back then already
apparently now its happening for me.
bitmessage must not read the total database and keep too many objects in ram, but should only read the database if needed and not store too many objects in ram.
My inventory of objects is currently 84 MB.
What steps should be taken to reproduce this error?
I am subscribed to a lot of lists and addresses. My database file is actually less than 520megabytes on the disk locally. The pybitmessage.log or similar file says its reading slightly more than 14thousand objects in total. so there must be some bloat caused by either the number of objects all in the inbox, the internal representation of those 520megabytes from the disk in the RAM or something related.
subscribe to a lot of stuff and being a long-term bitmessage user will probably show this bug behavior.
is there a way to set debuglevel to more detailed and more extensive level in any way? the log dont show much besides of what i wrote.
It must be the fact that you have so many messages in your inbox and sent folders. They cannot all be displayed in the UI at the same time. The simple fix is trash some of your messages. If you don't want to trash them then the better fix would be for me to change the way Bitmessage stores and displays message content: we could pull it from disk when it needs to be displayed rather than storing it in RAM until the user clicks the message. That would probably fix the problem. I'll do this. Don't trash any messages yet without backing up your messages.dat file- I want to see if this change fixes the problem for you.
@subscribernamegoeshere,
I have coded what I believe is the necessary fix and compiled an EXE for you. This uses the latest code. Would you like to test it?
https://bitmessage.org/download/windows/Bitmessage%20v0.4.1%20-%20test%20improved%20memory%20handling.exe
I would appreciate knowing whether this fixes it for you.
-Atheros
hey thanks for the update, dont get me wrong here, but after all we are talking privacy sensitive stuff, so is that compiled exe also available somewhere as source? is that on github in your own github usernames area from two days ago?
55568fa242thanks for your work and helping.
okay I even tried your exe file on an offline node where the gui wouldnt come up any more and crash, and this .exe version was bringing it up again fine. I did cut the internet connection to this while testing the new .exe
I found an interesting other bug I think: a such offlined node does "never" (waited fore quite many minutes after it started up fine) again connects to the outside world when it started up originally without a route to the external internet. It was just sitting there as a process as listening on its tcp/8884 port or whichever the default one is, but never making connection attempts. I then later shut down the bitmessage again and re-fired it up and then even before the gui came into visibility the listening port was there and it also had some connections established already again and the tray icon was showing that message bubble that bitmessage got connected. So maybe the connecting to other nodes is in some kind of a wrong loop or place in the source code as if it fails completely or early it seems to never try again, or something goes wrong.
As to the gui with this new exe, there is one bug, the reading as html/rendered (for example images) doesnt work any more. Maybe there is some sql statements and queries missing when rightclicking this from the context menu on a message. I tried many messages, large or small, mostly images I think, they never switched over into the colorful html rendered style and never displayed the pixels of the images. only the normal raw text-style message was being shown.
thanks.
Excellent. I'm glad to hear that it is fixing the problem. I will take care of the "View as HTML" feature.
Yes, that is the code used in the EXE. I didn't know that you were sophisticated enough to be using the code from Github; I thought that you were using the EXE.
About the connection issue, Bitmessage will attempt to connect to every known node once every half hour. If it runs out of nodes to try then it will just sit there until the half-hour-mark comes at which time it will try them all again.
okay thanks for the feedback on the re-connection cycle. maybe it could be enhanced to some better logics or algorithm. probably you intially have some bootstrap ip addresses or similar. if all fail, the intervall could be first a few minutes then extending into longer delays if still fail eventually reaching that half an hour although that sounds rather a high penalty when trying to reestablish. thank you.
This issue should be all fixed in the source code. Do let me know if there are any lingering side issues.
Thank you for reporting!
Maybe or very likely(?) related: I am still running this precompiled .exe binary you provided in your comment at
https://github.com/Bitmessage/PyBitmessage/issues/524#issuecomment-26308184
Today I tried to go for compiling bitmessage myself on win32, with latest files:
python-2.7.6.msi
PyQt4-4.10.3-gpl-Py2.7-Qt4.8.5-x32.exe
and the source from master
https://github.com/Bitmessage/PyBitmessage/archive/master.zip
I can start python.exe source\PyBitmessage-master\src\bitmessagemain.py
and it starts to output lot of debug/trace messages and loads the database files and already connects to fellow nodes, but eventually consumes past 1.88 gigabytes of ram (2gigs ram max is the limit for normal 32bit applications on 32bit windows) and then very probably stalls and stops and crashes as it runs out of memory.
It never brings up a visible Gui nor a trayicon or anything else visible, except for the debugmessages in the commandbox window where I start python on that bitmessagemain.py source file.
What now?
The last visible output in the commandline box are:
Could NOT connect to Peer(host='46.159.40.144', port=8444) during outgoing attem
pt. timed out
The length of sendDataQueues at sendDataThread init is: 3
The streamNumber of this sendDataThread (ID: 13596240) at setup() is -1
ID of the receiveDataThread is 13597744. The size of the shared.connectedHostsLi
st is now 3
<singleListener(Thread-14, started daemon 3492)> connected to 82.181.200.132 dur
ing INCOMING request.
remoteCommand 'version' from Peer(host='82.181.200.132', port=50207)
Remote node useragent: /PyBitmessage:0.4.1/ stream number: 1
setting the stream number in the sendData thread (ID: 13596240 ) to 1
setting the remote node's protocol version in the sendData thread (ID: 13596240
) to 2
The length of sendDataQueues at sendDataThread init is: 4
The streamNumber of this sendDataThread (ID: 13598256) at setup() is -1
Exception in thread Thread-14:
Traceback (most recent call last):
File "C:\Python27\lib\threading.py", line 810, in __bootstrap_inner
self.run()
File "C:......\source\PyBitmessage-master\src\class_singleL
istener.py", line 77, in run
sd.start()
File "C:\Python27\lib\threading.py", line 745, in start
_start_new_thread(self.__bootstrap, ())
error: can't start new thread
update: only problem now is after I shut down the hanging python.exe, is that the older binary that you provided doesnt start up (run) any more either. it does start up, spawns the second bitmessage.exe subprocess, then reads some stuff from disk and then dies, both these bitmessage.exe processes :(
https://bitmessage.org/download/windows/Bitmessage%20v0.4.1%20-%20test%20improved%20memory%20handling.exe
so my bitmessage doesnt work any more neither the current master nor that special build from back then.
something got completely corrupted, or the database format changed and the database files on disk were changed in such was that the bitmessage-0.4.1-improvedmemoryhandling cannot cope with the datastructures any more?
Database structures didn't change. The improvement in that binary and in the current source code is that Bitmessage no longer keeps messages in memory but rather loads them off of disk whenever you click a message in your inbox table to display in the inbox message box. It is possible, if you've received tens of thousands of messages recently, that the message meta-information itself is taking up too much memory. If you were subscribed to the General chan then this might be what is happening. Were you subscribed to the General chan? If yes then the way for me to deal with this is to only display X messages in the inbox at at time where X is something like 1000. Perhaps we could display the 1000 newest messages. Having folders would also help this situation which someone else is working on. A smart anti-spam mechanism would be better.
Hello, thanks for the quick reply on this, sorry if i was spamming too many places with that temporary bitmessage setup. yes I am or was on general chan address, and early this week or so there was that huge flood of messages there. Bitmessage was running all the time so its actually funny that it was able to run but when trying to run it freshly it could load its own stuff any more as you say. Maybe its that metadata. Yes a kind of window of N messages being displayed as a rolling window on the total database would probably help. if you release something on github or that master, I could probably run it and test as I have tried to run things from source by now :). regards.
did I miss a fix or patch for this issue, just asking since last time in this issue the fix was being coded really quickly, dont want to annoy the devs. Just making sure I am not overlooking anything here, my bitmessage is offline ever since and dataretention on the network is only two days or so, right? thanks for helping.
Sorry when annoying some more, but just to be able to catch on messages and not actually letting them go to waste, can I somehow run the Bitmessage product with just from command line or in a console, via ssh or similar with no-gui at all, so that it at least receives all the stuff that is happening on the network and I can later again display the stuff once you guys have enhanced or fixed the rest of the Bitmessage functionality? Would help me a lot. Thanks.
P.S. some console client interface (ncurses bitmessage?) would also be kind of cool. Old times such as pine/alpine/elm and more come to my mind.... :)
Any news on this? Offlined many nodes for months now. Thanks.
gave 0.4.2 win32 binary a try, this doesnt crash but also never produces any visible gui, tray icon or anything. it spawns two bitmessage.exe processes, then after these many months during the first attempt to use 0.4.2 .exe from the bitmessage.org website, it wrote some etilqs.... temporary file (sqlite), maybe some database migration or something to %temp%, then it write the messages.dat-log file to the same size as the messages.dat (maybe some database conversion), then the second bitmessage.exe processes started sending syn_sent to all kind of hosts, I left it sending it for half an hour or so but never connection/established. When it was doing these syn_sent and rewriting those database files and removing the etilqs.... file again the second bitmessage.exe processes used 1.9gigs of ram, the maximum possible default for a win32 process on 32bit windows, it didnt crash though as in older builds many months ago.
Eventually I had to reboot the whole machine because only the first smallish bitmessage.exe stub process was reacting to terminate requests. The second attempt to start up 0.4.2 .exe was quicker, also doing some etilqs stuff briefly, then again the second bitmessage.exe was back again quickly at 1.9gigs of ram consumption and sits there ever since, initially doing some syn_sent but it even stopped doing those (as opposed to my first attempt, it contined to do so until I had to reboot the whole machine).
Something is still seriously messed up with current bitmessage 0.4.2
Maybe I need to go for the sources and python interpreter again for 0.4.2 and report some better more details from the console debug log output?
tried to run via python interpreter and the current source via .zip file, it prints a few debug messages about dns bootstrap and a lot of deleting from known knodes because > 48hours, then the ram consumption of python.exe also goes up to 1.9gigs and then suddenly endless of this appears in the console window, probably other messages that I couldnt capture were printed before it started
QPainter::setPen: Painter not active
QPainter:🔚 Painter not active, aborted
QPainter::begin: Paint device returned engine == 0, type: 2
QPainter::setPen: Painter not active
QPainter::translate: Painter not active
QPainter::translate: Painter not active
QPainter::rotate: Painter not active
QPainter::setBrush: Painter not active
QPainter::setPen: Painter not active
QPainter:🔚 Painter not active, aborted
QPainter::begin: Paint device returned engine == 0, type: 2
QPainter::setPen: Painter not active
QPainter::translate: Painter not active
QPainter::translate: Painter not active
QPainter::rotate: Painter not active
QPainter::setBrush: Painter not active
QPainter::setPen: Painter not active
QPainter:🔚 Painter not active, aborted
QPainter::begin: Paint device returned engine == 0, type: 2
then needed to kill it with ctrl+break, python.exe ended
QPainter:🔚 Painter not active, aborted
QPainter::begin: Paint device returned engine == 0, type: 2
QPainter::setPen: Painter not active
QPainter::translate: Painter not active
QPainter::translate: Painter not active
QPainter::rotate: Painter not active
QPainter::setBrush: Painter not active
QPainter::setPen: Painter not active
Unfortunately you cannot use Ctrl+C when running the UI because the UI captures
the signal.
QPainter:🔚 Painter not active, aborted
QPainter::begin: Paint device returned engine == 0, type: 2
QPainter::setPen: Painter not active
QPainter::translate: Painter not active
QPainter::translate: Painter not active
QObject::killTimers: timers cannot be stopped from another thread
^C
0.4.2 is nonworking for my messages database file. my messages.dat is 899megabytes. will bitmessage ever again work with my messages.dat file? :(
more attempts:
as soon as python.exe hits 1.9gigs of ram consumption there is:
....
deleting Peer(host='70.178.61.243', port=55177) from knownNodes because it is m
ore than 48 hours old and we could not connect to it.
Could NOT connect to Peer(host='93.114.44.187', port=8444) during outgoing attem
pt. [Errno 10061] No connection could be made because the target machine activel
y refused it
deleting Peer(host='93.114.44.187', port=8444) from knownNodes because it is mo
re than 48 hours old and we could not connect to it.
Could NOT connect to Peer(host='216.56.161.130', port=32705) during outgoing att
empt. timed out
deleting Peer(host='216.56.161.130', port=32705) from knownNodes because it is
more than 48 hours old and we could not connect to it.
Could NOT connect to Peer(host='212.159.117.248', port=58887) during outgoing at
tempt. timed out
deleting Peer(host='212.159.117.248', port=58887) from knownNodes because it is
more than 48 hours old and we could not connect to it.
QPainter::begin: Paint device returned engine == 0, type: 2
QPainter::setPen: Painter not active
QPainter::translate: Painter not active
QPainter::translate: Painter not active
QPainter::rotate: Painter not active
QPainter::setBrush: Painter not active
QPainter::setPen: Painter not active
QPainter:🔚 Painter not active, aborted
QPainter::begin: Paint device returned engine == 0, type: 2
QPainter::setPen: Painter not active
QPainter::translate: Painter not active
QPainter::translate: Painter not active
QPainter::rotate: Painter not active
QPainter::setBrush: Painter not active
QPainter::setPen: Painter not active
QPainter:🔚 Painter not active, aborted
QPainter::begin: Paint device returned engine == 0, type: 2
Exception in thread Thread-8:
Traceback (most recent call last):
File "C:\Python27\lib\threading.py", line 810, in __bootstrap_inner
self.run()
File "C:\Program Files\Bitmessage\source\PyBitmessage-master\src\class_outgoin
gSynSender.py", line 35, in run
peer, = random.sample(shared.knownNodes[self.streamNumber], 1)
File "C:\Python27\lib\random.py", line 331, in sample
pool = list(population)
MemoryError
QPainter::setPen: Painter not active
QPainter::translate: Painter not active
QPainter::translate: Painter not active
QPainter::rotate: Painter not active
QPainter::setBrush: Painter not active
QPainter::setPen: Painter not active
QPainter:🔚 Painter not active, aborted
QPainter::begin: Paint device returned engine == 0, type: 2
QPainter::setPen: Painter not active
....
QPainter:🔚 Painter not active, aborted
QPainter::begin: Paint device returned engine == 0, type: 2
QPainter::setPen: Painter not active
QPainter::translate: Painter not active
QPainter::translate: Painter not active
QPainter::rotate: Painter not active
QPainter::setBrush: Painter not active
QPainter::setPen: Painter not active
Could NOT connect toQPainter:🔚 Painter not active, aborted
QPainter::begin: Paint device returned engine == 0, type: 2
PQPainter::setPen: Painter not active
eer(host='162.219.179.26', port=8444)QPainter::translate: Painter not active
dQPainter::translate: Painter not active
uring outgoing attempt.QPainter::rotate: Painter not active
QPainter::setBrush: Painter not active
[Errno 10061] No connection could be made because the target machine actively re
fused itQPainter::setPen: Painter not active
QPainter:🔚 Painter not active, aborted
Could NOT connect toUnfortunately you cannot use Ctrl+C when running the UI beca
use the UI captures the signal. Peer(host='203.79.111.218', port=2656)
QPainter::begin: Paint device returned engine == 0, type: 2
dQPainter::setPen: Painter not active
uring outgoing attempt.QPainter::translate: Painter not active
timed outQPainter::translate: Painter not active
QPainter::rotate: Painter not active
CQPainter::setBrush: Painter not active
ould NOT connect toQPainter::setPen: Painter not active
QPainter:🔚 Painter not active, aborted
Peer(host='218.104.71.166', port=8444)QPainter::begin: Paint device returned eng
ine == 0, type: 2
dQPainter::setPen: Painter not active
uring outgoing attempt.QPainter::translate: Painter not active
timed outQPainter::translate: Painter not active
QPainter::rotate: Painter not active
CQPainter::setBrush: Painter not active
ould NOT connect toQPainter::setPen: Painter not active
PQPainter:🔚 Painter not active, aborted
eer(host='46.39.241.111', port=9529) QPainter::begin: Paint device returned engi
ne == 0, type: 2
dQPainter::setPen: Painter not active
uring outgoing attempt.Unfortunately you cannot use Ctrl+C when running the UI b
ecause the UI captures the signal.
tQPainter::translate: Painter not active
QPainter::translate: Painter not active
imed outQPainter::rotate: Painter not active
QPainter::setBrush: Painter not active
CQPainter::setPen: Painter not active
ould NOT connect toQPainter:🔚 Painter not active, aborted
PQPainter::begin: Paint device returned engine == 0, type: 2
eer(host='76.20.160.178', port=8444)QPainter::setPen: Painter not active
during outgoing attempt.QPainter::translate: Painter not active
QPainter::translate: Painter not active
tQPainter::rotate: Painter not active
imed outQPainter::setBrush: Painter not active
QPainter::setPen: Painter not active
QPainter:🔚 Painter not active, aborted
QPainter:🔚 Painter not active, aborted
QObject::killTimers: timers cannot be stopped from another thread
^C
I suspect that it isn't displaying because you have many thousands of messages in your inbox but I'm not sure. Shall I create a branch of the source code which only displays ~250 messages at a time for you to try? You could download it as a source code ZIP and run bitmessagemain.py. If it fixes the problem then we will have solved it; otherwise I'm not sure what the cause could be.
Yes please do so, I already had thought you were already implementing this because of your 0.4.2 change log. I can run the source too via the zip here. Thank you.
tried with 0.4.3 and 0.4.4 now today after having quit in january 2014 refiring up first by running 0.4.3 (after 0.4.2 back then) and letting it work and mess with the local database, but to no avail, it eats up to 1.9 ram after a long while and crashes (win32 process memory limit on x86 windows), and then firing up 0.4.4 and the same crash at 1.9 gig ram consumption. is this ever gonna get fixed? :((((
Try my fork which has a better UI: https://github.com/mailchuck/PyBitmessage. Now each account, subscription and channel have their own list, and you can switch among them in a tree. If the large amount of subscriptions is the cause of your problems, this should make it better, possible even fix it. I plan to merge this UI to main pybitmessage by next stable release. Let me know how it worked out.
@subscribernamegoeshere Can you try the latest release from my fork?
There is a good chance it will help.
Well I got no feedback so I'm closing it. If you need a Win32 executable, let me know and I'll figure something out (I haven't figured out how to install both 32bit and 64bit qt at the same time and this prevents me from being able to create a Win32 bundle).
@subscribernamegoeshere Just a small note, the 0.5.4 binary of mailchuck fork is built for 32bit, and people reported that it works on XP. You may want to try it out.