Bitmessage/PyBitmessage-2024-08-21
Archived
2
0
Fork 0
Code Issues 168 Pull Requests 53 Actions Packages Projects Releases 5 Wiki Activity

python uses 100% CPU #183

New Issue
Closed
opened 2013-06-03 21:10:50 +02:00 by rubo77 · 40 comments
rubo77 commented 2013-06-03 21:10:50 +02:00 (Migrated from github.com)
Copy Link

On my Xubuntu 12.10 with xfce-desktop sometimes (not always) pyBitmessage causes a python process that uses 100% of one complete CPU:

#ps aux|grep bitmes

ruben     2703 88.3  1.7 1870944 70304 ?       Sl   20:44  19:14 python bitmessagemain.py

If I close the GUI, that process stays and continues to waste the CPU

only

#kill -kill 2703

stops bitmessage and releases the CPU

On my Xubuntu 12.10 with xfce-desktop sometimes (not always) pyBitmessage causes a python process that uses 100% of one complete CPU: ``` #ps aux|grep bitmes ``` --- ``` ruben 2703 88.3 1.7 1870944 70304 ? Sl 20:44 19:14 python bitmessagemain.py ``` --- If I close the GUI, that process stays and continues to waste the CPU only ``` #kill -kill 2703 ``` stops bitmessage and releases the CPU
ghost commented 2013-06-03 21:33:46 +02:00 (Migrated from github.com)
Copy Link

I get something similar although it's not repeatable and there can be many hours of normal negligible processor use before it occurs. Probably some benchmarking code should be added to find out what is causing this, because it's the one thing which is stopping the application from being really practical.

I get something similar although it's not repeatable and there can be many hours of normal negligible processor use before it occurs. Probably some benchmarking code should be added to find out what is causing this, because it's the one thing which is stopping the application from being really practical.
👍 1
acejam commented 2013-06-04 18:02:24 +02:00 (Migrated from github.com)
Copy Link

I'm also seeing this on Ubuntu 12.04 LTS x64 w/ Python 2.7.3. However, I'm running "daemon = true" and CPU usage continues to climb.

I'm also seeing this on Ubuntu 12.04 LTS x64 w/ Python 2.7.3. However, I'm running "daemon = true" and CPU usage continues to climb.
JamesTheAwesomeDude commented 2013-06-12 16:13:43 +02:00 (Migrated from github.com)
Copy Link

Same here; 12.04 LTS x64. Also tested on Xubuntu 13.04 x86. Problems on both OS's.

Same here; 12.04 LTS x64. Also tested on Xubuntu 13.04 x86. Problems on both OS's.
svaens commented 2013-06-12 16:50:23 +02:00 (Migrated from github.com)
Copy Link

yep, me too.
top even somehow reported 140 percent cpu.'
how is that even possible?

yep, me too. top even somehow reported 140 percent cpu.' how is that even possible?
acejam commented 2013-06-12 18:53:32 +02:00 (Migrated from github.com)
Copy Link

I'm seeing 300-400% CPU usage in top.

It's possible because top shows you 100% per core. So if you have an 8 core CPU, 800% is possible. An 8.0 system load would also be 100% load on an 8 core system.

I'm seeing 300-400% CPU usage in top. It's possible because top shows you 100% _per core_. So if you have an 8 core CPU, 800% is possible. An 8.0 system load would also be 100% load on an 8 core system.
Atheros1 commented 2013-06-12 18:53:35 +02:00 (Migrated from github.com)
Copy Link

Does the CPU usage ever go back down or does it stay at 100% as long as you leave it there?

Can anyone tell if the high CPU usage corresponds with new connections? Is there high CPU usage even when your connection count has been stable at 8?

Does the CPU usage ever go back down or does it stay at 100% as long as you leave it there? Can anyone tell if the high CPU usage corresponds with new connections? Is there high CPU usage even when your connection count has been stable at 8?
bytting commented 2013-06-18 13:11:36 +02:00 (Migrated from github.com)
Copy Link

I'm having a similar problem using 0.3.3-2 from git.

Most of the time when I start up bitmessage the CPU load is negligible and so is the messages and keys processed per minute. However, at some point the CPU usage went to the roof and it corresponded with messages and keys processed, not the number of connections, which was about 8 all the time as far as I can remember.

I'm behind a firewall and running with yellow status.

It may not be a bug but I'll post back here if it happens again

I'm having a similar problem using 0.3.3-2 from git. Most of the time when I start up bitmessage the CPU load is negligible and so is the messages and keys processed per minute. However, at some point the CPU usage went to the roof and it corresponded with messages and keys processed, not the number of connections, which was about 8 all the time as far as I can remember. I'm behind a firewall and running with yellow status. It may not be a bug but I'll post back here if it happens again
Atheros1 commented 2013-06-18 18:33:45 +02:00 (Migrated from github.com)
Copy Link

@corebob
There is a maximum number of messages that your client will process (~8000) after which point it will be caught up and should have no reason to be using CPU to process messages. How high does the number of messages processed climb? It is still using CPU after it finishes catching up?

@corebob There is a maximum number of messages that your client will process (~8000) after which point it will be caught up and should have no reason to be using CPU to process messages. How high does the number of messages processed climb? It is still using CPU after it finishes catching up?
bytting commented 2013-06-19 10:50:01 +02:00 (Migrated from github.com)
Copy Link

I'll have to check that if it happens again since I restarted the client when there was less than 1000 messages processed. Once I restarted it worked much slower, like 2 messages processed per minute and practially no CPU usage. I'll just have to keep an eye out and see if it slows down after a while next time it happens.
Is it correct to assume that the python client uses one thread at full throttle, potentially occupying 100% of a single core?

I'll have to check that if it happens again since I restarted the client when there was less than 1000 messages processed. Once I restarted it worked much slower, like 2 messages processed per minute and practially no CPU usage. I'll just have to keep an eye out and see if it slows down after a while next time it happens. Is it correct to assume that the python client uses one thread at full throttle, potentially occupying 100% of a single core?
rubo77 commented 2013-06-19 10:57:21 +02:00 (Migrated from github.com)
Copy Link

I noticed, that bitmessage never really quits, if I click the X Button at the window.

I always have to kill it afterwards with:

kill -kill $(pgrep python)
I noticed, that bitmessage never really quits, if I click the X Button at the window. I always have to kill it afterwards with: ``` kill -kill $(pgrep python) ```
ghost commented 2013-06-19 17:55:07 +02:00 (Migrated from github.com)
Copy Link

What does processing messages involve?

What does processing messages involve?
acejam commented 2013-06-19 18:21:28 +02:00 (Migrated from github.com)
Copy Link

My CPU usage is constantly spiked, even after all messages have been processed. I'm running in daemon mode so I have no way to see how many messages have been processed, but I suspect processing is complete because this will happen for weeks/days and never goes down to normal CPU levels.

My CPU usage is constantly spiked, even after all messages have been processed. I'm running in daemon mode so I have no way to see how many messages have been processed, but I suspect processing is complete because this will happen for weeks/days and never goes down to normal CPU levels.
Atheros1 commented 2013-06-19 21:07:59 +02:00 (Migrated from github.com)
Copy Link

To people who experience the high-CPU usage issue,
In daemon mode, when a large 'inv' message is received, the current version of the software prints lines just like these:

number of keys(hosts) in numberOfObjectsThatWeHaveYetToCheckAndSeeWhetherWeAlreadyHavePerPeer: 1
totalNumberOfObjectsThatWeHaveYetToCheckAndSeeWhetherWeAlreadyHave =  8731

When your CPU usage is high, is your "numberOfObjectsThatWeHaveYetToCheckAndSeeWhetherWeAlreadyHavePerPeer" low, like 0 or 1 or 2? Or does it remain high? I'm wondering if Linux clients have trouble getting the totalNumberOfObjectsThatWeHaveYetToCheckAndSeeWhetherWeAlreadyHave down to zero. (The current code doesn't print when you hit zero so just getting low is good enough). On my machine it climbs into the the tens of thousands when initially connecting but then goes back down as my machine checks to see which objects it already has in the inventory. Please let me know.

To people who experience the high-CPU usage issue, In daemon mode, when a large 'inv' message is received, the current version of the software prints lines just like these: ``` number of keys(hosts) in numberOfObjectsThatWeHaveYetToCheckAndSeeWhetherWeAlreadyHavePerPeer: 1 totalNumberOfObjectsThatWeHaveYetToCheckAndSeeWhetherWeAlreadyHave = 8731 ``` When your CPU usage is high, is your "numberOfObjectsThatWeHaveYetToCheckAndSeeWhetherWeAlreadyHavePerPeer" low, like 0 or 1 or 2? Or does it remain high? I'm wondering if Linux clients have trouble getting the totalNumberOfObjectsThatWeHaveYetToCheckAndSeeWhetherWeAlreadyHave down to zero. (The current code doesn't print when you hit zero so just getting low is good enough). On my machine it climbs into the the tens of thousands when initially connecting but then goes back down as my machine checks to see which objects it already has in the inventory. Please let me know.
ghost commented 2013-06-19 21:32:02 +02:00 (Migrated from github.com)
Copy Link

Sample output when CPU usage is near 100% persistently.

addr message contains 1 IP addresses.
knownNodes currently has 924 nodes for this stream.
remoteCommand 'inv' from 84.58.247.85
sending getdata to retrieve object with hash: a1dfdac227c4c6a7b4399add53c19c036ef8e521c15522e3f41c1031686bd80f
remoteCommand 'msg' from 84.58.247.85
broadcasting inv with hash: a1dfdac227c4c6a7b4399add53c19c036ef8e521c15522e3f41c1031686bd80f
This was NOT an acknowledgement bound for me.
Length of time program spent failing to decrypt this message: 0.0142960548401 seconds.
Timing attack mitigation: Sleeping for 0.585553979874 seconds.
remoteCommand 'getdata' from 68.113.120.119
received getdata request for item: a1dfdac227c4c6a7b4399add53c19c036ef8e521c15522e3f41c1031686bd80f
sending msg
Total message processing time: 0.600827932358 seconds.
remoteCommand 'addr' from 216.14.37.242
addr message contains 3 IP addresses.
knownNodes currently has 924 nodes for this stream.
remoteCommand 'inv' from 92.25.250.208
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 74.74.103.120
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 81.98.253.250
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 2.134.245.21
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 200.75.101.249
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 94.40.96.218
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 46.223.18.202
Inventory (in memory) has inventory item already.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 924 nodes for this stream.
remoteCommand 'inv' from 46.208.242.157
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 184.79.4.90
Inventory (in memory) has inventory item already.
remoteCommand 'addr' from 184.79.4.90
addr message contains 1 IP addresses.
knownNodes currently has 924 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 924 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 924 nodes for this stream.
remoteCommand 'addr' from 92.25.250.208
addr message contains 1 IP addresses.
added new node 88.207.43.94 to knownNodes in stream 1
Broadcasting addr with 1 entries.
remoteCommand 'addr' from 74.74.103.120
addr message contains 1 IP addresses.
knownNodes currently has 925 nodes for this stream.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 184.79.4.90
addr message contains 1 IP addresses.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 84.58.247.85
addr message contains 1 IP addresses.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 200.75.101.249
addr message contains 1 IP addresses.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 94.40.96.218
addr message contains 1 IP addresses.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 46.208.242.157
addr message contains 1 IP addresses.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 68.113.120.119
addr message contains 1 IP addresses.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 2.134.245.21
addr message contains 1 IP addresses.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 2 IP addresses.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 46.223.18.202
addr message contains 1 IP addresses.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 81.98.253.250
addr message contains 1 IP addresses.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 925 nodes for this stream.
remoteCommand 'addr' from 200.75.101.249
addr message contains 1 IP addresses.
added new node 92.30.239.3 to knownNodes in stream 1
Broadcasting addr with 1 entries.
knownNodes currently has 926 nodes for this stream.
remoteCommand 'addr' from 2.134.245.21
addr message contains 1 IP addresses.
knownNodes currently has 926 nodes for this stream.
remoteCommand 'addr' from 84.58.247.85
addr message contains 1 IP addresses.
knownNodes currently has 926 nodes for this stream.
remoteCommand 'addr' from 68.113.120.119
addr message contains 1 IP addresses.
knownNodes currently has 926 nodes for this stream.
remoteCommand 'addr' from 184.79.4.90
addr message contains 1 IP addresses.
knownNodes currently has 926 nodes for this stream.
remoteCommand 'addr' from 92.25.250.208
addr message contains 1 IP addresses.
knownNodes currently has 926 nodes for this stream.
remoteCommand 'addr' from 46.208.242.157
addr message contains 1 IP addresses.
knownNodes currently has 926 nodes for this stream.
remoteCommand 'addr' from 94.40.96.218
addr message contains 1 IP addresses.
knownNodes currently has 926 nodes for this stream.
remoteCommand 'addr' from 46.223.18.202
addr message contains 1 IP addresses.
knownNodes currently has 926 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 926 nodes for this stream.
remoteCommand 'addr' from 74.74.103.120
addr message contains 1 IP addresses.
knownNodes currently has 926 nodes for this stream.
remoteCommand 'addr' from 81.98.253.250
addr message contains 1 IP addresses.
knownNodes currently has 926 nodes for this stream.
remoteCommand 'addr' from 74.74.103.120
addr message contains 1 IP addresses.
knownNodes currently has 926 nodes for this stream.
remoteCommand 'addr' from 92.25.250.208
addr message contains 1 IP addresses.
added new node 5.135.85.67 to knownNodes in stream 1
Broadcasting addr with 1 entries.
knownNodes currently has 927 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 927 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 927 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 927 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 927 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 927 nodes for this stream.
remoteCommand 'inv' from 84.58.247.85
sending getdata to retrieve object with hash: b7c4982b720e0b09c23ca032409ded9c207ee235adeea2d8ab8f573e5a6511b8
remoteCommand 'pubkey' from 84.58.247.85
broadcasting inv with hash: b7c4982b720e0b09c23ca032409ded9c207ee235adeea2d8ab8f573e5a6511b8
ECDSA verify passed (within processpubkey)
within recpubkey, addressVersion: 3 , streamNumber: 1
ripe 00f60c70e83f28883b701079ce75db5aa3f2fcda
publicSigningKey in hex: 04098772c18d34276f764d2a811fd8591ca4d88cf3cfed49e1f657cadd75f65af540d475519751efa89d4751ad8dffa1665dcdf5e7fb0dfbfb0041a82150db7668
publicEncryptionKey in hex: 04fb56f359afdf2b964f50a228dd57a6f8e81bf889d5efab0a3888e38b29ab05e66c81b9a38710f8a14dcc7afa44017604015efd9e5017d286e681f0da3e6cca40
We have NOT used this pubkey personally. Inserting in database.
We don't need this pub key. We didn't ask for it. Pubkey hash: 00f60c70e83f28883b701079ce75db5aa3f2fcda
Timing attack mitigation: Sleeping for 0.192322921753 seconds.
Total pubkey processing time: 0.200307130814 seconds.
remoteCommand 'getdata' from 94.40.96.218
received getdata request for item: b7c4982b720e0b09c23ca032409ded9c207ee235adeea2d8ab8f573e5a6511b8
sending pubkey
remoteCommand 'inv' from 184.79.4.90
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 2.134.245.21
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 46.208.242.157
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 46.223.18.202
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 74.74.103.120
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 81.98.253.250
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 216.14.37.242
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 68.113.120.119
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 92.25.250.208
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 200.75.101.249
Inventory (in memory) has inventory item already.
Status bar: Doing housekeeping (Flushing inventory in memory to disk...)
remoteCommand 'inv' from 84.58.247.85
sending getdata to retrieve object with hash: f43ae6c998a7fe5621b65efea8b52fce06d682e97a9bcc59ff2a0cbbe89b6f6f
remoteCommand 'inv' from 200.75.101.249
sending getdata to retrieve object with hash: f43ae6c998a7fe5621b65efea8b52fce06d682e97a9bcc59ff2a0cbbe89b6f6f
remoteCommand 'msg' from 84.58.247.85
broadcasting inv with hash: f43ae6c998a7fe5621b65efea8b52fce06d682e97a9bcc59ff2a0cbbe89b6f6f
This was NOT an acknowledgement bound for me.
Length of time program spent failing to decrypt this message: 0.0100858211517 seconds.
Timing attack mitigation: Sleeping for 0.589886045456 seconds.
remoteCommand 'msg' from 200.75.101.249
We have already received this msg message. Ignoring.
Total message processing time: 0.600682020187 seconds.
remoteCommand 'inv' from 46.223.18.202
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 46.223.18.202
sending getdata to retrieve object with hash: d8933a92d7a261f06c38c6897e2fdbf8f6158dcfe46bd9786d240093ad61105a
remoteCommand 'msg' from 46.223.18.202
broadcasting inv with hash: d8933a92d7a261f06c38c6897e2fdbf8f6158dcfe46bd9786d240093ad61105a
This was NOT an acknowledgement bound for me.
Length of time program spent failing to decrypt this message: 0.00192499160767 seconds.
Timing attack mitigation: Sleeping for 0.597921943665 seconds.
Total message processing time: 0.600708961487 seconds.
remoteCommand 'inv' from 68.113.120.119
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 216.14.37.242
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 92.25.250.208
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 92.25.250.208
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 46.208.242.157
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 46.208.242.157
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 2.134.245.21
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 74.74.103.120
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 184.79.4.90
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 81.98.253.250
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 2.134.245.21
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 94.40.96.218
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 200.75.101.249
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 74.74.103.120
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 68.113.120.119
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 84.58.247.85
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 184.79.4.90
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 216.14.37.242
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 81.98.253.250
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 184.79.4.90
sending getdata to retrieve object with hash: 8bb5a02536a018c4a2d7accdb6259b4a8b36532b194f51d3b79694302c6dc625
remoteCommand 'inv' from 92.25.250.208
sending getdata to retrieve object with hash: 8bb5a02536a018c4a2d7accdb6259b4a8b36532b194f51d3b79694302c6dc625
remoteCommand 'broadcast' from 92.25.250.208
broadcasting inv with hash: 8bb5a02536a018c4a2d7accdb6259b4a8b36532b194f51d3b79694302c6dc625
Length of time program spent failing to decrypt this v2 broadcast: 0.0243439674377 seconds.
Timing attack mitigation: Sleeping for 0.575584983826 seconds.
remoteCommand 'broadcast' from 184.79.4.90
We have already received this broadcast object. Ignoring.
remoteCommand 'inv' from 46.208.242.157
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 68.113.120.119
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 216.14.37.242
Inventory (in memory) has inventory item already.
Total message processing time: 0.600677013397 seconds.
remoteCommand 'inv' from 92.25.250.208
sending getdata to retrieve object with hash: f44e6c426d25c117fe519fae0017491b07009748388dc4f9d7339323303f84aa
remoteCommand 'msg' from 92.25.250.208
broadcasting inv with hash: f44e6c426d25c117fe519fae0017491b07009748388dc4f9d7339323303f84aa
This was NOT an acknowledgement bound for me.
Length of time program spent failing to decrypt this message: 0.010861158371 seconds.
Timing attack mitigation: Sleeping for 0.589054918289 seconds.
remoteCommand 'inv' from 2.134.245.21
Inventory (in memory) has inventory item already.
Total message processing time: 0.600691080093 seconds.
remoteCommand 'inv' from 94.40.96.218
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 216.14.37.242
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 46.223.18.202
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 84.58.247.85
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 216.14.37.242
sending getdata to retrieve object with hash: 3662abebe280d51d24d86833f5287d1c169139468c36f670f1471f5e4ab20ea8
remoteCommand 'inv' from 2.134.245.21
Inventory (in memory) has inventory item already.
remoteCommand 'broadcast' from 216.14.37.242
broadcasting inv with hash: 3662abebe280d51d24d86833f5287d1c169139468c36f670f1471f5e4ab20ea8
Length of time program spent failing to decrypt this v2 broadcast: 0.0221109390259 seconds.
Timing attack mitigation: Sleeping for 0.577710962296 seconds.
remoteCommand 'inv' from 200.75.101.249
Inventory (in memory) has inventory item already.
Total message processing time: 0.600991010666 seconds.
remoteCommand 'inv' from 46.208.242.157
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 74.74.103.120
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 200.75.101.249
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 2.134.245.21
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 46.223.18.202
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 81.98.253.250
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 68.113.120.119
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 200.75.101.249
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 84.58.247.85
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 46.208.242.157
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 84.58.247.85
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 94.40.96.218
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 184.79.4.90
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 92.25.250.208
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 81.98.253.250
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 46.223.18.202
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 68.113.120.119
Inventory (in memory) has inventory item already.
remoteCommand 'inv' from 74.74.103.120
Inventory (in memory) has inventory item already.
remoteCommand 'addr' from 94.40.96.218
addr message contains 1 IP addresses.
knownNodes currently has 927 nodes for this stream.
remoteCommand 'addr' from 92.25.250.208
addr message contains 1 IP addresses.
knownNodes currently has 927 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 927 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 927 nodes for this stream.
remoteCommand 'addr' from 92.25.250.208
addr message contains 1 IP addresses.
added new node 72.83.194.254 to knownNodes in stream 1
Broadcasting addr with 1 entries.
knownNodes currently has 928 nodes for this stream.
remoteCommand 'addr' from 216.14.37.242
addr message contains 1 IP addresses.
knownNodes currently has 928 nodes for this stream.

Sample output when CPU usage is near 100% persistently. addr message contains 1 IP addresses. knownNodes currently has 924 nodes for this stream. remoteCommand 'inv' from 84.58.247.85 sending getdata to retrieve object with hash: a1dfdac227c4c6a7b4399add53c19c036ef8e521c15522e3f41c1031686bd80f remoteCommand 'msg' from 84.58.247.85 broadcasting inv with hash: a1dfdac227c4c6a7b4399add53c19c036ef8e521c15522e3f41c1031686bd80f This was NOT an acknowledgement bound for me. Length of time program spent failing to decrypt this message: 0.0142960548401 seconds. Timing attack mitigation: Sleeping for 0.585553979874 seconds. remoteCommand 'getdata' from 68.113.120.119 received getdata request for item: a1dfdac227c4c6a7b4399add53c19c036ef8e521c15522e3f41c1031686bd80f sending msg Total message processing time: 0.600827932358 seconds. remoteCommand 'addr' from 216.14.37.242 addr message contains 3 IP addresses. knownNodes currently has 924 nodes for this stream. remoteCommand 'inv' from 92.25.250.208 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 74.74.103.120 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 81.98.253.250 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 2.134.245.21 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 200.75.101.249 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 94.40.96.218 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 46.223.18.202 Inventory (in memory) has inventory item already. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 924 nodes for this stream. remoteCommand 'inv' from 46.208.242.157 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 184.79.4.90 Inventory (in memory) has inventory item already. remoteCommand 'addr' from 184.79.4.90 addr message contains 1 IP addresses. knownNodes currently has 924 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 924 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 924 nodes for this stream. remoteCommand 'addr' from 92.25.250.208 addr message contains 1 IP addresses. added new node 88.207.43.94 to knownNodes in stream 1 Broadcasting addr with 1 entries. remoteCommand 'addr' from 74.74.103.120 addr message contains 1 IP addresses. knownNodes currently has 925 nodes for this stream. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 184.79.4.90 addr message contains 1 IP addresses. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 84.58.247.85 addr message contains 1 IP addresses. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 200.75.101.249 addr message contains 1 IP addresses. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 94.40.96.218 addr message contains 1 IP addresses. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 46.208.242.157 addr message contains 1 IP addresses. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 68.113.120.119 addr message contains 1 IP addresses. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 2.134.245.21 addr message contains 1 IP addresses. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 2 IP addresses. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 46.223.18.202 addr message contains 1 IP addresses. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 81.98.253.250 addr message contains 1 IP addresses. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 925 nodes for this stream. remoteCommand 'addr' from 200.75.101.249 addr message contains 1 IP addresses. added new node 92.30.239.3 to knownNodes in stream 1 Broadcasting addr with 1 entries. knownNodes currently has 926 nodes for this stream. remoteCommand 'addr' from 2.134.245.21 addr message contains 1 IP addresses. knownNodes currently has 926 nodes for this stream. remoteCommand 'addr' from 84.58.247.85 addr message contains 1 IP addresses. knownNodes currently has 926 nodes for this stream. remoteCommand 'addr' from 68.113.120.119 addr message contains 1 IP addresses. knownNodes currently has 926 nodes for this stream. remoteCommand 'addr' from 184.79.4.90 addr message contains 1 IP addresses. knownNodes currently has 926 nodes for this stream. remoteCommand 'addr' from 92.25.250.208 addr message contains 1 IP addresses. knownNodes currently has 926 nodes for this stream. remoteCommand 'addr' from 46.208.242.157 addr message contains 1 IP addresses. knownNodes currently has 926 nodes for this stream. remoteCommand 'addr' from 94.40.96.218 addr message contains 1 IP addresses. knownNodes currently has 926 nodes for this stream. remoteCommand 'addr' from 46.223.18.202 addr message contains 1 IP addresses. knownNodes currently has 926 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 926 nodes for this stream. remoteCommand 'addr' from 74.74.103.120 addr message contains 1 IP addresses. knownNodes currently has 926 nodes for this stream. remoteCommand 'addr' from 81.98.253.250 addr message contains 1 IP addresses. knownNodes currently has 926 nodes for this stream. remoteCommand 'addr' from 74.74.103.120 addr message contains 1 IP addresses. knownNodes currently has 926 nodes for this stream. remoteCommand 'addr' from 92.25.250.208 addr message contains 1 IP addresses. added new node 5.135.85.67 to knownNodes in stream 1 Broadcasting addr with 1 entries. knownNodes currently has 927 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 927 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 927 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 927 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 927 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 927 nodes for this stream. remoteCommand 'inv' from 84.58.247.85 sending getdata to retrieve object with hash: b7c4982b720e0b09c23ca032409ded9c207ee235adeea2d8ab8f573e5a6511b8 remoteCommand 'pubkey' from 84.58.247.85 broadcasting inv with hash: b7c4982b720e0b09c23ca032409ded9c207ee235adeea2d8ab8f573e5a6511b8 ECDSA verify passed (within processpubkey) within recpubkey, addressVersion: 3 , streamNumber: 1 ripe 00f60c70e83f28883b701079ce75db5aa3f2fcda publicSigningKey in hex: 04098772c18d34276f764d2a811fd8591ca4d88cf3cfed49e1f657cadd75f65af540d475519751efa89d4751ad8dffa1665dcdf5e7fb0dfbfb0041a82150db7668 publicEncryptionKey in hex: 04fb56f359afdf2b964f50a228dd57a6f8e81bf889d5efab0a3888e38b29ab05e66c81b9a38710f8a14dcc7afa44017604015efd9e5017d286e681f0da3e6cca40 We have NOT used this pubkey personally. Inserting in database. We don't need this pub key. We didn't ask for it. Pubkey hash: 00f60c70e83f28883b701079ce75db5aa3f2fcda Timing attack mitigation: Sleeping for 0.192322921753 seconds. Total pubkey processing time: 0.200307130814 seconds. remoteCommand 'getdata' from 94.40.96.218 received getdata request for item: b7c4982b720e0b09c23ca032409ded9c207ee235adeea2d8ab8f573e5a6511b8 sending pubkey remoteCommand 'inv' from 184.79.4.90 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 2.134.245.21 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 46.208.242.157 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 46.223.18.202 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 74.74.103.120 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 81.98.253.250 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 216.14.37.242 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 68.113.120.119 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 92.25.250.208 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 200.75.101.249 Inventory (in memory) has inventory item already. Status bar: Doing housekeeping (Flushing inventory in memory to disk...) remoteCommand 'inv' from 84.58.247.85 sending getdata to retrieve object with hash: f43ae6c998a7fe5621b65efea8b52fce06d682e97a9bcc59ff2a0cbbe89b6f6f remoteCommand 'inv' from 200.75.101.249 sending getdata to retrieve object with hash: f43ae6c998a7fe5621b65efea8b52fce06d682e97a9bcc59ff2a0cbbe89b6f6f remoteCommand 'msg' from 84.58.247.85 broadcasting inv with hash: f43ae6c998a7fe5621b65efea8b52fce06d682e97a9bcc59ff2a0cbbe89b6f6f This was NOT an acknowledgement bound for me. Length of time program spent failing to decrypt this message: 0.0100858211517 seconds. Timing attack mitigation: Sleeping for 0.589886045456 seconds. remoteCommand 'msg' from 200.75.101.249 We have already received this msg message. Ignoring. Total message processing time: 0.600682020187 seconds. remoteCommand 'inv' from 46.223.18.202 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 46.223.18.202 sending getdata to retrieve object with hash: d8933a92d7a261f06c38c6897e2fdbf8f6158dcfe46bd9786d240093ad61105a remoteCommand 'msg' from 46.223.18.202 broadcasting inv with hash: d8933a92d7a261f06c38c6897e2fdbf8f6158dcfe46bd9786d240093ad61105a This was NOT an acknowledgement bound for me. Length of time program spent failing to decrypt this message: 0.00192499160767 seconds. Timing attack mitigation: Sleeping for 0.597921943665 seconds. Total message processing time: 0.600708961487 seconds. remoteCommand 'inv' from 68.113.120.119 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 216.14.37.242 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 92.25.250.208 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 92.25.250.208 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 46.208.242.157 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 46.208.242.157 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 2.134.245.21 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 74.74.103.120 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 184.79.4.90 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 81.98.253.250 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 2.134.245.21 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 94.40.96.218 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 200.75.101.249 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 74.74.103.120 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 68.113.120.119 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 84.58.247.85 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 184.79.4.90 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 216.14.37.242 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 81.98.253.250 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 184.79.4.90 sending getdata to retrieve object with hash: 8bb5a02536a018c4a2d7accdb6259b4a8b36532b194f51d3b79694302c6dc625 remoteCommand 'inv' from 92.25.250.208 sending getdata to retrieve object with hash: 8bb5a02536a018c4a2d7accdb6259b4a8b36532b194f51d3b79694302c6dc625 remoteCommand 'broadcast' from 92.25.250.208 broadcasting inv with hash: 8bb5a02536a018c4a2d7accdb6259b4a8b36532b194f51d3b79694302c6dc625 Length of time program spent failing to decrypt this v2 broadcast: 0.0243439674377 seconds. Timing attack mitigation: Sleeping for 0.575584983826 seconds. remoteCommand 'broadcast' from 184.79.4.90 We have already received this broadcast object. Ignoring. remoteCommand 'inv' from 46.208.242.157 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 68.113.120.119 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 216.14.37.242 Inventory (in memory) has inventory item already. Total message processing time: 0.600677013397 seconds. remoteCommand 'inv' from 92.25.250.208 sending getdata to retrieve object with hash: f44e6c426d25c117fe519fae0017491b07009748388dc4f9d7339323303f84aa remoteCommand 'msg' from 92.25.250.208 broadcasting inv with hash: f44e6c426d25c117fe519fae0017491b07009748388dc4f9d7339323303f84aa This was NOT an acknowledgement bound for me. Length of time program spent failing to decrypt this message: 0.010861158371 seconds. Timing attack mitigation: Sleeping for 0.589054918289 seconds. remoteCommand 'inv' from 2.134.245.21 Inventory (in memory) has inventory item already. Total message processing time: 0.600691080093 seconds. remoteCommand 'inv' from 94.40.96.218 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 216.14.37.242 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 46.223.18.202 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 84.58.247.85 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 216.14.37.242 sending getdata to retrieve object with hash: 3662abebe280d51d24d86833f5287d1c169139468c36f670f1471f5e4ab20ea8 remoteCommand 'inv' from 2.134.245.21 Inventory (in memory) has inventory item already. remoteCommand 'broadcast' from 216.14.37.242 broadcasting inv with hash: 3662abebe280d51d24d86833f5287d1c169139468c36f670f1471f5e4ab20ea8 Length of time program spent failing to decrypt this v2 broadcast: 0.0221109390259 seconds. Timing attack mitigation: Sleeping for 0.577710962296 seconds. remoteCommand 'inv' from 200.75.101.249 Inventory (in memory) has inventory item already. Total message processing time: 0.600991010666 seconds. remoteCommand 'inv' from 46.208.242.157 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 74.74.103.120 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 200.75.101.249 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 2.134.245.21 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 46.223.18.202 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 81.98.253.250 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 68.113.120.119 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 200.75.101.249 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 84.58.247.85 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 46.208.242.157 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 84.58.247.85 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 94.40.96.218 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 184.79.4.90 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 92.25.250.208 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 81.98.253.250 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 46.223.18.202 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 68.113.120.119 Inventory (in memory) has inventory item already. remoteCommand 'inv' from 74.74.103.120 Inventory (in memory) has inventory item already. remoteCommand 'addr' from 94.40.96.218 addr message contains 1 IP addresses. knownNodes currently has 927 nodes for this stream. remoteCommand 'addr' from 92.25.250.208 addr message contains 1 IP addresses. knownNodes currently has 927 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 927 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 927 nodes for this stream. remoteCommand 'addr' from 92.25.250.208 addr message contains 1 IP addresses. added new node 72.83.194.254 to knownNodes in stream 1 Broadcasting addr with 1 entries. knownNodes currently has 928 nodes for this stream. remoteCommand 'addr' from 216.14.37.242 addr message contains 1 IP addresses. knownNodes currently has 928 nodes for this stream.
rubo77 commented 2013-06-19 21:53:32 +02:00 (Migrated from github.com)
Copy Link

you should post such a sample output on http://pastebin.com/
That would less clutter this page. please edit our comment!

you should post such a sample output on http://pastebin.com/ That would less clutter this page. please edit our comment!
👍 1
ghost commented 2013-06-19 22:29:12 +02:00 (Migrated from github.com)
Copy Link

Another sample

http://pastebin.com/ETLnNLq5

Towards the end there is:

Status bar: Doing housekeeping (Flushing inventory in memory to disk...)
sock.sendall error: [Errno 104] Connection reset by peer
self.sock.sendall failed

which I don't know whether is significant.

Another sample http://pastebin.com/ETLnNLq5 Towards the end there is: Status bar: Doing housekeeping (Flushing inventory in memory to disk...) sock.sendall error: [Errno 104] Connection reset by peer self.sock.sendall failed which I don't know whether is significant.
Atheros1 commented 2013-06-19 22:38:20 +02:00 (Migrated from github.com)
Copy Link

Thank you; this answers my question. Linux is not having trouble getting through the totalNumberOfObjectsThatWeHaveYetToCheckAndSeeWhetherWeAlreadyHave. So I'm back to not having any suspicions about the cause of the issue.

The "self.sock.sendall failed" is normal; some host just got disconnected.

Thank you; this answers my question. Linux is not having trouble getting through the totalNumberOfObjectsThatWeHaveYetToCheckAndSeeWhetherWeAlreadyHave. So I'm back to not having any suspicions about the cause of the issue. The "self.sock.sendall failed" is normal; some host just got disconnected.
bytting commented 2013-06-20 00:11:21 +02:00 (Migrated from github.com)
Copy Link

Quite frankly I don't think there is a issue on my side. My CPU is working at about 20% for considerable periods, also after both connections and the processing of messages has stabilized, but I guess that is to be expected.

Quite frankly I don't think there is a issue on my side. My CPU is working at about 20% for considerable periods, also after both connections and the processing of messages has stabilized, but I guess that is to be expected.
kseistrup commented 2013-06-22 14:23:04 +02:00 (Migrated from github.com)
Copy Link

@Atheros1 I regularly experienced this issue, and always had to shut down bitmessage pretty soon (say, less than ½ or 1 hour after start). The log hasn't been helpful, so I haven't been able to pinpoint the problem.

Today I tried a new approach:

I stopped bitmessage, then ran the following commands from the CLI:

$ cd ~/.PyBitmessage
$ echo .dump | sqlite3 messages.dat > messages.sql
$ vi messages.sql

I now inserted a PRAGMA line to enable auto vacuuming:

PRAGMA foreign_keys=OFF;
PRAGMA auto_vacuum=INCREMENTAL;
BEGIN TRANSACTION;

Then I restored the database before starting bitmessage:

$ mv -f messages.dat messages.dat-prev
$ sqlite3 messages.dat < messages.sql
$ rm -f messages.sql

The size of messages.dat before and after this operation was appr. 69 MB and 1 MB.

Bitmessage has been running for some 2½ hours now, and there's still no CPU spike. This would have been impossible before database cleaning. I only wonder if this is a lasting change…

Cheers.

@Atheros1 I regularly experienced this issue, and always had to shut down bitmessage pretty soon (say, less than ½ or 1 hour after start). The log hasn't been helpful, so I haven't been able to pinpoint the problem. Today I tried a new approach: I stopped bitmessage, then ran the following commands from the CLI: ``` sh $ cd ~/.PyBitmessage $ echo .dump | sqlite3 messages.dat > messages.sql $ vi messages.sql ``` I now inserted a PRAGMA line to enable auto vacuuming: ``` sqlite3 PRAGMA foreign_keys=OFF; PRAGMA auto_vacuum=INCREMENTAL; BEGIN TRANSACTION; ``` Then I restored the database before starting bitmessage: ``` sh $ mv -f messages.dat messages.dat-prev $ sqlite3 messages.dat < messages.sql $ rm -f messages.sql ``` The size of messages.dat before and after this operation was appr. 69 MB and 1 MB. Bitmessage has been running for some 2½ hours now, and there's still no CPU spike. This would have been impossible before database cleaning. I only wonder if this is a lasting change… Cheers.
ghost commented 2013-06-22 14:44:38 +02:00 (Migrated from github.com)
Copy Link

https://www.sqlite.org/pragma.html#pragma_auto_vacuum

https://www.sqlite.org/pragma.html#pragma_auto_vacuum
ghost commented 2013-06-22 16:12:53 +02:00 (Migrated from github.com)
Copy Link

It was a nice theory but it didn't work for me. After about an hour the CPU went to 100% usage and remained there until I quit the application.

It was a nice theory but it didn't work for me. After about an hour the CPU went to 100% usage and remained there until I quit the application.
kseistrup commented 2013-06-22 23:38:27 +02:00 (Migrated from github.com)
Copy Link

@fuzzgun, unfortunately you're right. It took some 4+ hours for bitmessage to get there on my machine, but eventually I had to shut it down. Darn…

@fuzzgun, unfortunately you're right. It took some 4+ hours for bitmessage to get there on my machine, but eventually I had to shut it down. Darn…
rubo77 commented 2013-06-24 18:41:06 +02:00 (Migrated from github.com)
Copy Link

I Updated my Ubuntu to 13.04 now with python 2.7.4

no CPU-lack so far for some hours...

I Updated my Ubuntu to 13.04 now with python 2.7.4 no CPU-lack so far for some hours...
fiatflux commented 2013-06-24 18:55:55 +02:00 (Migrated from github.com)
Copy Link

I've been getting this on the latest Python 2.7.5, so unless there's a
regression, sadly, I don't think that's it.

I've been getting this on the latest Python 2.7.5, so unless there's a regression, sadly, I don't think that's it.
ghost commented 2013-06-24 19:06:58 +02:00 (Migrated from github.com)
Copy Link

I've been on 13.04/2.7.4 for quite a while. Sometimes it can run for 2-3 hours before the CPU goes into overdrive.

I've been on 13.04/2.7.4 for quite a while. Sometimes it can run for 2-3 hours before the CPU goes into overdrive.
rubo77 commented 2013-06-24 19:17:08 +02:00 (Migrated from github.com)
Copy Link

yes, problem is back now ;(

yes, problem is back now ;(
DivineOmega commented 2013-06-27 14:52:25 +02:00 (Migrated from github.com)
Copy Link

I've not yet looked into this at all yet. However, a possibly theory is that it could be related to disconnections of other users, perhaps leaving some element of the program in a less than ideal state. This would explain the 'random' nature of the occurrences.

I've not yet looked into this at all yet. However, a possibly theory is that it could be related to disconnections of other users, perhaps leaving some element of the program in a less than ideal state. This would explain the 'random' nature of the occurrences.
acejam commented 2013-06-27 19:28:27 +02:00 (Migrated from github.com)
Copy Link

This CPU usage bug happens for me as soon as I start up the application. I don't think it's related to user disconnections, but I could be wrong. With the CPU usage this is delivering, I would think it's more likely that the client is to trying to decrypt every message, and not just the ones intended for it. (or something alone those lines) On my 8-core server, I'm seeing about 400% CPU usage. That tells me the app is using up 4 cores @ 100% each.

This CPU usage bug happens for me as soon as I start up the application. I don't think it's related to user disconnections, but I could be wrong. With the CPU usage this is delivering, I would think it's more likely that the client is to trying to decrypt _every_ message, and not just the ones intended for it. (or something alone those lines) On my 8-core server, I'm seeing about 400% CPU usage. That tells me the app is using up 4 cores @ 100% each.
acejam commented 2013-06-27 19:31:35 +02:00 (Migrated from github.com)
Copy Link

I would like to help debug this CPU issue as having the daemon run on my server @ 400% CPU is not fun. Is there some type of Python app profiling we can add on to see what threads are using up resources?

I would like to help debug this CPU issue as having the daemon run on my server @ 400% CPU is not fun. Is there some type of Python app profiling we can add on to see what threads are using up resources?
fiatflux commented 2013-06-27 19:39:11 +02:00 (Migrated from github.com)
Copy Link

If I wasn't running around doing travel stuff (restricting my work here to
about 5 minutes at a time), I would rig it up with yappi.
What I'd try is to start yappi in the main thread early in the program
setup, and start a profiling thread that keeps track of invocations and
estimated time spent in each. I'd dump this into a file every 30 seconds
along with total CPU use for the process. Even running something simple
like NNLS regression on the dataset will show something interesting, I
think. Perhaps simpler than that, try just poking around in strace and see
what pops up around the time that a spike in CPU is observed.

If I wasn't running around doing travel stuff (restricting my work here to about 5 minutes at a time), I would rig it up with yappi. What I'd try is to start yappi in the main thread early in the program setup, and start a profiling thread that keeps track of invocations and estimated time spent in each. I'd dump this into a file every 30 seconds along with total CPU use for the process. Even running something simple like NNLS regression on the dataset will show something interesting, I think. Perhaps simpler than that, try just poking around in strace and see what pops up around the time that a spike in CPU is observed.
fiatflux commented 2013-06-28 02:17:57 +02:00 (Migrated from github.com)
Copy Link

I found a half a jiffy of time after all, so I decided to write something like what I describe above. It does nothing with the data, and it's not made to be pulled upstream (since it has such a specific audience, and the finesse of an ogre), but folks are welcome to give it a try: https://github.com/fiatflux/PyBitmessage/tree/profile
You'll probably need to get yappi from bitbucket for it to work (arch users can now use python2-yappi-hg from AUR).

Yappi actually might not need much in the way of regression calibration for our purposes. And come to think of it I don't expect to find interesting trends in the fit coefficients between CPU use and this profiler's output, since the CPU problem seems to be a cross-platform bug. But I'm not sure about either.

I'm fairly sure that I'm going to be really, really offline for about a week or two. So it might be a good idea to assume I'm not going to do anything about this until then -- tinker away?

I found a half a jiffy of time after all, so I decided to write something like what I describe above. It does nothing with the data, and it's not made to be pulled upstream (since it has such a specific audience, and the finesse of an ogre), but folks are welcome to give it a try: https://github.com/fiatflux/PyBitmessage/tree/profile You'll probably need to get yappi from bitbucket for it to work (arch users can now use python2-yappi-hg from AUR). Yappi actually might not need much in the way of regression calibration for our purposes. And come to think of it I don't expect to find interesting trends in the fit coefficients between CPU use and this profiler's output, since the CPU problem seems to be a cross-platform bug. But I'm not sure about either. I'm fairly sure that I'm going to be really, really offline for about a week or two. So it might be a good idea to assume I'm not going to do anything about this until then -- tinker away?
Atheros1 commented 2013-06-28 06:31:05 +02:00 (Migrated from github.com)
Copy Link

Has anyone not on Linux experience this issue? I don't remember anyone saying that they had.

Has anyone not on Linux experience this issue? I don't remember anyone saying that they had.
acejam commented 2013-06-28 07:10:04 +02:00 (Migrated from github.com)
Copy Link

@Atheros1 When I run the python client on OSX Mountain Lion 10.8.4, I do not see any abnormal CPU usage. It only seems to creep up on Linux. (Ubuntu 12.04 x64)

@Atheros1 When I run the python client on OSX Mountain Lion 10.8.4, I do _not_ see any abnormal CPU usage. It only seems to creep up on Linux. (Ubuntu 12.04 x64)
fiatflux commented 2013-06-28 10:08:31 +02:00 (Migrated from github.com)
Copy Link

I think I did, but I did not notice it yesterday after running the client
for several hours.

I think I did, but I did not notice it yesterday after running the client for several hours.
linkerlin commented 2013-06-28 15:49:34 +02:00 (Migrated from github.com)
Copy Link

#269 by using gevent will save a lot of CPU

#269 by using gevent will save a lot of CPU
RexMorgan commented 2013-06-30 05:24:10 +02:00 (Migrated from github.com)
Copy Link

I'm also seeing this issue on Debian 7 64bit. 100% cpu usage after a couple hours.

I'm also seeing this issue on Debian 7 64bit. 100% cpu usage after a couple hours.
ghost commented 2013-07-01 08:19:27 +02:00 (Migrated from github.com)
Copy Link

Spot-on, DivineOmega!

Spot-on, DivineOmega!
DivineOmega commented 2013-07-01 10:19:32 +02:00 (Migrated from github.com)
Copy Link

@pgimeno Great stuff! Glad my theory was correct and fantastic job in nailing this one down.

@pgimeno Great stuff! Glad my theory was correct and fantastic job in nailing this one down.
Dokument commented 2013-07-24 04:43:24 +02:00 (Migrated from github.com)
Copy Link

This can be marked as closed I believe.

This can be marked as closed I believe.
marcomarsala commented 2017-10-16 07:18:42 +02:00 (Migrated from github.com)
Copy Link

Bitmessage_x64_0.6.2.exe on Windows always uses 100% of 1 core when idle. The whole GUI is slow and unresponsive. Objects to be synced is 0.

Bitmessage_x64_0.6.2.exe on Windows always uses 100% of 1 core when idle. The whole GUI is slow and unresponsive. Objects to be synced is 0.
This repo is archived. You cannot comment on issues.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something isn't working as it's supposed to

  
build

Related to building or build system

  
dependencies

Pull requests that update a dependency file

  
developers

The issue is relevant mainly for developers rather than users

  
documentation

   
duplicate

   
enhancement

New feature

  
formatting

Code or UI text formatting

  
invalid

   
legal

   
mobile

Frontend for mobile devices (kivy)

  
obsolete

   
packaging

   
performance

   
protocol

Protocol change

  
question

   
refactoring

   
regression

Some issue, fixed in the past, appears again

  
security

   
test

   
translation

   
usability

   
wontfix
No Label
bug
build
dependencies
developers
documentation
duplicate
enhancement
formatting
invalid
legal
mobile
obsolete
packaging
performance
protocol
question
refactoring
regression
security
test
translation
usability
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: Bitmessage/PyBitmessage-2024-08-21#183
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?