Python and performance of "Proof of Work" #612

New Issue

2014-01-13T22:24:09+01:00

yurivict commented

2014-01-13 22:24:09 +01:00

(Migrated from github.com)

This is not a bug report, but a suggestion.

Python is slow, and POW computation by definition requires a lot of CPU resource. Therefore, there is a problem that Python implementation "steals" CPU resource. Also it creates a false impression that that much CPU work is required to send a message, when in reality native implementation would require probably few times less CPU. Since POW is important for bitmessage, this is a significant issue.

My suggestion is to write the corresponding POW function in llvm code, and then bind with llvm and compile and run it. llvm has the ability to run in JIT, but this is also not a native way. llvm also has the ability to write native code into binary blob (ELF or other depending in architecture). Maybe this is all that is needed, you can just run a native function from there.

You can actually write POW function in C first, and compile it into llvm language.

This is not a bug report, but a suggestion. Python is slow, and POW computation by definition requires a lot of CPU resource. Therefore, there is a problem that Python implementation "steals" CPU resource. Also it creates a false impression that that much CPU work is required to send a message, when in reality native implementation would require probably few times less CPU. Since POW is important for bitmessage, this is a significant issue. My suggestion is to write the corresponding POW function in llvm code, and then bind with llvm and compile and run it. llvm has the ability to run in JIT, but this is also not a native way. llvm also has the ability to write native code into binary blob (ELF or other depending in architecture). Maybe this is all that is needed, you can just run a native function from there. You can actually write POW function in C first, and compile it into llvm language.

grant-olson commented

2014-01-29 22:02:27 +01:00

(Migrated from github.com)

Here is a plugin that will use native code to do the PoW calculation. I think we were talking at one point about including this in the main binary distribution. I'm not sure if we actually decided against it, or if the feature just never got implemented.

https://github.com/grant-olson/bitmessage-powfaster

Here is a plugin that will use native code to do the PoW calculation. I think we were talking at one point about including this in the main binary distribution. I'm not sure if we actually decided against it, or if the feature just never got implemented. https://github.com/grant-olson/bitmessage-powfaster

yurivict commented

2014-01-30 03:25:16 +01:00

(Migrated from github.com)

Platform compatibility would be an issue here. Pthread probably isn't available in windows by default.

PeterSurda commented

2015-10-18 12:49:51 +02:00

(Migrated from github.com)

I'll test this on windows. I tested it already on Linux, both OpenCL and the native CPU one work fine.

PeterSurda commented

2015-11-10 18:12:29 +01:00

(Migrated from github.com)

I merged the C implementation it a couple of days ago in the mailchuck fork, cleaned it up (fixed compiler warnings), make it compile cross-platform and made it thread-safe. Also, it now is able to detect the number of available cores, by asking for process affinity. On OSX process affinity is not available, so it uses the system available cores, which is less optimal but acceptable. I was unable to find out how to bundle it for the EXE/DMG releases on time for v0.5.2, but yesterday I fixed that too, so v0.5.3 (probably this weekend) will have it working properly. I successfully tested it on Windows, OSX and linux.

OpenCL implementation is available too, but it needs to be turned on manually in the config file, and it does not work on AMD for some reason (returns wrong results) yet. On Nvidia it works ok.

So it will be a part of 0.6.

I merged the C implementation it a couple of days ago in the mailchuck fork, cleaned it up (fixed compiler warnings), make it compile cross-platform and made it thread-safe. Also, it now is able to detect the number of available cores, by asking for process affinity. On OSX process affinity is not available, so it uses the system available cores, which is less optimal but acceptable. I was unable to find out how to bundle it for the EXE/DMG releases on time for v0.5.2, but yesterday I fixed that too, so v0.5.3 (probably this weekend) will have it working properly. I successfully tested it on Windows, OSX and linux. OpenCL implementation is available too, but it needs to be turned on manually in the config file, and it does not work on AMD for some reason (returns wrong results) yet. On Nvidia it works ok. So it will be a part of 0.6.

This repo is archived. You cannot comment on issues.

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Bitmessage/PyBitmessage-2025-02-27#612