Bitmessage/PyBitmessage-2024-08-21
Archived
2
0
Fork 0
Code Issues 168 Pull Requests 53 Actions Packages Projects Releases 5 Wiki Activity

python 3 port plan #1712

New Issue
Open
opened 2021-01-10 14:56:46 +01:00 by PeterSurda · 22 comments
PeterSurda commented 2021-01-10 14:56:46 +01:00 (Migrated from github.com)
Copy Link

Using the principles of continuous delivery, port to python 3 should progress as following:

  • the first PR will add allow travis to run python3 while successfully doing nothing. The pybitmessage binary will check if it's running python 3 and if yes, exit with return code 0. Unit tests will be decorated to skip all tests when using python 3. This should allow travis to succeed in python 3. Not sure what to do if requirements.txt diverge between python 2 and python 3.
  • gradually tests will be updated to allow running in python 3
  • maybe new tests will be written to cater for areas that currently don't have tests to facilitate gradual porting
  • once everything is working correctly in python 3, as well as binary builds for the main platforms, python 2 support will be dropped
Using the principles of continuous delivery, port to python 3 should progress as following: - the first PR will add allow travis to run python3 while successfully doing nothing. The `pybitmessage` binary will check if it's running python 3 and if yes, exit with return code 0. Unit tests will be decorated to skip all tests when using python 3. This should allow travis to succeed in python 3. Not sure what to do if requirements.txt diverge between python 2 and python 3. - gradually tests will be updated to allow running in python 3 - maybe new tests will be written to cater for areas that currently don't have tests to facilitate gradual porting - once everything is working correctly in python 3, as well as binary builds for the main platforms, python 2 support will be dropped
👍 3
g1itch commented 2021-01-10 15:05:32 +01:00 (Migrated from github.com)
Copy Link

the first PR will add allow travis to run python3 while successfully doing nothing. The pybitmessage binary will check if it's running python 3 and if yes, exit with return code 0. Unit tests will be decorated to skip all tests when using python 3. This should allow travis to succeed in python 3.

In my ci branch I have python "3.6" in allow_failures and build itself succeeds if python3 part fails. But we all see the failures.

> the first PR will add allow travis to run python3 while successfully doing nothing. The pybitmessage binary will check if it's running python 3 and if yes, exit with return code 0. Unit tests will be decorated to skip all tests when using python 3. This should allow travis to succeed in python 3. In my `ci` branch I [have](https://github.com/g1itch/PyBitmessage/blob/ci/.travis.yml#L20) `python "3.6"` in `allow_failures` and build itself succeeds if python3 part fails. But we all see the failures.
g1itch commented 2021-01-10 15:14:21 +01:00 (Migrated from github.com)
Copy Link

Not sure what to do if requirements.txt diverge between python 2 and python 3.

Unlikely

> Not sure what to do if requirements.txt diverge between python 2 and python 3. Unlikely
PeterSurda commented 2021-01-10 16:01:19 +01:00 (Migrated from github.com)
Copy Link

In my ci branch I have python "3.6" in allow_failures and build itself succeeds if python3 part fails. But we all see the failures.

This is interesting, unfortunately it doesn't help me here. I need something that allows a gradual compatibility with python 3 with as little human intervention as possible. If all the errors are just ignored, then a PR reviewer needs more time to figure out what's working and what not by looking at the detailed output. The creator of the PR also needs to spend more time evaluating whether he still needs to continue working or wait for a review. Whereas in my proposal, a failure means the PR is rejected and the PR creator has immediate feedback that the isn't finished, and a success means it's ready for a review. This way, PR after PR, python 3 support will be implemented.

I already tried several approaches for getting python 3 support intergrated but none produced a PR that can be merged. One of the developers produced a branch which has most of the functionality as well as most unit tests working (or at least it looks like it's working), but they don't have the proper quality and aren't chunked nicely for PRs. My proposal allows this to happen (refactor or write from scratch, depending on which makes more sense).

Then there are other minor issues, like travis2bash not understanding the full travis syntax, the non-developers not having proper feedback. In my proposal, pybitmessage could simply print "python3 version not ready yet, use python2" when run under python3.

> In my `ci` branch I [have](https://github.com/g1itch/PyBitmessage/blob/ci/.travis.yml#L20) `python "3.6"` in `allow_failures` and build itself succeeds if python3 part fails. But we all see the failures. This is interesting, unfortunately it doesn't help me here. I need something that allows a gradual compatibility with python 3 with as little human intervention as possible. If all the errors are just ignored, then a PR reviewer needs more time to figure out what's working and what not by looking at the detailed output. The creator of the PR also needs to spend more time evaluating whether he still needs to continue working or wait for a review. Whereas in my proposal, a failure means the PR is rejected and the PR creator has immediate feedback that the isn't finished, and a success means it's ready for a review. This way, PR after PR, python 3 support will be implemented. I already tried several approaches for getting python 3 support intergrated but none produced a PR that can be merged. One of the developers produced a branch which has most of the functionality as well as most unit tests working (or at least it looks like it's working), but they don't have the proper quality and aren't chunked nicely for PRs. My proposal allows this to happen (refactor or write from scratch, depending on which makes more sense). Then there are other minor issues, like travis2bash not understanding the full travis syntax, the non-developers not having proper feedback. In my proposal, `pybitmessage` could simply print "python3 version not ready yet, use python2" when run under python3.
ghost commented 2021-01-28 06:33:57 +01:00 (Migrated from github.com)
Copy Link

@g1itch I don't have python2! It is not supported in any new OS. Not even in Debian 11!

I need a Debian 10 virtual machine with Gnome Boxes to run PyBitmessage. Unfortunately.

@g1itch I don't have python2! It is not supported in any new OS. Not even in Debian 11! I need a Debian 10 virtual machine with Gnome Boxes to run PyBitmessage. Unfortunately.
g1itch commented 2021-01-28 18:22:43 +01:00 (Migrated from github.com)
Copy Link

@g1itch I don't have python2! It is not supported in any new OS. Not even in Debian 11!

I need a Debian 10 virtual machine with Gnome Boxes to run PyBitmessage. Unfortunately.

Well, for me Ubuntu Focal is recent enough distro and it has python2. Proper flatpack packaging will probably be enough to run current PyBitmessage code on Linux distros with no python2, though I would prefer merging qt5 code first.

> @g1itch I don't have python2! It is not supported in any new OS. Not even in Debian 11! > > I need a Debian 10 virtual machine with Gnome Boxes to run PyBitmessage. Unfortunately. Well, for me Ubuntu Focal is recent enough distro and it has python2. Proper flatpack packaging will probably be enough to run current PyBitmessage code on Linux distros with no python2, though I would prefer merging qt5 code first.
g1itch commented 2021-01-29 19:15:12 +01:00 (Migrated from github.com)
Copy Link

Then there are other minor issues, like travis2bash not understanding the full travis syntax, the non-developers not having proper feedback.

It seems that python: - "3.6" is OK for your buildbot tasks - it's simply ignored: buildbot/travis_focal succeed.

> Then there are other minor issues, like travis2bash not understanding the full travis syntax, the non-developers not having proper feedback. It seems that `python: - "3.6"` is OK for your buildbot tasks - it's simply ignored: [buildbot/travis_focal](https://buildbot.bitmessage.org/#builders/17/builds/2307) succeed.
PeterSurda commented 2021-01-30 16:10:11 +01:00 (Migrated from github.com)
Copy Link

That particular feature is supported. However the VMs don't have python 3.6 so the tests can't run. I'll add python 3.6 into the VMs and then rerun your tests, but it may take a couple of days.

That particular feature is supported. However the VMs don't have python 3.6 so the tests can't run. I'll add python 3.6 into the VMs and then rerun your tests, but it may take a couple of days.
g1itch commented 2021-01-30 16:28:44 +01:00 (Migrated from github.com)
Copy Link

I'll add python 3.6 into the VMs and then rerun your tests, but it may take a couple of days.

There is no rush. If you take a close look into python 2.7 job of the PR you will see that it's not fully correct. Now I'm discovering some strange behavior of test-mode.

> I'll add python 3.6 into the VMs and then rerun your tests, but it may take a couple of days. There is no rush. If you take a close look into [python 2.7 job](https://travis-ci.org/github/Bitmessage/PyBitmessage/jobs/756751477) of the PR you will see that it's not fully correct. Now I'm discovering some strange behavior of test-mode.
ghost commented 2021-01-30 17:03:13 +01:00 (Migrated from github.com)
Copy Link
Python 3.9.1 (default, Dec  8 2020, 07:51:42) 
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Well, I only have Python 3.9 maybe I will flip that 9 to 6 somehow.

qt5 should be priority, just as to make sure that PyBitmessage works above Python 3.5 at least.

``` Python 3.9.1 (default, Dec 8 2020, 07:51:42) [GCC 10.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> ``` Well, I only have Python 3.9 maybe I will flip that 9 to 6 somehow. qt5 should be priority, just as to make sure that PyBitmessage works above Python 3.5 at least.
g1itch commented 2021-01-30 17:10:57 +01:00 (Migrated from github.com)
Copy Link

just as to make sure that PyBitmessage works above Python 3.5 at least.

PyBitmessage currently does not support python3.

> just as to make sure that PyBitmessage works above Python 3.5 at least. PyBitmessage currently does not support python3.
PeterSurda commented 2021-01-30 20:57:06 +01:00 (Migrated from github.com)
Copy Link

@g1itch The VMs I use for buildbot now should support python3.6 through python3.10. No need to change anything, just put the desired version(s) into .travis.yml like you're doing in #1725. It will run all the tests for all the python versions listed.

@g1itch The VMs I use for buildbot now should support python3.6 through python3.10. No need to change anything, just put the desired version(s) into `.travis.yml` like you're doing in #1725. It will run all the tests for all the python versions listed.
g1itch commented 2021-05-01 13:46:51 +02:00 (Migrated from github.com)
Copy Link

It seems I read here something you didn't write and maybe didn't even mean. Do you agree that if somebody wants to participate in the porting process, he should:

  1. chose an entity: a structure, a module, a thread or a couple of them (for example objectProcessor works together with addressGenerator)
  2. cover it by tests
  3. when the tests pass with python2, edit the code to support python3

Furthermore, it's not related to the python3 porting process but it used to be so - in order to pass a review PR should fit the following criteria:

  1. comply with PEP8
  2. have one commit per change with descriptive messages
  3. do not include unrelated changes
  4. do not introduce unneeded extra files

The latest merge violates all the points. I'm surprised that it even passed pylint check.

It seems I read here something you didn't write and maybe didn't even mean. Do you agree that if somebody wants to participate in the porting process, he should: 1. chose an entity: a structure, a module, a thread or a couple of them (for example objectProcessor works together with addressGenerator) 2. cover it by tests 3. when the tests pass with python2, edit the code to support python3 Furthermore, it's not related to the python3 porting process but it used to be so - in order to pass a review PR should fit the following criteria: 1. comply with PEP8 2. have one commit per change with descriptive messages 3. do not include unrelated changes 4. do not introduce unneeded extra files The latest merge violates all the points. I'm surprised that it even passed pylint check.
PeterSurda commented 2021-05-01 15:53:44 +02:00 (Migrated from github.com)
Copy Link

@g1itch Your proposals make sense. I appreciate you giving feedback to the other developers.

Unfortunately for personal reasons I have very little time to code review nowadays. I also didn't have any help from the developer that normally helps me in this process as she had time off for personal reasons too (for about 1.5 months). At the same time however, I'm paying developers to work on the project, and they either need feedback or their code merged, or I pay them to do nothing. The project is in an annoying phase when python2 compatibility is being dropped by more and more distros and dependencies, but python3 code isn't working sufficiently well to have the whole program run.

For these reasons I approved PRs I otherwise wouldn't have. Hopefully, now more python3 code can be worked on, and some tests that require full app can be refactored or created from scratch using mocks. Existing code can be cleaned up. And developers can work without having to wait for each other's PRs to be merged, so development can work in parallel.

I should have more time starting in about 3-4 weeks, around the end of May. Perhaps then we can make the process more strict again.

@g1itch Your proposals make sense. I appreciate you giving feedback to the other developers. Unfortunately for personal reasons I have very little time to code review nowadays. I also didn't have any help from the developer that normally helps me in this process as she had time off for personal reasons too (for about 1.5 months). At the same time however, I'm paying developers to work on the project, and they either need feedback or their code merged, or I pay them to do nothing. The project is in an annoying phase when python2 compatibility is being dropped by more and more distros and dependencies, but python3 code isn't working sufficiently well to have the whole program run. For these reasons I approved PRs I otherwise wouldn't have. Hopefully, now more python3 code can be worked on, and some tests that require full app can be refactored or created from scratch using mocks. Existing code can be cleaned up. And developers can work without having to wait for each other's PRs to be merged, so development can work in parallel. I should have more time starting in about 3-4 weeks, around the end of May. Perhaps then we can make the process more strict again.
g1itch commented 2021-07-28 19:39:37 +02:00 (Migrated from github.com)
Copy Link

Meanwhile the official doc explicitly mentions test coverage and the bad practice of using sys.version_info: https://docs.python.org/3/howto/pyporting.html#use-feature-detection-instead-of-version-detection

As for requirements, you can separate them by version using environment markers e.g.

diff --git a/requirements.txt b/requirements.txt
index 899753cb..65fafe75 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,5 +1,6 @@
 coverage
 python_prctl;platform_system=="Linux"
+PyQt5;python_version>="3.8"
 psutil
 pycrypto
 six
Meanwhile the [official doc](https://docs.python.org/3/howto/pyporting.html) explicitly mentions test coverage and the bad practice of using `sys.version_info`: https://docs.python.org/3/howto/pyporting.html#use-feature-detection-instead-of-version-detection As for requirements, you can separate them by version using [environment markers](https://www.python.org/dev/peps/pep-0508/#environment-markers) e.g. ```diff diff --git a/requirements.txt b/requirements.txt index 899753cb..65fafe75 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,5 +1,6 @@ coverage python_prctl;platform_system=="Linux" +PyQt5;python_version>="3.8" psutil pycrypto six ```
tuxayo commented 2022-06-08 03:29:19 +02:00 (Migrated from github.com)
Copy Link

Is there a branch/repo/milestone to follow the python3 port?

Is there a branch/repo/milestone to follow the python3 port?
cornfeedhobo commented 2022-08-19 05:56:47 +02:00 (Migrated from github.com)
Copy Link

@g1itch are you still working on this?

@g1itch are you still working on this?
Neustradamus commented 2023-02-17 22:45:54 +01:00 (Migrated from github.com)
Copy Link

Have you progressed on it?

Have you progressed on it?
cornfeedhobo commented 2023-06-01 17:18:21 +02:00 (Migrated from github.com)
Copy Link

After revisiting this 9 months later, I think it's safe to say that bitmessage is dying. There are few distros that still package pyqt4, leaving this to be deployable to few. Unless someone decides to take this on, probably best to archive your keys and move your chats elsewhere.

After revisiting this 9 months later, I think it's safe to say that bitmessage is dying. There are few distros that still package pyqt4, leaving this to be deployable to few. Unless someone decides to take this on, probably best to archive your keys and move your chats elsewhere.
PeterSurda commented 2023-06-10 05:50:15 +02:00 (Migrated from github.com)
Copy Link

The development is currently paused but should resume in a couple of months. While running from source may be problematic, you can use the binaries (like appimage) until the python3 port is done.

The development is currently paused but should resume in a couple of months. While running from source may be problematic, you can use the binaries (like appimage) until the python3 port is done.
cornfeedhobo commented 2023-06-17 14:43:58 +02:00 (Migrated from github.com)
Copy Link

@PeterSurda Is there a link to the AppImage that's published from an official build source, preferably signed?

@PeterSurda Is there a link to the AppImage that's published from an official build source, preferably signed?
PeterSurda commented 2023-06-19 12:57:57 +02:00 (Migrated from github.com)
Copy Link

This is the latest one: https://appimage.bitmessage.org/releases/20230116/ . I understand it's a bit of indirect. Automated signing isn't implemented at the moment.

This is the latest one: https://appimage.bitmessage.org/releases/20230116/ . I understand it's a bit of indirect. Automated signing isn't implemented at the moment.
cornfeedhobo commented 2023-06-19 23:28:37 +02:00 (Migrated from github.com)
Copy Link

@PeterSurda Thanks for following up. Good luck with the python3 migration. I'll check back in at the end of the year :)

@PeterSurda Thanks for following up. Good luck with the python3 migration. I'll check back in at the end of the year :)
This repo is archived. You cannot comment on issues.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something isn't working as it's supposed to

  
build

Related to building or build system

  
dependencies

Pull requests that update a dependency file

  
developers

The issue is relevant mainly for developers rather than users

  
documentation

   
duplicate

   
enhancement

New feature

  
formatting

Code or UI text formatting

  
invalid

   
legal

   
mobile

Frontend for mobile devices (kivy)

  
obsolete

   
packaging

   
performance

   
protocol

Protocol change

  
question

   
refactoring

   
regression

Some issue, fixed in the past, appears again

  
security

   
test

   
translation

   
usability

   
wontfix
No Label
bug
build
dependencies
developers
documentation
duplicate
enhancement
formatting
invalid
legal
mobile
obsolete
packaging
performance
protocol
question
refactoring
regression
security
test
translation
usability
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: Bitmessage/PyBitmessage-2024-08-21#1712
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?