Bitmessage/PyBitmessage-2024-12-19
Archived
2
0
Fork 0
Code Issues 168 Pull Requests 52 Actions Packages Projects Releases 5 Wiki Activity

Updated origin by upstream

Browse Source
This commit is contained in:
jai.s 2020-01-15 12:26:23 +05:30
commit 6b1abe7048
No known key found for this signature in database
GPG Key ID: 360CFA25EFC67D12
50 changed files with 687 additions and 544 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
LICENSE
View File

@ -22,7 +22,7 @@ SOFTWARE.
===== qidenticon.py identicon python implementation with QPixmap output by sendiulo <sendiulo@gmx.net> ===== qidenticon.py identicon python implementation with QPixmap output by sendiulo <sendiulo@gmx.net>
qidenticon.py is Licesensed under FreeBSD License. qidenticon.py is Licensed under FreeBSD License.
(http://www.freebsd.org/copyright/freebsd-license.html) (http://www.freebsd.org/copyright/freebsd-license.html)
Copyright 2013 "Sendiulo". All rights reserved. Copyright 2013 "Sendiulo". All rights reserved.
@ -36,7 +36,7 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER ``AS IS'' AND ANY EXPRESS OR I
===== based on identicon.py identicon python implementation. by Shin Adachi <shn@glucose.jp> ===== based on identicon.py identicon python implementation. by Shin Adachi <shn@glucose.jp>
identicon.py is Licesensed under FreeBSD License. identicon.py is Licensed under FreeBSD License.
(http://www.freebsd.org/copyright/freebsd-license.html) (http://www.freebsd.org/copyright/freebsd-license.html)
Copyright 1994-2009 Shin Adachi. All rights reserved. Copyright 1994-2009 Shin Adachi. All rights reserved.
@ -47,3 +47,24 @@ Redistribution and use in source and binary forms, with or without modification,
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
===== based on asyncore_pollchoose.py asyncore_pollchoose python implementation. by Sam Rushing <rushing@nightmare.com>
Copyright 1996 by Sam Rushing. All Rights Reserved
Permission to use, copy, modify, and distribute this software and
its documentation for any purpose and without fee is hereby
granted, provided that the above copyright notice appear in all
copies and that both that copyright notice and this permission
notice appear in supporting documentation, and that the name of Sam
Rushing not be used in advertising or publicity pertaining to
distribution of the software without specific, written prior
permission.
SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN
NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR
CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

4
android_instruction.rst
View File

@ -1,6 +1,6 @@
PyBitmessage(Android) PyBitmessage(Android)
This sample aims to be as close to a real world example of a mobile. It has a more refined design and also provides a practical example of how a mobile app would interact and communicate with its adresses. This sample aims to be as close to a real world example of a mobile. It has a more refined design and also provides a practical example of how a mobile app would interact and communicate with its addresses.
Steps for trying out this sample: Steps for trying out this sample:
@ -13,7 +13,7 @@ This sample uses the kivy as Kivy is an open source, cross-platform Python frame
Kivy is written in Python and Cython, supports various input devices and has an extensive widget library. With the same codebase, you can target Windows, OS X, Linux, Android and iOS. All Kivy widgets are built with multitouch support. Kivy is written in Python and Cython, supports various input devices and has an extensive widget library. With the same codebase, you can target Windows, OS X, Linux, Android and iOS. All Kivy widgets are built with multitouch support.
Kivy in support take Buildozer which is a tool that automates the entire build process. It downloads and sets up all the prequisites for python-for-android, including the android SDK and NDK, then builds an apk that can be automatically pushed to the device. Kivy in support take Buildozer which is a tool that automates the entire build process. It downloads and sets up all the prerequisite for python-for-android, including the android SDK and NDK, then builds an apk that can be automatically pushed to the device.
Buildozer currently works only in Linux, and is an alpha release, but it already works well and can significantly simplify the apk build. Buildozer currently works only in Linux, and is an alpha release, but it already works well and can significantly simplify the apk build.

2
checkdeps.py
View File

@ -1,6 +1,6 @@
#!/usr/bin/env python2 #!/usr/bin/env python2
""" """
Check dependendies and give recommendations about how to satisfy them Check dependencies and give recommendations about how to satisfy them
Limitations: Limitations:

4
docs/contribute.dir/develop.dir/opsec.rst
View File

@ -21,12 +21,12 @@ If we are to make bold claims about protecting your privacy we should demonstrat
- looking to audit - looking to audit
- warrant canary - warrant canary
Digital foootprint Digital footprint
------------------ ------------------
Your internet use can reveal metadata you wouldn't expect. This can be connected with other information about you if you're not careful. Your internet use can reveal metadata you wouldn't expect. This can be connected with other information about you if you're not careful.
* Use separate addresses for different puprose * Use separate addresses for different purposes
* Don't make the same mistakes all the time * Don't make the same mistakes all the time
* Your language use is unique. The more you type, the more you fingerprint yourself. The words you know and use often vs the words you don't know or use often. * Your language use is unique. The more you type, the more you fingerprint yourself. The words you know and use often vs the words you don't know or use often.

8
docs/contribute.dir/develop.dir/overview.rst
View File

@ -11,17 +11,17 @@ Bitmessage makes use of fabric_ to define tasks such as building documentation o
Code style and linters Code style and linters
---------------------- ----------------------
We aim to be PEP8 compliant but we recognise that we have a long way still to go. Currently we have style and lint exceptions specified at the most specific place we can. We are ignoring certain issues project-wide in order to avoid alert-blindess, avoid style and lint regressions and to allow continuous integration to hook into the output from the tools. While it is hoped that all new changes pass the checks, fixing some existing violations are mini-projects in themselves. Current thinking on ignorable violations is reflected in the options and comments in setup.cfg. Module and line-level lint warnings represent refactoring opportunities. We aim to be PEP8 compliant but we recognize that we have a long way still to go. Currently we have style and lint exceptions specified at the most specific place we can. We are ignoring certain issues project-wide in order to avoid alert-blindness, avoid style and lint regressions and to allow continuous integration to hook into the output from the tools. While it is hoped that all new changes pass the checks, fixing some existing violations are mini-projects in themselves. Current thinking on ignorable violations is reflected in the options and comments in setup.cfg. Module and line-level lint warnings represent refactoring opportunities.
Pull requests Pull requests
------------- -------------
There is a template at PULL_REQUEST_TEMPLATE.md that appears in the pull-request description. Please replace this text with something appropriate to your changes based off the ideas in the template. There is a template at PULL_REQUEST_TEMPLATE.md that appears in the pull-request description. Please replace this text with something appropriate to your changes based on the ideas in the template.
Bike-shedding Bike-shedding
------------- -------------
Beyond having well-documented, Pythonic code with static analysis tool checks, extensive test coverage and powerful devops tools, what else can we have? Without violating any linters there is room for making arbirary decisions solely for the sake of project consistency. These are the stuff of the pedant's PR comments. Rather than have such conversations in PR comments, we can lay out the result of discussion here. Beyond having well-documented, Pythonic code with static analysis tool checks, extensive test coverage and powerful devops tools, what else can we have? Without violating any linters there is room for making arbitrary decisions solely for the sake of project consistency. These are the stuff of the pedant's PR comments. Rather than have such conversations in PR comments, we can lay out the result of discussion here.
I'm putting up a strawman for each topic here, mostly based on my memory of reading related Stack Overflow articles etc. If contributors feel strongly (and we don't have anything better to do) then maybe we can convince each other to update this section. I'm putting up a strawman for each topic here, mostly based on my memory of reading related Stack Overflow articles etc. If contributors feel strongly (and we don't have anything better to do) then maybe we can convince each other to update this section.
@ -49,7 +49,7 @@ British vs American spelling
Dependency graph Dependency graph
---------------- ----------------
These images are not very useful right now but the aim is to tweak the settings of one or more of them to be informative, and/or divide them up into smaller grapghs. These images are not very useful right now but the aim is to tweak the settings of one or more of them to be informative, and/or divide them up into smaller graphs.
To re-build them, run `fab build_docs:dep_graphs=true`. Note that the dot graph takes a lot of time. To re-build them, run `fab build_docs:dep_graphs=true`. Note that the dot graph takes a lot of time.

4
fabfile/README.md
View File

@ -1,6 +1,6 @@
# Fabric # Fabric
[Fabric](https://www.fabfile.org) is a Python library for performing devops tasks. You can thing of it a bit like a [Fabric](https://www.fabfile.org) is a Python library for performing devops tasks. You can think of it a bit like a
makefile on steroids for Python. Its api abstracts away the clunky way you would run shell commands in Python, check makefile on steroids for Python. Its api abstracts away the clunky way you would run shell commands in Python, check
return values and manage stdio. Tasks may be targetted at particular hosts or group of hosts. return values and manage stdio. Tasks may be targetted at particular hosts or group of hosts.
@ -46,7 +46,7 @@ Furthermore, you can use -- to run arbitrary shell commands rather than tasks:
There are a number of advantages that should benefit us: There are a number of advantages that should benefit us:
* Common tasks can be writen in Python and executed consistently * Common tasks can be written in Python and executed consistently
* Common tasks are now under source control * Common tasks are now under source control
* All developers can run the same commands, if the underlying command sequence for a task changes (after review, obv) * All developers can run the same commands, if the underlying command sequence for a task changes (after review, obv)
the user does not have to care the user does not have to care

2
packages/README.md
View File

@ -15,7 +15,7 @@ OSX:
https://github.com/Bitmessage/PyBitmessage/releases https://github.com/Bitmessage/PyBitmessage/releases
Wors on OSX 10.7.5 or higher Works on OSX 10.7.5 or higher
Arch linux: Arch linux:

186
src/bitmessagecurses/__init__.py
View File

@ -1,8 +1,6 @@
""" """
src/bitmessagecurses/__init__.py Bitmessage commandline interface
================================
""" """
# Copyright (c) 2014 Luke Montalvo <lukemontalvo@gmail.com> # Copyright (c) 2014 Luke Montalvo <lukemontalvo@gmail.com>
# This file adds a alternative commandline interface, feel free to critique and fork # This file adds a alternative commandline interface, feel free to critique and fork
# #
@ -20,21 +18,22 @@ import time
from textwrap import fill from textwrap import fill
from threading import Timer from threading import Timer
from addresses import addBMIfNotPresent, decodeAddress
from bmconfigparser import BMConfigParser
from dialog import Dialog from dialog import Dialog
from helper_ackPayload import genAckPayload
from helper_sql import sqlExecute, sqlQuery
from inventory import Inventory
import l10n import l10n
import network.stats import network.stats
from pyelliptic.openssl import OpenSSL
import queues import queues
import shared import shared
import shutdown import shutdown
from addresses import addBMIfNotPresent, decodeAddress
from bmconfigparser import BMConfigParser
from helper_ackPayload import genAckPayload
from helper_sql import sqlExecute, sqlQuery
from inventory import Inventory
# pylint: disable=global-statement
quit = False # pylint: disable=redefined-builtin
quit_ = False
menutab = 1 menutab = 1
menu = ["Inbox", "Send", "Sent", "Your Identities", "Subscriptions", "Address Book", "Blacklist", "Network Status"] menu = ["Inbox", "Send", "Sent", "Your Identities", "Subscriptions", "Address Book", "Blacklist", "Network Status"]
naptime = 100 naptime = 100
@ -61,26 +60,31 @@ bwtype = "black"
BROADCAST_STR = "[Broadcast subscribers]" BROADCAST_STR = "[Broadcast subscribers]"
class printLog: # pylint: disable=no-self-use, no-init, old-style-class class printLog(object):
"""Printing logs""" """Printing logs"""
# pylint: disable=no-self-use
def write(self, output): def write(self, output):
# pylint: disable=global-statement """Write logs"""
global log global log
log += output log += output
def flush(self): def flush(self):
"""Flush logs"""
pass pass
class errLog: # pylint: disable=no-self-use, no-init, old-style-class class errLog(object):
"""Error logs""" """Error logs"""
# pylint: disable=no-self-use
def write(self, output): def write(self, output):
# pylint: disable=global-statement """Write error logs"""
global log global log
log += "!" + output log += "!" + output
def flush(self): def flush(self):
"""Flush error logs"""
pass pass
@ -138,14 +142,15 @@ def scrollbox(d, text, height=None, width=None):
def resetlookups(): def resetlookups():
"""Reset the Inventory Lookups""" """Reset the Inventory Lookups"""
global inventorydata # pylint: disable=global-statement global inventorydata
inventorydata = Inventory().numberOfInventoryLookupsPerformed inventorydata = Inventory().numberOfInventoryLookupsPerformed
Inventory().numberOfInventoryLookupsPerformed = 0 Inventory().numberOfInventoryLookupsPerformed = 0
Timer(1, resetlookups, ()).start() Timer(1, resetlookups, ()).start()
def drawtab(stdscr): # pylint: disable=too-many-branches, too-many-statements def drawtab(stdscr):
"""Method for drawing different tabs""" """Method for drawing different tabs"""
# pylint: disable=too-many-branches, too-many-statements
if menutab in range(1, len(menu) + 1): if menutab in range(1, len(menu) + 1):
if menutab == 1: # Inbox if menutab == 1: # Inbox
stdscr.addstr(3, 5, "To", curses.A_BOLD) stdscr.addstr(3, 5, "To", curses.A_BOLD)
@ -282,12 +287,12 @@ def drawtab(stdscr): # pylint: disable=too-many-branches, too-many-statem
stdscr.addstr(13, 6, "Log", curses.A_BOLD) stdscr.addstr(13, 6, "Log", curses.A_BOLD)
n = log.count('\n') n = log.count('\n')
if n > 0: if n > 0:
l = log.split('\n') lg = log.split('\n')
if n > 512: if n > 512:
del l[:(n - 256)] del lg[:(n - 256)]
logpad.erase() logpad.erase()
n = len(l) n = len(lg)
for i, item in enumerate(l): for i, item in enumerate(lg):
a = 0 a = 0
if item and item[0] == '!': if item and item[0] == '!':
a = curses.color_pair(1) a = curses.color_pair(1)
@ -314,7 +319,8 @@ def dialogreset(stdscr):
# pylint: disable=too-many-branches, too-many-statements # pylint: disable=too-many-branches, too-many-statements
def handlech(c, stdscr): def handlech(c, stdscr):
# pylint: disable=redefined-outer-name, too-many-nested-blocks, too-many-locals, global-statement """Handle character given on the command-line interface"""
# pylint: disable=redefined-outer-name, too-many-nested-blocks, too-many-locals
if c != curses.ERR: if c != curses.ERR:
global inboxcur, addrcur, sentcur, subcur, abookcur, blackcur global inboxcur, addrcur, sentcur, subcur, abookcur, blackcur
if c in range(256): if c in range(256):
@ -322,8 +328,8 @@ def handlech(c, stdscr):
global menutab global menutab
menutab = int(chr(c)) menutab = int(chr(c))
elif chr(c) == 'q': elif chr(c) == 'q':
global quit global quit_
quit = True quit_ = True
elif chr(c) == '\n': elif chr(c) == '\n':
curses.curs_set(1) curses.curs_set(1)
d = Dialog(dialog="dialog") d = Dialog(dialog="dialog")
@ -363,10 +369,10 @@ def handlech(c, stdscr):
inbox[inboxcur][7] = 1 inbox[inboxcur][7] = 1
else: else:
scrollbox(d, unicode("Could not fetch message.")) scrollbox(d, unicode("Could not fetch message."))
elif t == "2": # Mark unread elif t == "2": # Mark unread
sqlExecute("UPDATE inbox SET read=0 WHERE msgid=?", inbox[inboxcur][0]) sqlExecute("UPDATE inbox SET read=0 WHERE msgid=?", inbox[inboxcur][0])
inbox[inboxcur][7] = 0 inbox[inboxcur][7] = 0
elif t == "3": # Reply elif t == "3": # Reply
curses.curs_set(1) curses.curs_set(1)
m = inbox[inboxcur] m = inbox[inboxcur]
fromaddr = m[4] fromaddr = m[4]
@ -375,7 +381,7 @@ def handlech(c, stdscr):
if fromaddr == item[2] and item[3] != 0: if fromaddr == item[2] and item[3] != 0:
ischan = True ischan = True
break break
if not addresses[i][1]: # pylint: disable=undefined-loop-variable if not addresses[i][1]: # pylint: disable=undefined-loop-variable
scrollbox(d, unicode( scrollbox(d, unicode(
"Sending address disabled, please either enable it" "Sending address disabled, please either enable it"
"or choose a different address.")) "or choose a different address."))
@ -396,7 +402,7 @@ def handlech(c, stdscr):
sendMessage(fromaddr, toaddr, ischan, subject, body, True) sendMessage(fromaddr, toaddr, ischan, subject, body, True)
dialogreset(stdscr) dialogreset(stdscr)
elif t == "4": # Add to Address Book elif t == "4": # Add to Address Book
addr = inbox[inboxcur][4] addr = inbox[inboxcur][4]
if addr not in [item[1] for i, item in enumerate(addrbook)]: if addr not in [item[1] for i, item in enumerate(addrbook)]:
r, t = d.inputbox("Label for address \"" + addr + "\"") r, t = d.inputbox("Label for address \"" + addr + "\"")
@ -409,7 +415,7 @@ def handlech(c, stdscr):
addrbook.reverse() addrbook.reverse()
else: else:
scrollbox(d, unicode("The selected address is already in the Address Book.")) scrollbox(d, unicode("The selected address is already in the Address Book."))
elif t == "5": # Save message elif t == "5": # Save message
set_background_title(d, "Save \"" + inbox[inboxcur][5] + "\" as text file") set_background_title(d, "Save \"" + inbox[inboxcur][5] + "\" as text file")
r, t = d.inputbox("Filename", init=inbox[inboxcur][5] + ".txt") r, t = d.inputbox("Filename", init=inbox[inboxcur][5] + ".txt")
if r == d.DIALOG_OK: if r == d.DIALOG_OK:
@ -418,12 +424,12 @@ def handlech(c, stdscr):
if ret != []: if ret != []:
for row in ret: for row in ret:
msg, = row msg, = row
fh = open(t, "a") # Open in append mode just in case fh = open(t, "a") # Open in append mode just in case
fh.write(msg) fh.write(msg)
fh.close() fh.close()
else: else:
scrollbox(d, unicode("Could not fetch message.")) scrollbox(d, unicode("Could not fetch message."))
elif t == "6": # Move to trash elif t == "6": # Move to trash
sqlExecute("UPDATE inbox SET folder='trash' WHERE msgid=?", inbox[inboxcur][0]) sqlExecute("UPDATE inbox SET folder='trash' WHERE msgid=?", inbox[inboxcur][0])
del inbox[inboxcur] del inbox[inboxcur]
scrollbox(d, unicode( scrollbox(d, unicode(
@ -431,7 +437,7 @@ def handlech(c, stdscr):
" \nbut the message is still on disk if you are desperate to recover it.")) " \nbut the message is still on disk if you are desperate to recover it."))
elif menutab == 2: elif menutab == 2:
a = "" a = ""
if addresses[addrcur][3] != 0: # if current address is a chan if addresses[addrcur][3] != 0: # if current address is a chan
a = addresses[addrcur][2] a = addresses[addrcur][2]
sendMessage(addresses[addrcur][2], a) sendMessage(addresses[addrcur][2], a)
elif menutab == 3: elif menutab == 3:
@ -467,7 +473,7 @@ def handlech(c, stdscr):
scrollbox(d, unicode(ascii(msg)), 30, 80) scrollbox(d, unicode(ascii(msg)), 30, 80)
else: else:
scrollbox(d, unicode("Could not fetch message.")) scrollbox(d, unicode("Could not fetch message."))
elif t == "2": # Move to trash elif t == "2": # Move to trash
sqlExecute( sqlExecute(
"UPDATE sent SET folder='trash' WHERE subject=? AND ackdata=?", "UPDATE sent SET folder='trash' WHERE subject=? AND ackdata=?",
sentbox[sentcur][4], sentbox[sentcur][4],
@ -495,7 +501,7 @@ def handlech(c, stdscr):
("6", "Delete"), ("6", "Delete"),
("7", "Special address behavior")]) ("7", "Special address behavior")])
if r == d.DIALOG_OK: if r == d.DIALOG_OK:
if t == "1": # Create new address if t == "1": # Create new address
set_background_title(d, "Create new address") set_background_title(d, "Create new address")
scrollbox( scrollbox(
d, unicode( d, unicode(
@ -598,12 +604,12 @@ def handlech(c, stdscr):
str(passphrase), shorten)) str(passphrase), shorten))
else: else:
scrollbox(d, unicode("Passphrases do not match")) scrollbox(d, unicode("Passphrases do not match"))
elif t == "2": # Send a message elif t == "2": # Send a message
a = "" a = ""
if addresses[addrcur][3] != 0: # if current address is a chan if addresses[addrcur][3] != 0: # if current address is a chan
a = addresses[addrcur][2] a = addresses[addrcur][2]
sendMessage(addresses[addrcur][2], a) sendMessage(addresses[addrcur][2], a)
elif t == "3": # Rename address label elif t == "3": # Rename address label
a = addresses[addrcur][2] a = addresses[addrcur][2]
label = addresses[addrcur][0] label = addresses[addrcur][0]
r, t = d.inputbox("New address label", init=label) r, t = d.inputbox("New address label", init=label)
@ -613,35 +619,35 @@ def handlech(c, stdscr):
# Write config # Write config
BMConfigParser().save() BMConfigParser().save()
addresses[addrcur][0] = label addresses[addrcur][0] = label
elif t == "4": # Enable address elif t == "4": # Enable address
a = addresses[addrcur][2] a = addresses[addrcur][2]
BMConfigParser().set(a, "enabled", "true") # Set config BMConfigParser().set(a, "enabled", "true") # Set config
# Write config # Write config
BMConfigParser().save() BMConfigParser().save()
# Change color # Change color
if BMConfigParser().safeGetBoolean(a, 'chan'): if BMConfigParser().safeGetBoolean(a, 'chan'):
addresses[addrcur][3] = 9 # orange addresses[addrcur][3] = 9 # orange
elif BMConfigParser().safeGetBoolean(a, 'mailinglist'): elif BMConfigParser().safeGetBoolean(a, 'mailinglist'):
addresses[addrcur][3] = 5 # magenta addresses[addrcur][3] = 5 # magenta
else: else:
addresses[addrcur][3] = 0 # black addresses[addrcur][3] = 0 # black
addresses[addrcur][1] = True addresses[addrcur][1] = True
shared.reloadMyAddressHashes() # Reload address hashes shared.reloadMyAddressHashes() # Reload address hashes
elif t == "5": # Disable address elif t == "5": # Disable address
a = addresses[addrcur][2] a = addresses[addrcur][2]
BMConfigParser().set(a, "enabled", "false") # Set config BMConfigParser().set(a, "enabled", "false") # Set config
addresses[addrcur][3] = 8 # Set color to gray addresses[addrcur][3] = 8 # Set color to gray
# Write config # Write config
BMConfigParser().save() BMConfigParser().save()
addresses[addrcur][1] = False addresses[addrcur][1] = False
shared.reloadMyAddressHashes() # Reload address hashes shared.reloadMyAddressHashes() # Reload address hashes
elif t == "6": # Delete address elif t == "6": # Delete address
r, t = d.inputbox("Type in \"I want to delete this address\"", width=50) r, t = d.inputbox("Type in \"I want to delete this address\"", width=50)
if r == d.DIALOG_OK and t == "I want to delete this address": if r == d.DIALOG_OK and t == "I want to delete this address":
BMConfigParser().remove_section(addresses[addrcur][2]) BMConfigParser().remove_section(addresses[addrcur][2])
BMConfigParser().save() BMConfigParser().save()
del addresses[addrcur] del addresses[addrcur]
elif t == "7": # Special address behavior elif t == "7": # Special address behavior
a = addresses[addrcur][2] a = addresses[addrcur][2]
set_background_title(d, "Special address behavior") set_background_title(d, "Special address behavior")
if BMConfigParser().safeGetBoolean(a, "chan"): if BMConfigParser().safeGetBoolean(a, "chan"):
@ -658,9 +664,9 @@ def handlech(c, stdscr):
if t == "1" and m: if t == "1" and m:
BMConfigParser().set(a, "mailinglist", "false") BMConfigParser().set(a, "mailinglist", "false")
if addresses[addrcur][1]: if addresses[addrcur][1]:
addresses[addrcur][3] = 0 # Set color to black addresses[addrcur][3] = 0 # Set color to black
else: else:
addresses[addrcur][3] = 8 # Set color to gray addresses[addrcur][3] = 8 # Set color to gray
elif t == "2" and m is False: elif t == "2" and m is False:
try: try:
mn = BMConfigParser().get(a, "mailinglistname") mn = BMConfigParser().get(a, "mailinglistname")
@ -671,7 +677,7 @@ def handlech(c, stdscr):
mn = t mn = t
BMConfigParser().set(a, "mailinglist", "true") BMConfigParser().set(a, "mailinglist", "true")
BMConfigParser().set(a, "mailinglistname", mn) BMConfigParser().set(a, "mailinglistname", mn)
addresses[addrcur][3] = 6 # Set color to magenta addresses[addrcur][3] = 6 # Set color to magenta
# Write config # Write config
BMConfigParser().save() BMConfigParser().save()
elif menutab == 5: elif menutab == 5:
@ -877,7 +883,7 @@ def sendMessage(sender="", recv="", broadcast=None, subject="", body="", reply=F
10, 10,
60) 60)
if r != d.DIALOG_OK: if r != d.DIALOG_OK:
global menutab # pylint: disable=global-statement global menutab
menutab = 6 menutab = 6
return return
recv = t recv = t
@ -890,7 +896,7 @@ def sendMessage(sender="", recv="", broadcast=None, subject="", body="", reply=F
if r != d.DIALOG_OK: if r != d.DIALOG_OK:
return return
broadcast = False broadcast = False
if t == "2": # Broadcast if t == "2": # Broadcast
broadcast = True broadcast = True
if subject == "" or reply: if subject == "" or reply:
r, t = d.inputbox("Message subject", width=60, init=subject) r, t = d.inputbox("Message subject", width=60, init=subject)
@ -906,9 +912,9 @@ def sendMessage(sender="", recv="", broadcast=None, subject="", body="", reply=F
if not broadcast: if not broadcast:
recvlist = [] recvlist = []
for i, item in enumerate(recv.replace(",", ";").split(";")): for _, item in enumerate(recv.replace(",", ";").split(";")):
recvlist.append(item.strip()) recvlist.append(item.strip())
list(set(recvlist)) # Remove exact duplicates list(set(recvlist)) # Remove exact duplicates
for addr in recvlist: for addr in recvlist:
if addr != "": if addr != "":
# pylint: disable=redefined-outer-name # pylint: disable=redefined-outer-name
@ -968,16 +974,16 @@ def sendMessage(sender="", recv="", broadcast=None, subject="", body="", reply=F
subject, subject,
body, body,
ackdata, ackdata,
int(time.time()), # sentTime (this will never change) int(time.time()), # sentTime (this will never change)
int(time.time()), # lastActionTime int(time.time()), # lastActionTime
0, # sleepTill time. This will get set when the POW gets done. 0, # sleepTill time. This will get set when the POW gets done.
"msgqueued", "msgqueued",
0, # retryNumber 0, # retryNumber
"sent", "sent",
2, # encodingType 2, # encodingType
BMConfigParser().getint('bitmessagesettings', 'ttl')) BMConfigParser().getint('bitmessagesettings', 'ttl'))
queues.workerQueue.put(("sendmessage", addr)) queues.workerQueue.put(("sendmessage", addr))
else: # Broadcast else: # Broadcast
if recv == "": if recv == "":
set_background_title(d, "Empty sender error") set_background_title(d, "Empty sender error")
scrollbox(d, unicode("You must specify an address to send the message from.")) scrollbox(d, unicode("You must specify an address to send the message from."))
@ -995,13 +1001,13 @@ def sendMessage(sender="", recv="", broadcast=None, subject="", body="", reply=F
subject, subject,
body, body,
ackdata, ackdata,
int(time.time()), # sentTime (this will never change) int(time.time()), # sentTime (this will never change)
int(time.time()), # lastActionTime int(time.time()), # lastActionTime
0, # sleepTill time. This will get set when the POW gets done. 0, # sleepTill time. This will get set when the POW gets done.
"broadcastqueued", "broadcastqueued",
0, # retryNumber 0, # retryNumber
"sent", # folder "sent", # folder
2, # encodingType 2, # encodingType
BMConfigParser().getint('bitmessagesettings', 'ttl')) BMConfigParser().getint('bitmessagesettings', 'ttl'))
queues.workerQueue.put(('sendbroadcast', '')) queues.workerQueue.put(('sendbroadcast', ''))
@ -1039,12 +1045,12 @@ def loadInbox():
fromlabel = "" fromlabel = ""
if BMConfigParser().has_section(fromaddr): if BMConfigParser().has_section(fromaddr):
fromlabel = BMConfigParser().get(fromaddr, "label") fromlabel = BMConfigParser().get(fromaddr, "label")
if fromlabel == "": # Check Address Book if fromlabel == "": # Check Address Book
qr = sqlQuery("SELECT label FROM addressbook WHERE address=?", fromaddr) qr = sqlQuery("SELECT label FROM addressbook WHERE address=?", fromaddr)
if qr != []: if qr != []:
for r in qr: for r in qr:
fromlabel, = r fromlabel, = r
if fromlabel == "": # Check Subscriptions if fromlabel == "": # Check Subscriptions
qr = sqlQuery("SELECT label FROM subscriptions WHERE address=?", fromaddr) qr = sqlQuery("SELECT label FROM subscriptions WHERE address=?", fromaddr)
if qr != []: if qr != []:
for r in qr: for r in qr:
@ -1170,7 +1176,7 @@ def loadSubscriptions():
def loadBlackWhiteList(): def loadBlackWhiteList():
"""load black/white list""" """load black/white list"""
global bwtype # pylint: disable=global-statement global bwtype
bwtype = BMConfigParser().get("bitmessagesettings", "blackwhitelist") bwtype = BMConfigParser().get("bitmessagesettings", "blackwhitelist")
if bwtype == "black": if bwtype == "black":
ret = sqlQuery("SELECT label, address, enabled FROM blacklist") ret = sqlQuery("SELECT label, address, enabled FROM blacklist")
@ -1183,10 +1189,10 @@ def loadBlackWhiteList():
def runwrapper(): def runwrapper():
"""Main method"""
sys.stdout = printlog sys.stdout = printlog
# sys.stderr = errlog # sys.stderr = errlog
# Load messages from database
loadInbox() loadInbox()
loadSent() loadSent()
loadAddrBook() loadAddrBook()
@ -1195,7 +1201,7 @@ def runwrapper():
stdscr = curses.initscr() stdscr = curses.initscr()
global logpad # pylint: disable=global-statement global logpad
logpad = curses.newpad(1024, curses.COLS) logpad = curses.newpad(1024, curses.COLS)
stdscr.nodelay(0) stdscr.nodelay(0)
@ -1207,26 +1213,27 @@ def runwrapper():
def run(stdscr): def run(stdscr):
"""Main loop"""
# Schedule inventory lookup data # Schedule inventory lookup data
resetlookups() resetlookups()
# Init color pairs # Init color pairs
if curses.has_colors(): if curses.has_colors():
curses.init_pair(1, curses.COLOR_RED, curses.COLOR_BLACK) # red curses.init_pair(1, curses.COLOR_RED, curses.COLOR_BLACK) # red
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK) # green curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK) # green
curses.init_pair(3, curses.COLOR_YELLOW, curses.COLOR_BLACK) # yellow curses.init_pair(3, curses.COLOR_YELLOW, curses.COLOR_BLACK) # yellow
curses.init_pair(4, curses.COLOR_BLUE, curses.COLOR_BLACK) # blue curses.init_pair(4, curses.COLOR_BLUE, curses.COLOR_BLACK) # blue
curses.init_pair(5, curses.COLOR_MAGENTA, curses.COLOR_BLACK) # magenta curses.init_pair(5, curses.COLOR_MAGENTA, curses.COLOR_BLACK) # magenta
curses.init_pair(6, curses.COLOR_CYAN, curses.COLOR_BLACK) # cyan curses.init_pair(6, curses.COLOR_CYAN, curses.COLOR_BLACK) # cyan
curses.init_pair(7, curses.COLOR_WHITE, curses.COLOR_BLACK) # white curses.init_pair(7, curses.COLOR_WHITE, curses.COLOR_BLACK) # white
if curses.can_change_color(): if curses.can_change_color():
curses.init_color(8, 500, 500, 500) # gray curses.init_color(8, 500, 500, 500) # gray
curses.init_pair(8, 8, 0) curses.init_pair(8, 8, 0)
curses.init_color(9, 844, 465, 0) # orange curses.init_color(9, 844, 465, 0) # orange
curses.init_pair(9, 9, 0) curses.init_pair(9, 9, 0)
else: else:
curses.init_pair(8, curses.COLOR_WHITE, curses.COLOR_BLACK) # grayish curses.init_pair(8, curses.COLOR_WHITE, curses.COLOR_BLACK) # grayish
curses.init_pair(9, curses.COLOR_YELLOW, curses.COLOR_BLACK) # orangish curses.init_pair(9, curses.COLOR_YELLOW, curses.COLOR_BLACK) # orangish
# Init list of address in 'Your Identities' tab # Init list of address in 'Your Identities' tab
configSections = BMConfigParser().addresses() configSections = BMConfigParser().addresses()
@ -1235,18 +1242,18 @@ def run(stdscr):
addresses.append([BMConfigParser().get(addressInKeysFile, "label"), isEnabled, addressInKeysFile]) addresses.append([BMConfigParser().get(addressInKeysFile, "label"), isEnabled, addressInKeysFile])
# Set address color # Set address color
if not isEnabled: if not isEnabled:
addresses[len(addresses) - 1].append(8) # gray addresses[len(addresses) - 1].append(8) # gray
elif BMConfigParser().safeGetBoolean(addressInKeysFile, 'chan'): elif BMConfigParser().safeGetBoolean(addressInKeysFile, 'chan'):
addresses[len(addresses) - 1].append(9) # orange addresses[len(addresses) - 1].append(9) # orange
elif BMConfigParser().safeGetBoolean(addressInKeysFile, 'mailinglist'): elif BMConfigParser().safeGetBoolean(addressInKeysFile, 'mailinglist'):
addresses[len(addresses) - 1].append(5) # magenta addresses[len(addresses) - 1].append(5) # magenta
else: else:
addresses[len(addresses) - 1].append(0) # black addresses[len(addresses) - 1].append(0) # black
addresses.reverse() addresses.reverse()
stdscr.clear() stdscr.clear()
redraw(stdscr) redraw(stdscr)
while quit is False: while quit_ is False:
drawtab(stdscr) drawtab(stdscr)
handlech(stdscr.getch(), stdscr) handlech(stdscr.getch(), stdscr)
@ -1259,5 +1266,4 @@ def doShutdown():
shutdown.doCleanShutdown() shutdown.doCleanShutdown()
sys.stdout = sys.__stdout__ sys.stdout = sys.__stdout__
sys.stderr = sys.__stderr__ sys.stderr = sys.__stderr__
os._exit(0) # pylint: disable=protected-access
os._exit(0) # pylint: disable=protected-access

13
src/class_objectProcessor.py
View File

@ -150,11 +150,14 @@ class objectProcessor(threading.Thread):
'ackreceived', int(time.time()), data[readPosition:]) 'ackreceived', int(time.time()), data[readPosition:])
queues.UISignalQueue.put(( queues.UISignalQueue.put((
'updateSentItemStatusByAckdata', 'updateSentItemStatusByAckdata',
(data[readPosition:], (
tr._translate( data[readPosition:],
"MainWindow", tr._translate(
"Acknowledgement of the message received %1").arg( "MainWindow",
l10n.formatTimestamp())))) "Acknowledgement of the message received %1"
).arg(l10n.formatTimestamp())
)
))
else: else:
logger.debug('This object is not an acknowledgement bound for me.') logger.debug('This object is not an acknowledgement bound for me.')

3
src/class_smtpDeliver.py
View File

@ -1,6 +1,5 @@
""" """
src/class_smtpDeliver.py SMTP client thread for delivering emails
========================
""" """
# pylint: disable=unused-variable # pylint: disable=unused-variable
import smtplib import smtplib

4
src/multiqueue.py
View File

@ -1,6 +1,6 @@
""" """
src/multiqueue.py A queue with multiple internal subqueues.
================= Elements are added into a random subqueue, and retrieval rotates
""" """
import queue as Queue import queue as Queue

1
src/network/addrthread.py
View File

@ -12,6 +12,7 @@ from network.threads import StoppableThread
class AddrThread(StoppableThread): class AddrThread(StoppableThread):
"""(Node) address broadcasting thread"""
name = "AddrBroadcaster" name = "AddrBroadcaster"
def run(self): def run(self):

34
src/network/advanceddispatcher.py
View File

@ -1,6 +1,5 @@
""" """
src/network/advanceddispatcher.py Improved version of asyncore dispatcher
=================================
""" """
# pylint: disable=attribute-defined-outside-init # pylint: disable=attribute-defined-outside-init
import socket import socket
@ -13,7 +12,8 @@ from network.threads import BusyError, nonBlocking
class ProcessingError(Exception): class ProcessingError(Exception):
"""General class for protocol parser exception, use as a base for others.""" """General class for protocol parser exception,
use as a base for others."""
pass pass
@ -23,7 +23,8 @@ class UnknownStateError(ProcessingError):
class AdvancedDispatcher(asyncore.dispatcher): class AdvancedDispatcher(asyncore.dispatcher):
"""Improved version of asyncore dispatcher, with buffers and protocol state.""" """Improved version of asyncore dispatcher,
with buffers and protocol state."""
# pylint: disable=too-many-instance-attributes # pylint: disable=too-many-instance-attributes
_buf_len = 131072 # 128kB _buf_len = 131072 # 128kB
@ -74,7 +75,8 @@ class AdvancedDispatcher(asyncore.dispatcher):
del self.read_buf[0:length] del self.read_buf[0:length]
def process(self): def process(self):
"""Process (parse) data that's in the buffer, as long as there is enough data and the connection is open.""" """Process (parse) data that's in the buffer,
as long as there is enough data and the connection is open."""
while self.connected and not state.shutdown: while self.connected and not state.shutdown:
try: try:
with nonBlocking(self.processingLock): with nonBlocking(self.processingLock):
@ -106,8 +108,9 @@ class AdvancedDispatcher(asyncore.dispatcher):
if asyncore.maxUploadRate > 0: if asyncore.maxUploadRate > 0:
self.uploadChunk = int(asyncore.uploadBucket) self.uploadChunk = int(asyncore.uploadBucket)
self.uploadChunk = min(self.uploadChunk, len(self.write_buf)) self.uploadChunk = min(self.uploadChunk, len(self.write_buf))
return asyncore.dispatcher.writable(self) and \ return asyncore.dispatcher.writable(self) and (
(self.connecting or (self.connected and self.uploadChunk > 0)) self.connecting or (
self.connected and self.uploadChunk > 0))
def readable(self): def readable(self):
"""Is the read buffer ready to accept data from the network?""" """Is the read buffer ready to accept data from the network?"""
@ -116,13 +119,15 @@ class AdvancedDispatcher(asyncore.dispatcher):
self.downloadChunk = int(asyncore.downloadBucket) self.downloadChunk = int(asyncore.downloadBucket)
try: try:
if self.expectBytes > 0 and not self.fullyEstablished: if self.expectBytes > 0 and not self.fullyEstablished:
self.downloadChunk = min(self.downloadChunk, self.expectBytes - len(self.read_buf)) self.downloadChunk = min(
self.downloadChunk, self.expectBytes - len(self.read_buf))
if self.downloadChunk < 0: if self.downloadChunk < 0:
self.downloadChunk = 0 self.downloadChunk = 0
except AttributeError: except AttributeError:
pass pass
return asyncore.dispatcher.readable(self) and \ return asyncore.dispatcher.readable(self) and (
(self.connecting or self.accepting or (self.connected and self.downloadChunk > 0)) self.connecting or self.accepting or (
self.connected and self.downloadChunk > 0))
def handle_read(self): def handle_read(self):
"""Append incoming data to the read buffer.""" """Append incoming data to the read buffer."""
@ -146,20 +151,21 @@ class AdvancedDispatcher(asyncore.dispatcher):
try: try:
asyncore.dispatcher.handle_connect_event(self) asyncore.dispatcher.handle_connect_event(self)
except socket.error as e: except socket.error as e:
if e.args[0] not in asyncore._DISCONNECTED: # pylint: disable=protected-access # pylint: disable=protected-access
if e.args[0] not in asyncore._DISCONNECTED:
raise raise
def handle_connect(self): def handle_connect(self):
"""Method for handling connection established implementations.""" """Method for handling connection established implementations."""
self.lastTx = time.time() self.lastTx = time.time()
def state_close(self): def state_close(self): # pylint: disable=no-self-use
"""Signal to the processing loop to end.""" """Signal to the processing loop to end."""
# pylint: disable=no-self-use
return False return False
def handle_close(self): def handle_close(self):
"""Callback for connection being closed, but can also be called directly when you want connection to close.""" """Callback for connection being closed,
but can also be called directly when you want connection to close."""
with self.readLock: with self.readLock:
self.read_buf = bytearray() self.read_buf = bytearray()
with self.writeLock: with self.writeLock:

7
src/network/announcethread.py
View File

@ -1,8 +1,6 @@
""" """
src/network/announcethread.py Announce myself (node address)
=================================
""" """
import time import time
import state import state
@ -43,6 +41,7 @@ class AnnounceThread(StoppableThread):
# connection.append_write_buf(BMProto.assembleAddr([addr])) # connection.append_write_buf(BMProto.assembleAddr([addr]))
Peer( Peer(
'127.0.0.1', '127.0.0.1',
BMConfigParser().safeGetInt('bitmessagesettings', 'port')), BMConfigParser().safeGetInt(
'bitmessagesettings', 'port')),
time.time()) time.time())
connection.append_write_buf(assemble_addr([addr])) connection.append_write_buf(assemble_addr([addr]))

16
src/network/assemble.py
View File

@ -1,7 +1,6 @@
""" """
Create bitmessage protocol command packets Create bitmessage protocol command packets
""" """
import struct import struct
import addresses import addresses
@ -15,18 +14,19 @@ def assemble_addr(peerList):
if isinstance(peerList, Peer): if isinstance(peerList, Peer):
peerList = [peerList] peerList = [peerList]
if not peerList: if not peerList:
return b'' return bytes()
retval = b'' retval = bytes()
for i in range(0, len(peerList), MAX_ADDR_COUNT): for i in range(0, len(peerList), MAX_ADDR_COUNT):
payload = addresses.encodeVarint( payload = addresses.encodeVarint(len(peerList[i:i + MAX_ADDR_COUNT]))
len(peerList[i:i + MAX_ADDR_COUNT]))
for stream, peer, timestamp in peerList[i:i + MAX_ADDR_COUNT]: for stream, peer, timestamp in peerList[i:i + MAX_ADDR_COUNT]:
payload += struct.pack( payload += struct.pack(
'>Q', int(timestamp)) # 64-bit time '>Q', int(timestamp)) # 64-bit time
payload += struct.pack('>I', stream) payload += struct.pack('>I', stream)
payload += struct.pack( # service bit flags offered by this node
'>q', 1) # service bit flags offered by this node payload += struct.pack('>q', 1)
payload += encodeHost(peer.host) payload += encodeHost(peer.host)
payload += struct.pack('>H', peer.port) # remote port # remote port
payload += struct.pack('>H', peer.port)
retval += CreatePacket('addr', payload) retval += CreatePacket('addr', payload)
return retval return retval

158
src/network/asyncore_pollchoose.py
View File

@ -1,56 +1,11 @@
"""
Basic infrastructure for asynchronous socket service clients and servers.
"""
# -*- Mode: Python -*- # -*- Mode: Python -*-
# Id: asyncore.py,v 2.51 2000/09/07 22:29:26 rushing Exp # Id: asyncore.py,v 2.51 2000/09/07 22:29:26 rushing Exp
# Author: Sam Rushing <rushing@nightmare.com> # Author: Sam Rushing <rushing@nightmare.com>
# pylint: disable=too-many-statements,too-many-branches,no-self-use,too-many-lines,attribute-defined-outside-init # pylint: disable=too-many-branches,too-many-lines,global-statement
# pylint: disable=global-statement # pylint: disable=redefined-builtin,no-self-use
"""
src/network/asyncore_pollchoose.py
==================================
# ======================================================================
# Copyright 1996 by Sam Rushing
#
# All Rights Reserved
#
# Permission to use, copy, modify, and distribute this software and
# its documentation for any purpose and without fee is hereby
# granted, provided that the above copyright notice appear in all
# copies and that both that copyright notice and this permission
# notice appear in supporting documentation, and that the name of Sam
# Rushing not be used in advertising or publicity pertaining to
# distribution of the software without specific, written prior
# permission.
#
# SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
# INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN
# NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR
# CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
# OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
# CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
# ======================================================================
Basic infrastructure for asynchronous socket service clients and servers.
There are only two ways to have a program on a single processor do "more
than one thing at a time". Multi-threaded programming is the simplest and
most popular way to do it, but there is another very different technique,
that lets you have nearly all the advantages of multi-threading, without
actually using multiple threads. it's really only practical if your program
is largely I/O bound. If your program is CPU bound, then pre-emptive
scheduled threads are probably what you really need. Network servers are
rarely CPU-bound, however.
If your operating system supports the select() system call in its I/O
library (and nearly all do), then you can use it to juggle multiple
communication channels at once; doing other work while your I/O is taking
place in the "background." Although this strategy can seem strange and
complex, especially at first, it is in many ways easier to understand and
control than multi-threaded programming. The module documented here solves
many of the difficult problems for you, making the task of building
sophisticated high-performance network servers and clients a snap.
"""
import os import os
import select import select
import socket import socket
@ -58,8 +13,9 @@ import sys
import time import time
import warnings import warnings
from errno import ( from errno import (
EADDRINUSE, EAGAIN, EALREADY, EBADF, ECONNABORTED, ECONNREFUSED, ECONNRESET, EHOSTUNREACH, EINPROGRESS, EINTR, EADDRINUSE, EAGAIN, EALREADY, EBADF, ECONNABORTED, ECONNREFUSED,
EINVAL, EISCONN, ENETUNREACH, ENOTCONN, ENOTSOCK, EPIPE, ESHUTDOWN, ETIMEDOUT, EWOULDBLOCK, errorcode ECONNRESET, EHOSTUNREACH, EINPROGRESS, EINTR, EINVAL, EISCONN, ENETUNREACH,
ENOTCONN, ENOTSOCK, EPIPE, ESHUTDOWN, ETIMEDOUT, EWOULDBLOCK, errorcode
) )
from threading import current_thread from threading import current_thread
@ -108,7 +64,8 @@ def _strerror(err):
class ExitNow(Exception): class ExitNow(Exception):
"""We don't use directly but may be necessary as we replace asyncore due to some library raising or expecting it""" """We don't use directly but may be necessary as we replace
asyncore due to some library raising or expecting it"""
pass pass
@ -153,7 +110,8 @@ def write(obj):
def set_rates(download, upload): def set_rates(download, upload):
"""Set throttling rates""" """Set throttling rates"""
global maxDownloadRate, maxUploadRate, downloadBucket, uploadBucket, downloadTimestamp, uploadTimestamp global maxDownloadRate, maxUploadRate, downloadBucket
global uploadBucket, downloadTimestamp, uploadTimestamp
maxDownloadRate = float(download) * 1024 maxDownloadRate = float(download) * 1024
maxUploadRate = float(upload) * 1024 maxUploadRate = float(upload) * 1024
@ -183,7 +141,8 @@ def update_received(download=0):
currentTimestamp = time.time() currentTimestamp = time.time()
receivedBytes += download receivedBytes += download
if maxDownloadRate > 0: if maxDownloadRate > 0:
bucketIncrease = maxDownloadRate * (currentTimestamp - downloadTimestamp) bucketIncrease = \
maxDownloadRate * (currentTimestamp - downloadTimestamp)
downloadBucket += bucketIncrease downloadBucket += bucketIncrease
if downloadBucket > maxDownloadRate: if downloadBucket > maxDownloadRate:
downloadBucket = int(maxDownloadRate) downloadBucket = int(maxDownloadRate)
@ -243,7 +202,6 @@ def readwrite(obj, flags):
def select_poller(timeout=0.0, map=None): def select_poller(timeout=0.0, map=None):
"""A poller which uses select(), available on most platforms.""" """A poller which uses select(), available on most platforms."""
# pylint: disable=redefined-builtin
if map is None: if map is None:
map = socket_map map = socket_map
@ -298,7 +256,6 @@ def select_poller(timeout=0.0, map=None):
def poll_poller(timeout=0.0, map=None): def poll_poller(timeout=0.0, map=None):
"""A poller which uses poll(), available on most UNIXen.""" """A poller which uses poll(), available on most UNIXen."""
# pylint: disable=redefined-builtin
if map is None: if map is None:
map = socket_map map = socket_map
@ -356,7 +313,6 @@ poll2 = poll3 = poll_poller
def epoll_poller(timeout=0.0, map=None): def epoll_poller(timeout=0.0, map=None):
"""A poller which uses epoll(), supported on Linux 2.5.44 and newer.""" """A poller which uses epoll(), supported on Linux 2.5.44 and newer."""
# pylint: disable=redefined-builtin
if map is None: if map is None:
map = socket_map map = socket_map
@ -412,7 +368,7 @@ def epoll_poller(timeout=0.0, map=None):
def kqueue_poller(timeout=0.0, map=None): def kqueue_poller(timeout=0.0, map=None):
"""A poller which uses kqueue(), BSD specific.""" """A poller which uses kqueue(), BSD specific."""
# pylint: disable=redefined-builtin,no-member # pylint: disable=no-member,too-many-statements
if map is None: if map is None:
map = socket_map map = socket_map
@ -440,14 +396,20 @@ def kqueue_poller(timeout=0.0, map=None):
poller_flags |= select.KQ_EV_ENABLE poller_flags |= select.KQ_EV_ENABLE
else: else:
poller_flags |= select.KQ_EV_DISABLE poller_flags |= select.KQ_EV_DISABLE
updates.append(select.kevent(fd, filter=select.KQ_FILTER_READ, flags=poller_flags)) updates.append(
select.kevent(
fd, filter=select.KQ_FILTER_READ,
flags=poller_flags))
if kq_filter & 2 != obj.poller_filter & 2: if kq_filter & 2 != obj.poller_filter & 2:
poller_flags = select.KQ_EV_ADD poller_flags = select.KQ_EV_ADD
if kq_filter & 2: if kq_filter & 2:
poller_flags |= select.KQ_EV_ENABLE poller_flags |= select.KQ_EV_ENABLE
else: else:
poller_flags |= select.KQ_EV_DISABLE poller_flags |= select.KQ_EV_DISABLE
updates.append(select.kevent(fd, filter=select.KQ_FILTER_WRITE, flags=poller_flags)) updates.append(
select.kevent(
fd, filter=select.KQ_FILTER_WRITE,
flags=poller_flags))
obj.poller_filter = kq_filter obj.poller_filter = kq_filter
if not selectables: if not selectables:
@ -481,7 +443,6 @@ def kqueue_poller(timeout=0.0, map=None):
def loop(timeout=30.0, _=False, map=None, count=None, poller=None): def loop(timeout=30.0, _=False, map=None, count=None, poller=None):
"""Poll in a loop, until count or timeout is reached""" """Poll in a loop, until count or timeout is reached"""
# pylint: disable=redefined-builtin
if map is None: if map is None:
map = socket_map map = socket_map
@ -522,7 +483,7 @@ def loop(timeout=30.0, _=False, map=None, count=None, poller=None):
class dispatcher(object): class dispatcher(object):
"""Dispatcher for socket objects""" """Dispatcher for socket objects"""
# pylint: disable=too-many-public-methods,too-many-instance-attributes,old-style-class # pylint: disable=too-many-public-methods,too-many-instance-attributes
debug = False debug = False
connected = False connected = False
@ -537,7 +498,6 @@ class dispatcher(object):
minTx = 1500 minTx = 1500
def __init__(self, sock=None, map=None): def __init__(self, sock=None, map=None):
# pylint: disable=redefined-builtin
if map is None: if map is None:
self._map = socket_map self._map = socket_map
else: else:
@ -586,8 +546,7 @@ class dispatcher(object):
def add_channel(self, map=None): def add_channel(self, map=None):
"""Add a channel""" """Add a channel"""
# pylint: disable=redefined-builtin # pylint: disable=attribute-defined-outside-init
if map is None: if map is None:
map = self._map map = self._map
map[self._fileno] = self map[self._fileno] = self
@ -596,8 +555,6 @@ class dispatcher(object):
def del_channel(self, map=None): def del_channel(self, map=None):
"""Delete a channel""" """Delete a channel"""
# pylint: disable=redefined-builtin
fd = self._fileno fd = self._fileno
if map is None: if map is None:
map = self._map map = self._map
@ -605,12 +562,14 @@ class dispatcher(object):
del map[fd] del map[fd]
if self._fileno: if self._fileno:
try: try:
kqueue_poller.pollster.control([select.kevent(fd, select.KQ_FILTER_READ, select.KQ_EV_DELETE)], 0) kqueue_poller.pollster.control([select.kevent(
except (AttributeError, KeyError, TypeError, IOError, OSError): fd, select.KQ_FILTER_READ, select.KQ_EV_DELETE)], 0)
except(AttributeError, KeyError, TypeError, IOError, OSError):
pass pass
try: try:
kqueue_poller.pollster.control([select.kevent(fd, select.KQ_FILTER_WRITE, select.KQ_EV_DELETE)], 0) kqueue_poller.pollster.control([select.kevent(
except (AttributeError, KeyError, TypeError, IOError, OSError): fd, select.KQ_FILTER_WRITE, select.KQ_EV_DELETE)], 0)
except(AttributeError, KeyError, TypeError, IOError, OSError):
pass pass
try: try:
epoll_poller.pollster.unregister(fd) epoll_poller.pollster.unregister(fd)
@ -627,8 +586,10 @@ class dispatcher(object):
self.poller_filter = 0 self.poller_filter = 0
self.poller_registered = False self.poller_registered = False
def create_socket(self, family=socket.AF_INET, socket_type=socket.SOCK_STREAM): def create_socket(
self, family=socket.AF_INET, socket_type=socket.SOCK_STREAM):
"""Create a socket""" """Create a socket"""
# pylint: disable=attribute-defined-outside-init
self.family_and_type = family, socket_type self.family_and_type = family, socket_type
sock = socket.socket(family, socket_type) sock = socket.socket(family, socket_type)
sock.setblocking(0) sock.setblocking(0)
@ -636,20 +597,16 @@ class dispatcher(object):
def set_socket(self, sock, map=None): def set_socket(self, sock, map=None):
"""Set socket""" """Set socket"""
# pylint: disable=redefined-builtin
self.socket = sock self.socket = sock
self._fileno = sock.fileno() self._fileno = sock.fileno()
self.add_channel(map) self.add_channel(map)
def set_reuse_addr(self): def set_reuse_addr(self):
"""try to re-use a server port if possible""" """try to re-use a server port if possible"""
try: try:
self.socket.setsockopt( self.socket.setsockopt(
socket.SOL_SOCKET, socket.SO_REUSEADDR, socket.SOL_SOCKET, socket.SO_REUSEADDR, self.socket.getsockopt(
self.socket.getsockopt(socket.SOL_SOCKET, socket.SOL_SOCKET, socket.SO_REUSEADDR) | 1
socket.SO_REUSEADDR) | 1
) )
except socket.error: except socket.error:
pass pass
@ -704,13 +661,16 @@ class dispatcher(object):
raise socket.error(err, errorcode[err]) raise socket.error(err, errorcode[err])
def accept(self): def accept(self):
"""Accept incoming connections. Returns either an address pair or None.""" """Accept incoming connections.
Returns either an address pair or None."""
try: try:
conn, addr = self.socket.accept() conn, addr = self.socket.accept()
except TypeError: except TypeError:
return None return None
except socket.error as why: except socket.error as why:
if why.args[0] in (EWOULDBLOCK, WSAEWOULDBLOCK, ECONNABORTED, EAGAIN, ENOTCONN): if why.args[0] in (
EWOULDBLOCK, WSAEWOULDBLOCK, ECONNABORTED,
EAGAIN, ENOTCONN):
return None return None
else: else:
raise raise
@ -771,9 +731,10 @@ class dispatcher(object):
except AttributeError: except AttributeError:
raise AttributeError("{} instance has no attribute {}" raise AttributeError("{} instance has no attribute {}"
.format(self.__class__.__name__, attr)) .format(self.__class__.__name__, attr))
else: else:
msg = "%(me)s.%(attr)s is deprecated; use %(me)s.socket.%(attr)s " \ msg = "%(me)s.%(attr)s is deprecated; use %(me)s.socket.%(attr)s"\
"instead" % {'me': self.__class__.__name__, 'attr': attr} " instead" % {'me': self.__class__.__name__, 'attr': attr}
warnings.warn(msg, DeprecationWarning, stacklevel=2) warnings.warn(msg, DeprecationWarning, stacklevel=2)
return retattr return retattr
@ -855,13 +816,8 @@ class dispatcher(object):
self.log_info( self.log_info(
'uncaptured python exception, closing channel %s (%s:%s %s)' % ( 'uncaptured python exception, closing channel %s (%s:%s %s)' % (
self_repr, self_repr, t, v, tbinfo),
t, 'error')
v,
tbinfo
),
'error'
)
self.handle_close() self.handle_close()
def handle_accept(self): def handle_accept(self):
@ -902,11 +858,8 @@ class dispatcher_with_send(dispatcher):
adds simple buffered output capability, useful for simple clients. adds simple buffered output capability, useful for simple clients.
[for more sophisticated usage use asynchat.async_chat] [for more sophisticated usage use asynchat.async_chat]
""" """
# pylint: disable=redefined-builtin
def __init__(self, sock=None, map=None): def __init__(self, sock=None, map=None):
# pylint: disable=redefined-builtin
dispatcher.__init__(self, sock, map) dispatcher.__init__(self, sock, map)
self.out_buffer = b'' self.out_buffer = b''
@ -941,7 +894,8 @@ def compact_traceback():
"""Return a compact traceback""" """Return a compact traceback"""
t, v, tb = sys.exc_info() t, v, tb = sys.exc_info()
tbinfo = [] tbinfo = []
if not tb: # Must have a traceback # Must have a traceback
if not tb:
raise AssertionError("traceback does not exist") raise AssertionError("traceback does not exist")
while tb: while tb:
tbinfo.append(( tbinfo.append((
@ -961,7 +915,6 @@ def compact_traceback():
def close_all(map=None, ignore_all=False): def close_all(map=None, ignore_all=False):
"""Close all connections""" """Close all connections"""
# pylint: disable=redefined-builtin
if map is None: if map is None:
map = socket_map map = socket_map
@ -998,13 +951,13 @@ def close_all(map=None, ignore_all=False):
if os.name == 'posix': if os.name == 'posix':
import fcntl import fcntl
class file_wrapper: class file_wrapper: # pylint: disable=old-style-class
""" """
Here we override just enough to make a file look like a socket for the purposes of asyncore. Here we override just enough to make a file look
like a socket for the purposes of asyncore.
The passed fd is automatically os.dup()'d The passed fd is automatically os.dup()'d
""" """
# pylint: disable=old-style-class
def __init__(self, fd): def __init__(self, fd):
self.fd = os.dup(fd) self.fd = os.dup(fd)
@ -1019,12 +972,11 @@ if os.name == 'posix':
def getsockopt(self, level, optname, buflen=None): def getsockopt(self, level, optname, buflen=None):
"""Fake getsockopt()""" """Fake getsockopt()"""
if (level == socket.SOL_SOCKET and if (level == socket.SOL_SOCKET and optname == socket.SO_ERROR and
optname == socket.SO_ERROR and
not buflen): not buflen):
return 0 return 0
raise NotImplementedError("Only asyncore specific behaviour " raise NotImplementedError(
"implemented.") "Only asyncore specific behaviour implemented.")
read = recv read = recv
write = send write = send
@ -1041,8 +993,6 @@ if os.name == 'posix':
"""A dispatcher for file_wrapper objects""" """A dispatcher for file_wrapper objects"""
def __init__(self, fd, map=None): def __init__(self, fd, map=None):
# pylint: disable=redefined-builtin
dispatcher.__init__(self, None, map) dispatcher.__init__(self, None, map)
self.connected = True self.connected = True
try: try:

49
src/network/bmobject.py
View File

@ -1,7 +1,6 @@
""" """
BMObject and it's exceptions. BMObject and it's exceptions.
""" """
import logging import logging
import time import time
@ -15,12 +14,14 @@ logger = logging.getLogger('default')
class BMObjectInsufficientPOWError(Exception): class BMObjectInsufficientPOWError(Exception):
"""Exception indicating the object doesn't have sufficient proof of work.""" """Exception indicating the object
doesn't have sufficient proof of work."""
errorCodes = ("Insufficient proof of work") errorCodes = ("Insufficient proof of work")
class BMObjectInvalidDataError(Exception): class BMObjectInvalidDataError(Exception):
"""Exception indicating the data being parsed does not match the specification.""" """Exception indicating the data being parsed
does not match the specification."""
errorCodes = ("Data invalid") errorCodes = ("Data invalid")
@ -30,7 +31,8 @@ class BMObjectExpiredError(Exception):
class BMObjectUnwantedStreamError(Exception): class BMObjectUnwantedStreamError(Exception):
"""Exception indicating the object is in a stream we didn't advertise as being interested in.""" """Exception indicating the object is in a stream
we didn't advertise as being interested in."""
errorCodes = ("Object in unwanted stream") errorCodes = ("Object in unwanted stream")
@ -44,9 +46,8 @@ class BMObjectAlreadyHaveError(Exception):
errorCodes = ("Already have this object") errorCodes = ("Already have this object")
class BMObject(object): class BMObject(object): # pylint: disable=too-many-instance-attributes
"""Bitmessage Object as a class.""" """Bitmessage Object as a class."""
# pylint: disable=too-many-instance-attributes
# max TTL, 28 days and 3 hours # max TTL, 28 days and 3 hours
maxTTL = 28 * 24 * 60 * 60 + 10800 maxTTL = 28 * 24 * 60 * 60 + 10800
@ -81,31 +82,36 @@ class BMObject(object):
raise BMObjectInsufficientPOWError() raise BMObjectInsufficientPOWError()
def checkEOLSanity(self): def checkEOLSanity(self):
"""Check if object's lifetime isn't ridiculously far in the past or future.""" """Check if object's lifetime
isn't ridiculously far in the past or future."""
# EOL sanity check # EOL sanity check
if self.expiresTime - int(time.time()) > BMObject.maxTTL: if self.expiresTime - int(time.time()) > BMObject.maxTTL:
logger.info( logger.info(
'This object\'s End of Life time is too far in the future. Ignoring it. Time is %i', 'This object\'s End of Life time is too far in the future.'
self.expiresTime) ' Ignoring it. Time is %i', self.expiresTime)
# .. todo:: remove from download queue # .. todo:: remove from download queue
raise BMObjectExpiredError() raise BMObjectExpiredError()
if self.expiresTime - int(time.time()) < BMObject.minTTL: if self.expiresTime - int(time.time()) < BMObject.minTTL:
logger.info( logger.info(
'This object\'s End of Life time was too long ago. Ignoring the object. Time is %i', 'This object\'s End of Life time was too long ago.'
self.expiresTime) ' Ignoring the object. Time is %i', self.expiresTime)
# .. todo:: remove from download queue # .. todo:: remove from download queue
raise BMObjectExpiredError() raise BMObjectExpiredError()
def checkStream(self): def checkStream(self):
"""Check if object's stream matches streams we are interested in""" """Check if object's stream matches streams we are interested in"""
if self.streamNumber not in state.streamsInWhichIAmParticipating: if self.streamNumber not in state.streamsInWhichIAmParticipating:
logger.debug('The streamNumber %i isn\'t one we are interested in.', self.streamNumber) logger.debug(
'The streamNumber %i isn\'t one we are interested in.',
self.streamNumber)
raise BMObjectUnwantedStreamError() raise BMObjectUnwantedStreamError()
def checkAlreadyHave(self): def checkAlreadyHave(self):
""" """
Check if we already have the object (so that we don't duplicate it in inventory or advertise it unnecessarily) Check if we already have the object
(so that we don't duplicate it in inventory
or advertise it unnecessarily)
""" """
# if it's a stem duplicate, pretend we don't have it # if it's a stem duplicate, pretend we don't have it
if Dandelion().hasHash(self.inventoryHash): if Dandelion().hasHash(self.inventoryHash):
@ -114,7 +120,8 @@ class BMObject(object):
raise BMObjectAlreadyHaveError() raise BMObjectAlreadyHaveError()
def checkObjectByType(self): def checkObjectByType(self):
"""Call a object type specific check (objects can have additional checks based on their types)""" """Call a object type specific check
(objects can have additional checks based on their types)"""
if self.objectType == protocol.OBJECT_GETPUBKEY: if self.objectType == protocol.OBJECT_GETPUBKEY:
self.checkGetpubkey() self.checkGetpubkey()
elif self.objectType == protocol.OBJECT_PUBKEY: elif self.objectType == protocol.OBJECT_PUBKEY:
@ -125,20 +132,21 @@ class BMObject(object):
self.checkBroadcast() self.checkBroadcast()
# other objects don't require other types of tests # other objects don't require other types of tests
def checkMessage(self): def checkMessage(self): # pylint: disable=no-self-use
""""Message" object type checks.""" """"Message" object type checks."""
# pylint: disable=no-self-use
return return
def checkGetpubkey(self): def checkGetpubkey(self):
""""Getpubkey" object type checks.""" """"Getpubkey" object type checks."""
if len(self.data) < 42: if len(self.data) < 42:
logger.info('getpubkey message doesn\'t contain enough data. Ignoring.') logger.info(
'getpubkey message doesn\'t contain enough data. Ignoring.')
raise BMObjectInvalidError() raise BMObjectInvalidError()
def checkPubkey(self): def checkPubkey(self):
""""Pubkey" object type checks.""" """"Pubkey" object type checks."""
if len(self.data) < 146 or len(self.data) > 440: # sanity check # sanity check
if len(self.data) < 146 or len(self.data) > 440:
logger.info('pubkey object too short or too long. Ignoring.') logger.info('pubkey object too short or too long. Ignoring.')
raise BMObjectInvalidError() raise BMObjectInvalidError()
@ -146,8 +154,9 @@ class BMObject(object):
""""Broadcast" object type checks.""" """"Broadcast" object type checks."""
if len(self.data) < 180: if len(self.data) < 180:
logger.debug( logger.debug(
'The payload length of this broadcast packet is unreasonably low.' 'The payload length of this broadcast'
' Someone is probably trying funny business. Ignoring message.') ' packet is unreasonably low. Someone is probably'
' trying funny business. Ignoring message.')
raise BMObjectInvalidError() raise BMObjectInvalidError()
# this isn't supported anymore # this isn't supported anymore

58
src/network/bmproto.py
View File

@ -1,8 +1,7 @@
""" """
src/network/bmproto.py Bitmessage Protocol
==================================
""" """
# pylint: disable=attribute-defined-outside-init # pylint: disable=attribute-defined-outside-init, too-few-public-methods
import base64 import base64
import hashlib import hashlib
import logging import logging
@ -19,17 +18,16 @@ import state
from bmconfigparser import BMConfigParser from bmconfigparser import BMConfigParser
from inventory import Inventory from inventory import Inventory
from network.advanceddispatcher import AdvancedDispatcher from network.advanceddispatcher import AdvancedDispatcher
from network.constants import (
ADDRESS_ALIVE,
MAX_MESSAGE_SIZE,
MAX_OBJECT_COUNT,
MAX_OBJECT_PAYLOAD_SIZE,
MAX_TIME_OFFSET)
from network.dandelion import Dandelion
from network.bmobject import ( from network.bmobject import (
BMObject, BMObjectInsufficientPOWError, BMObjectInvalidDataError, BMObject, BMObjectInsufficientPOWError, BMObjectInvalidDataError,
BMObjectExpiredError, BMObjectUnwantedStreamError, BMObjectExpiredError, BMObjectUnwantedStreamError,
BMObjectInvalidError, BMObjectAlreadyHaveError) BMObjectInvalidError, BMObjectAlreadyHaveError
)
from network.constants import (
ADDRESS_ALIVE, MAX_MESSAGE_SIZE, MAX_OBJECT_COUNT,
MAX_OBJECT_PAYLOAD_SIZE, MAX_TIME_OFFSET
)
from network.dandelion import Dandelion
from network.proxy import ProxyError from network.proxy import ProxyError
from network.objectracker import missingObjects, ObjectTracker from network.objectracker import missingObjects, ObjectTracker
from network.node import Node, Peer from network.node import Node, Peer
@ -73,7 +71,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
# pylint: disable=too-many-instance-attributes, too-many-public-methods # pylint: disable=too-many-instance-attributes, too-many-public-methods
timeOffsetWrongCount = 0 timeOffsetWrongCount = 0
def __init__(self, address=None, sock=None): # pylint: disable=unused-argument, super-init-not-called def __init__(self, address=None, sock=None):
# pylint: disable=unused-argument, super-init-not-called
AdvancedDispatcher.__init__(self, sock) AdvancedDispatcher.__init__(self, sock)
self.isOutbound = False self.isOutbound = False
# packet/connection from a local IP # packet/connection from a local IP
@ -190,7 +189,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
def decode_payload_varint(self): def decode_payload_varint(self):
"""Decode a varint from the payload""" """Decode a varint from the payload"""
value, offset = addresses.decodeVarint(self.payload[self.payloadOffset:]) value, offset = addresses.decodeVarint(
self.payload[self.payloadOffset:])
self.payloadOffset += offset self.payloadOffset += offset
return value return value
@ -212,8 +212,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
return Node(services, host, port) return Node(services, host, port)
def decode_payload_content(self, pattern="v"): # pylint: disable=too-many-branches, too-many-statements # pylint: disable=too-many-branches, too-many-statements
def decode_payload_content(self, pattern="v"):
""" """
Decode the payload depending on pattern: Decode the payload depending on pattern:
@ -229,7 +229,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
, = end of array , = end of array
""" """
def decode_simple(self, char="v"): # pylint: disable=inconsistent-return-statements # pylint: disable=inconsistent-return-statements
def decode_simple(self, char="v"):
"""Decode the payload using one char pattern""" """Decode the payload using one char pattern"""
if char == "v": if char == "v":
return self.decode_payload_varint() return self.decode_payload_varint()
@ -442,6 +443,7 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
if self.object.inventoryHash in Inventory() and Dandelion().hasHash(self.object.inventoryHash): if self.object.inventoryHash in Inventory() and Dandelion().hasHash(self.object.inventoryHash):
Dandelion().removeHash(self.object.inventoryHash, "cycle detection") Dandelion().removeHash(self.object.inventoryHash, "cycle detection")
[self.object.inventoryHash] = ( [self.object.inventoryHash] = (
self.object.objectType, self.object.streamNumber, self.object.objectType, self.object.streamNumber,
memoryview(self.payload[objectOffset:]), self.object.expiresTime, memoryview(self.payload[objectOffset:]), self.object.expiresTime,
memoryview(self.object.tag) memoryview(self.object.tag)
@ -463,23 +465,26 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
for i in addresses: for i in addresses:
seenTime, stream, _, ip, port = i seenTime, stream, _, ip, port = i
decodedIP = protocol.checkIPAddress(bytes(ip)) decodedIP = protocol.checkIPAddress(bytes(ip))
if stream not in state.streamsInWhichIAmParticipating: if stream not in state.streamsInWhichIAmParticipating:
continue continue
if ( if (
decodedIP and time.time() - seenTime > 0 and decodedIP and time.time() - seenTime > 0 and
seenTime > time.time() - ADDRESS_ALIVE and seenTime > time.time() - ADDRESS_ALIVE and
port > 0 port > 0
): ):
peer = Peer(decodedIP, port) peer = Peer(decodedIP, port)
try: try:
if knownnodes.knownNodes[stream][peer]["lastseen"] > seenTime: if knownnodes.knownNodes[stream][peer]["lastseen"] > \
seenTime:
continue continue
except KeyError: except KeyError:
pass pass
if len(knownnodes.knownNodes[stream]) < int(BMConfigParser().safeGet("knownnodes", "maxnodes")): if len(knownnodes.knownNodes[stream]) < int(BMConfigParser().safeGet("knownnodes", "maxnodes")):
with knownnodes.knownNodesLock: with knownnodes.knownNodesLock:
try: try:
knownnodes.knownNodes[stream][peer]["lastseen"] = seenTime knownnodes.knownNodes[stream][peer]["lastseen"] = \
seenTime
except (TypeError, KeyError): except (TypeError, KeyError):
knownnodes.knownNodes[stream][peer] = { knownnodes.knownNodes[stream][peer] = {
"lastseen": seenTime, "lastseen": seenTime,
@ -573,7 +578,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
length=self.payloadLength, expectBytes=0) length=self.payloadLength, expectBytes=0)
return False return False
def peerValidityChecks(self): # pylint: disable=too-many-return-statements # pylint: disable=too-many-return-statements
def peerValidityChecks(self):
"""Check the validity of the peer""" """Check the validity of the peer"""
if self.remoteProtocolVersion < 3: if self.remoteProtocolVersion < 3:
self.append_write_buf(protocol.assembleErrorMessage( self.append_write_buf(protocol.assembleErrorMessage(
@ -585,8 +591,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
return False return False
if self.timeOffset > MAX_TIME_OFFSET: if self.timeOffset > MAX_TIME_OFFSET:
self.append_write_buf(protocol.assembleErrorMessage( self.append_write_buf(protocol.assembleErrorMessage(
errorText="Your time is too far in the future compared to mine." errorText="Your time is too far in the future"
" Closing connection.", fatal=2)) " compared to mine. Closing connection.", fatal=2))
logger.info( logger.info(
"%s's time is too far in the future (%s seconds)." "%s's time is too far in the future (%s seconds)."
" Closing connection to it.", self.destination, self.timeOffset) " Closing connection to it.", self.destination, self.timeOffset)
@ -608,8 +614,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
errorText="We don't have shared stream interests." errorText="We don't have shared stream interests."
" Closing connection.", fatal=2)) " Closing connection.", fatal=2))
logger.debug( logger.debug(
'Closed connection to %s because there is no overlapping interest' 'Closed connection to %s because there is no overlapping'
' in streams.', self.destination) ' interest in streams.', self.destination)
return False return False
if self.destination in connectionpool.BMConnectionPool().inboundConnections: if self.destination in connectionpool.BMConnectionPool().inboundConnections:
try: try:

8
src/network/connectionchooser.py
View File

@ -1,3 +1,6 @@
"""
Select which node to connect to
"""
# pylint: disable=too-many-branches # pylint: disable=too-many-branches
import logging import logging
import random # nosec import random # nosec
@ -12,6 +15,7 @@ logger = logging.getLogger('default')
def getDiscoveredPeer(): def getDiscoveredPeer():
"""Get a peer from the local peer discovery list"""
try: try:
peer = random.choice([key for key in state.discoveredPeers.keys()]) peer = random.choice([key for key in state.discoveredPeers.keys()])
except (IndexError, KeyError): except (IndexError, KeyError):
@ -24,6 +28,7 @@ def getDiscoveredPeer():
def chooseConnection(stream): def chooseConnection(stream):
"""Returns an appropriate connection"""
haveOnion = BMConfigParser().safeGet( haveOnion = BMConfigParser().safeGet(
"bitmessagesettings", "socksproxytype")[0:5] == 'SOCKS' "bitmessagesettings", "socksproxytype")[0:5] == 'SOCKS'
onionOnly = BMConfigParser().safeGetBoolean( onionOnly = BMConfigParser().safeGetBoolean(
@ -49,7 +54,8 @@ def chooseConnection(stream):
logger.warning('Error in {}'.format(peer)) logger.warning('Error in {}'.format(peer))
rating = 0 rating = 0
if haveOnion: if haveOnion:
# do not connect to raw IP addresses--keep all traffic within Tor overlay # do not connect to raw IP addresses
# --keep all traffic within Tor overlay
if onionOnly and not peer.host.endswith('.onion'): if onionOnly and not peer.host.endswith('.onion'):
continue continue
# onion addresses have a higher priority when SOCKS # onion addresses have a higher priority when SOCKS

18
src/network/constants.py
View File

@ -3,9 +3,15 @@ Network protocol constants
""" """
ADDRESS_ALIVE = 10800 #: address is online if online less than this many seconds ago #: address is online if online less than this many seconds ago
MAX_ADDR_COUNT = 1000 #: protocol specification says max 1000 addresses in one addr command ADDRESS_ALIVE = 10800
MAX_MESSAGE_SIZE = 1600100 #: ~1.6 MB which is the maximum possible size of an inv message. #: protocol specification says max 1000 addresses in one addr command
MAX_OBJECT_PAYLOAD_SIZE = 2**18 #: 2**18 = 256kB is the maximum size of an object payload MAX_ADDR_COUNT = 1000
MAX_OBJECT_COUNT = 50000 #: protocol specification says max 50000 objects in one inv command #: ~1.6 MB which is the maximum possible size of an inv message.
MAX_TIME_OFFSET = 3600 #: maximum time offset MAX_MESSAGE_SIZE = 1600100
#: 2**18 = 256kB is the maximum size of an object payload
MAX_OBJECT_PAYLOAD_SIZE = 2**18
#: protocol specification says max 50000 objects in one inv command
MAX_OBJECT_COUNT = 50000
#: maximum time offset
MAX_TIME_OFFSET = 3600

7
src/network/dandelion.py
View File

@ -1,10 +1,9 @@
""" """
src/network/dandelion.py Dandelion class definition, tracks stages
========================
""" """
import logging import logging
from collections import namedtuple from collections import namedtuple
from random import choice, sample, expovariate from random import choice, expovariate, sample
from threading import RLock from threading import RLock
from time import time from time import time
@ -131,6 +130,8 @@ class Dandelion(object):
for k in ( for k in (
k for k, v in iter(self.nodeMap.items()) if v == connection k for k, v in iter(self.nodeMap.items()) if v == connection
# k for k, v in self.nodeMap.iteritems()
# if v == connection
): ):
self.nodeMap[k] = None self.nodeMap[k] = None
for k, v in { for k, v in {

4
src/network/downloadthread.py
View File

@ -1,7 +1,6 @@
""" """
`DownloadThread` class definition `DownloadThread` class definition
""" """
import time import time
import addresses import addresses
@ -31,6 +30,9 @@ class DownloadThread(StoppableThread):
deadline = time.time() - self.requestExpires deadline = time.time() - self.requestExpires
try: try:
toDelete = [k for k, v in iter(missingObjects.items()) if v < deadline] toDelete = [k for k, v in iter(missingObjects.items()) if v < deadline]
# toDelete = [
# k for k, v in missingObjects.iteritems()
# if v < deadline]
except RuntimeError: except RuntimeError:
pass pass
else: else:

20
src/network/invthread.py
View File

@ -1,6 +1,5 @@
""" """
src/network/invthread.py Thread to send inv annoucements
========================
""" """
import queue as Queue import queue as Queue
import random import random
@ -34,7 +33,7 @@ def handleExpiredDandelion(expired):
class InvThread(StoppableThread): class InvThread(StoppableThread):
"""A thread to send inv annoucements.""" """Main thread that sends inv annoucements"""
name = "InvBroadcaster" name = "InvBroadcaster"
@ -43,12 +42,13 @@ class InvThread(StoppableThread):
"""Locally generated inventory items require special handling""" """Locally generated inventory items require special handling"""
Dandelion().addHash(hashId, stream=stream) Dandelion().addHash(hashId, stream=stream)
for connection in BMConnectionPool().connections(): for connection in BMConnectionPool().connections():
if state.dandelion and connection != Dandelion().objectChildStem(hashId): if state.dandelion and connection != \
Dandelion().objectChildStem(hashId):
continue continue
connection.objectsNewToThem[hashId] = time() connection.objectsNewToThem[hashId] = time()
def run(self): # pylint: disable=too-many-branches def run(self): # pylint: disable=too-many-branches
while not state.shutdown: # pylint: disable=too-many-nested-blocks while not state.shutdown: # pylint: disable=too-many-nested-blocks
chunk = [] chunk = []
while True: while True:
# Dandelion fluff trigger by expiration # Dandelion fluff trigger by expiration
@ -92,15 +92,17 @@ class InvThread(StoppableThread):
random.shuffle(fluffs) random.shuffle(fluffs)
connection.append_write_buf(protocol.CreatePacket( connection.append_write_buf(protocol.CreatePacket(
'inv', 'inv',
addresses.encodeVarint(len(fluffs)) + ''.join(fluffs))) addresses.encodeVarint(
len(fluffs)) + ''.join(fluffs)))
if stems: if stems:
random.shuffle(stems) random.shuffle(stems)
connection.append_write_buf(protocol.CreatePacket( connection.append_write_buf(protocol.CreatePacket(
'dinv', 'dinv',
addresses.encodeVarint(len(stems)) + ''.join(stems))) addresses.encodeVarint(
len(stems)) + ''.join(stems)))
invQueue.iterate() invQueue.iterate()
for i in range(len(chunk)): for _ in range(len(chunk)):
invQueue.task_done() invQueue.task_done()
if Dandelion().refresh < time(): if Dandelion().refresh < time():

5
src/network/networkthread.py
View File

@ -1,3 +1,6 @@
"""
A thread to handle network concerns
"""
import network.asyncore_pollchoose as asyncore import network.asyncore_pollchoose as asyncore
import state import state
from network.connectionpool import BMConnectionPool from network.connectionpool import BMConnectionPool
@ -6,7 +9,7 @@ from network.threads import StoppableThread
class BMNetworkThread(StoppableThread): class BMNetworkThread(StoppableThread):
"""A thread to handle network concerns""" """Main network thread"""
name = "Asyncore" name = "Asyncore"
def run(self): def run(self):

26
src/network/objectracker.py
View File

@ -1,6 +1,5 @@
""" """
src/network/objectracker.py Module for tracking objects
===========================
""" """
import time import time
from threading import RLock from threading import RLock
@ -50,15 +49,18 @@ class ObjectTracker(object):
"""Init bloom filter for tracking. WIP.""" """Init bloom filter for tracking. WIP."""
if haveBloom: if haveBloom:
# lock? # lock?
self.invBloom = BloomFilter(capacity=ObjectTracker.invInitialCapacity, self.invBloom = BloomFilter(
error_rate=ObjectTracker.invErrorRate) capacity=ObjectTracker.invInitialCapacity,
error_rate=ObjectTracker.invErrorRate)
def initAddrBloom(self): def initAddrBloom(self):
"""Init bloom filter for tracking addrs, WIP. This either needs to be moved to addrthread.py or removed.""" """Init bloom filter for tracking addrs, WIP.
This either needs to be moved to addrthread.py or removed."""
if haveBloom: if haveBloom:
# lock? # lock?
self.addrBloom = BloomFilter(capacity=ObjectTracker.invInitialCapacity, self.addrBloom = BloomFilter(
error_rate=ObjectTracker.invErrorRate) capacity=ObjectTracker.invInitialCapacity,
error_rate=ObjectTracker.invErrorRate)
def clean(self): def clean(self):
"""Clean up tracking to prevent memory bloat""" """Clean up tracking to prevent memory bloat"""
@ -72,6 +74,10 @@ class ObjectTracker(object):
deadline = time.time() - ObjectTracker.trackingExpires deadline = time.time() - ObjectTracker.trackingExpires
with self.objectsNewToThemLock: with self.objectsNewToThemLock:
self.objectsNewToThem = {k: v for k, v in iter(self.objectsNewToThem.items()) if v >= deadline} self.objectsNewToThem = {k: v for k, v in iter(self.objectsNewToThem.items()) if v >= deadline}
# self.objectsNewToThem = {
# k: v
# for k, v in self.objectsNewToThem.iteritems()
# if v >= deadline}
self.lastCleaned = time.time() self.lastCleaned = time.time()
def hasObj(self, hashid): def hasObj(self, hashid):
@ -102,10 +108,12 @@ class ObjectTracker(object):
del i.objectsNewToMe[hashid] del i.objectsNewToMe[hashid]
except KeyError: except KeyError:
if streamNumber in i.streams and ( if streamNumber in i.streams and (
not Dandelion().hasHash(hashid) or Dandelion().objectChildStem(hashid) == i): not Dandelion().hasHash(hashid) or
Dandelion().objectChildStem(hashid) == i):
with i.objectsNewToThemLock: with i.objectsNewToThemLock:
i.objectsNewToThem[hashid] = time.time() i.objectsNewToThem[hashid] = time.time()
# update stream number, which we didn't have when we just received the dinv # update stream number,
# which we didn't have when we just received the dinv
# also resets expiration of the stem mode # also resets expiration of the stem mode
Dandelion().setHashStream(hashid, streamNumber) Dandelion().setHashStream(hashid, streamNumber)

6
src/network/proxy.py
View File

@ -1,6 +1,5 @@
""" """
src/network/proxy.py Set proxy if avaiable otherwise exception
====================
""" """
# pylint: disable=protected-access # pylint: disable=protected-access
import logging import logging
@ -123,8 +122,7 @@ class Proxy(AdvancedDispatcher):
BMConfigParser().safeGet( BMConfigParser().safeGet(
"bitmessagesettings", "socksusername"), "bitmessagesettings", "socksusername"),
BMConfigParser().safeGet( BMConfigParser().safeGet(
"bitmessagesettings", "sockspassword") "bitmessagesettings", "sockspassword"))
)
else: else:
self.auth = None self.auth = None
self.connect( self.connect(

30
src/network/randomtrackingdict.py
View File

@ -1,8 +1,6 @@
""" """
src/randomtrackingdict.py Track randomize ordered dict
=========================
""" """
import random import random
from threading import RLock from threading import RLock
from time import time from time import time
@ -14,12 +12,12 @@ class RandomTrackingDict(object):
""" """
Dict with randomised order and tracking. Dict with randomised order and tracking.
Keeps a track of how many items have been requested from the dict, and timeouts. Keeps a track of how many items have been requested from the dict,
Resets after all objects have been retrieved and timed out. and timeouts. Resets after all objects have been retrieved and timed out.
The main purpose of this isn't as much putting related code together The main purpose of this isn't as much putting related code together
as performance optimisation and anonymisation of downloading of objects from other peers. as performance optimisation and anonymisation of downloading of objects
If done using a standard dict or array, it takes too much CPU (and looks convoluted). from other peers. If done using a standard dict or array, it takes
Randomisation helps with anonymity. too much CPU (and looks convoluted). Randomisation helps with anonymity.
""" """
# pylint: disable=too-many-instance-attributes # pylint: disable=too-many-instance-attributes
maxPending = 10 maxPending = 10
@ -87,13 +85,14 @@ class RandomTrackingDict(object):
def setMaxPending(self, maxPending): def setMaxPending(self, maxPending):
""" """
Sets maximum number of objects that can be retrieved from the class simultaneously as long as there is no Sets maximum number of objects that can be retrieved from the class
timeout simultaneously as long as there is no timeout
""" """
self.maxPending = maxPending self.maxPending = maxPending
def setPendingTimeout(self, pendingTimeout): def setPendingTimeout(self, pendingTimeout):
"""Sets how long to wait for a timeout if max pending is reached (or all objects have been retrieved)""" """Sets how long to wait for a timeout if max pending is reached
(or all objects have been retrieved)"""
self.pendingTimeout = pendingTimeout self.pendingTimeout = pendingTimeout
def setLastObject(self): def setLastObject(self):
@ -101,7 +100,8 @@ class RandomTrackingDict(object):
self.lastObject = time() self.lastObject = time()
def randomKeys(self, count=1): def randomKeys(self, count=1):
"""Retrieve count random keys from the dict that haven't already been retrieved""" """Retrieve count random keys from the dict
that haven't already been retrieved"""
if self.len == 0 or ((self.pendingLen >= self.maxPending or if self.len == 0 or ((self.pendingLen >= self.maxPending or
self.pendingLen == self.len) and self.lastPoll + self.pendingLen == self.len) and self.lastPoll +
self.pendingTimeout > time()): self.pendingTimeout > time()):
@ -111,13 +111,15 @@ class RandomTrackingDict(object):
with self.lock: with self.lock:
# reset if we've requested all # reset if we've requested all
# and if last object received too long time ago # and if last object received too long time ago
if self.pendingLen == self.len and self.lastObject + self.pendingTimeout < time(): if self.pendingLen == self.len and self.lastObject + \
self.pendingTimeout < time():
self.pendingLen = 0 self.pendingLen = 0
self.setLastObject() self.setLastObject()
available = self.len - self.pendingLen available = self.len - self.pendingLen
if count > available: if count > available:
count = available count = available
randomIndex = helper_random.randomsample(range(self.len - self.pendingLen), count) randomIndex = helper_random.randomsample(
range(self.len - self.pendingLen), count)
retval = [self.indexDict[i] for i in randomIndex] retval = [self.indexDict[i] for i in randomIndex]
for i in sorted(randomIndex, reverse=True): for i in sorted(randomIndex, reverse=True):

11
src/network/receivequeuethread.py
View File

@ -1,3 +1,6 @@
"""
Process data incoming from network
"""
import errno import errno
import queue as Queue import queue as Queue
import socket import socket
@ -10,6 +13,8 @@ from network.threads import StoppableThread
class ReceiveQueueThread(StoppableThread): class ReceiveQueueThread(StoppableThread):
"""This thread processes data received from the network
(which is done by the asyncore thread)"""
def __init__(self, num=0): def __init__(self, num=0):
super(ReceiveQueueThread, self).__init__(name="ReceiveQueue_%i" % num) super(ReceiveQueueThread, self).__init__(name="ReceiveQueue_%i" % num)
@ -32,12 +37,14 @@ class ReceiveQueueThread(StoppableThread):
try: try:
connection = BMConnectionPool().getConnectionByAddr(dest) connection = BMConnectionPool().getConnectionByAddr(dest)
except KeyError: # connection object not found # connection object not found
except KeyError:
receiveDataQueue.task_done() receiveDataQueue.task_done()
continue continue
try: try:
connection.process() connection.process()
except UnknownStateError: # state isn't implemented # state isn't implemented
except UnknownStateError:
pass pass
except socket.error as err: except socket.error as err:
if err.errno == errno.EBADF: if err.errno == errno.EBADF:

3
src/network/socks4a.py
View File

@ -1,6 +1,5 @@
""" """
src/network/socks4a.py SOCKS4a proxy module
=================================
""" """
# pylint: disable=attribute-defined-outside-init # pylint: disable=attribute-defined-outside-init
import socket import socket

7
src/network/socks5.py
View File

@ -1,7 +1,5 @@
""" """
src/network/socks5.py SOCKS5 proxy module
=====================
""" """
# pylint: disable=attribute-defined-outside-init # pylint: disable=attribute-defined-outside-init
@ -157,7 +155,8 @@ class Socks5(Proxy):
return True return True
def proxy_sock_name(self): def proxy_sock_name(self):
"""Handle return value when using SOCKS5 for DNS resolving instead of connecting.""" """Handle return value when using SOCKS5
for DNS resolving instead of connecting."""
return socket.inet_ntoa(self.__proxysockname[0]) return socket.inet_ntoa(self.__proxysockname[0])

10
src/network/stats.py
View File

@ -1,6 +1,5 @@
""" """
src/network/stats.py Network statistics
====================
""" """
import time import time
@ -34,7 +33,9 @@ def uploadSpeed():
currentTimestamp = time.time() currentTimestamp = time.time()
if int(lastSentTimestamp) < int(currentTimestamp): if int(lastSentTimestamp) < int(currentTimestamp):
currentSentBytes = asyncore.sentBytes currentSentBytes = asyncore.sentBytes
currentSentSpeed = int((currentSentBytes - lastSentBytes) / (currentTimestamp - lastSentTimestamp)) currentSentSpeed = int(
(currentSentBytes - lastSentBytes) / (
currentTimestamp - lastSentTimestamp))
lastSentBytes = currentSentBytes lastSentBytes = currentSentBytes
lastSentTimestamp = currentTimestamp lastSentTimestamp = currentTimestamp
return currentSentSpeed return currentSentSpeed
@ -53,7 +54,8 @@ def downloadSpeed():
if int(lastReceivedTimestamp) < int(currentTimestamp): if int(lastReceivedTimestamp) < int(currentTimestamp):
currentReceivedBytes = asyncore.receivedBytes currentReceivedBytes = asyncore.receivedBytes
currentReceivedSpeed = int( currentReceivedSpeed = int(
(currentReceivedBytes - lastReceivedBytes) / (currentTimestamp - lastReceivedTimestamp)) (currentReceivedBytes - lastReceivedBytes) / (
currentTimestamp - lastReceivedTimestamp))
lastReceivedBytes = currentReceivedBytes lastReceivedBytes = currentReceivedBytes
lastReceivedTimestamp = currentTimestamp lastReceivedTimestamp = currentTimestamp
return currentReceivedSpeed return currentReceivedSpeed

26
src/network/tcp.py
View File

@ -1,9 +1,7 @@
"""
TCP protocol handler
"""
# pylint: disable=too-many-ancestors # pylint: disable=too-many-ancestors
"""
src/network/tcp.py
==================
"""
import logging import logging
import math import math
import random import random
@ -33,13 +31,13 @@ from network.tls import TLSDispatcher
from .node import Peer from .node import Peer
from queues import UISignalQueue, invQueue, receiveDataQueue from queues import UISignalQueue, invQueue, receiveDataQueue
logger = logging.getLogger('default') logger = logging.getLogger('default')
class TCPConnection(BMProto, TLSDispatcher): class TCPConnection(BMProto, TLSDispatcher):
# pylint: disable=too-many-instance-attributes # pylint: disable=too-many-instance-attributes
""" """
.. todo:: Look to understand and/or fix the non-parent-init-called .. todo:: Look to understand and/or fix the non-parent-init-called
""" """
@ -90,7 +88,8 @@ class TCPConnection(BMProto, TLSDispatcher):
not protocol.checkSocksIP(self.destination.host) not protocol.checkSocksIP(self.destination.host)
) )
except socket.error: except socket.error:
pass # it's probably a hostname # it's probably a hostname
pass
self.network_group = protocol.network_group(self.destination.host) self.network_group = protocol.network_group(self.destination.host)
ObjectTracker.__init__(self) # pylint: disable=non-parent-init-called ObjectTracker.__init__(self) # pylint: disable=non-parent-init-called
self.bm_proto_reset() self.bm_proto_reset()
@ -142,12 +141,12 @@ class TCPConnection(BMProto, TLSDispatcher):
def set_connection_fully_established(self): def set_connection_fully_established(self):
"""Initiate inventory synchronisation.""" """Initiate inventory synchronisation."""
shared.clientHasReceivedIncomingConnections = True if not self.isOutbound and not self.local:
UISignalQueue.put(('setStatusIcon', 'green')) shared.clientHasReceivedIncomingConnections = True
UISignalQueue.put(( UISignalQueue.put(('setStatusIcon', 'green'))
'updateNetworkStatusTab', UISignalQueue.put(
(self.isOutbound, True, self.destination) ('updateNetworkStatusTab', (
)) self.isOutbound, True, self.destination)))
self.antiIntersectionDelay(True) self.antiIntersectionDelay(True)
self.fullyEstablished = True self.fullyEstablished = True
if self.isOutbound: if self.isOutbound:
@ -221,6 +220,7 @@ class TCPConnection(BMProto, TLSDispatcher):
payload = bytes() payload = bytes()
# Now let us start appending all of these hashes together. They will be # Now let us start appending all of these hashes together. They will be
# sent out in a big inv message to our new peer. # sent out in a big inv message to our new peer.
for obj_hash, _ in bigInvList.items(): for obj_hash, _ in bigInvList.items():
payload += obj_hash payload += obj_hash
objectCount += 1 objectCount += 1

61
src/network/tls.py
View File

@ -9,6 +9,7 @@ import sys
from network.advanceddispatcher import AdvancedDispatcher from network.advanceddispatcher import AdvancedDispatcher
import network.asyncore_pollchoose as asyncore import network.asyncore_pollchoose as asyncore
from queues import receiveDataQueue from queues import receiveDataQueue
import paths import paths
@ -28,7 +29,8 @@ if sys.version_info >= (2, 7, 13):
except AttributeError: except AttributeError:
sslProtocolVersion = ssl.PROTOCOL_SSLv23 sslProtocolVersion = ssl.PROTOCOL_SSLv23
elif sys.version_info >= (2, 7, 9): elif sys.version_info >= (2, 7, 9):
# this means any SSL/TLS. SSLv2 and 3 are excluded with an option after context is created # this means any SSL/TLS.
# SSLv2 and 3 are excluded with an option after context is created
sslProtocolVersion = ssl.PROTOCOL_SSLv23 sslProtocolVersion = ssl.PROTOCOL_SSLv23
else: else:
# this means TLSv1, there is no way to set "TLSv1 or higher" or # this means TLSv1, there is no way to set "TLSv1 or higher" or
@ -37,7 +39,8 @@ else:
# ciphers # ciphers
if ssl.OPENSSL_VERSION_NUMBER >= 0x10100000 and not ssl.OPENSSL_VERSION.startswith("LibreSSL"): if ssl.OPENSSL_VERSION_NUMBER >= 0x10100000 and not \
ssl.OPENSSL_VERSION.startswith("LibreSSL"):
sslProtocolCiphers = "AECDH-AES256-SHA@SECLEVEL=0" sslProtocolCiphers = "AECDH-AES256-SHA@SECLEVEL=0"
else: else:
sslProtocolCiphers = "AECDH-AES256-SHA" sslProtocolCiphers = "AECDH-AES256-SHA"
@ -45,19 +48,19 @@ else:
class TLSDispatcher(AdvancedDispatcher): class TLSDispatcher(AdvancedDispatcher):
"""TLS functionality for classes derived from AdvancedDispatcher""" """TLS functionality for classes derived from AdvancedDispatcher"""
# pylint: disable=too-many-instance-attributes # pylint: disable=too-many-instance-attributes, too-many-arguments
# pylint: disable=too-many-arguments,super-init-not-called,unused-argument # pylint: disable=super-init-not-called
def __init__( def __init__(self, _=None, sock=None, certfile=None, keyfile=None,
self, address=None, sock=None, certfile=None, keyfile=None, server_side=False, ciphers=sslProtocolCiphers):
server_side=False, ciphers=sslProtocolCiphers
):
self.want_read = self.want_write = True self.want_read = self.want_write = True
if certfile is None: if certfile is None:
self.certfile = os.path.join(paths.codePath(), 'sslkeys', 'cert.pem') self.certfile = os.path.join(
paths.codePath(), 'sslkeys', 'cert.pem')
else: else:
self.certfile = certfile self.certfile = certfile
if keyfile is None: if keyfile is None:
self.keyfile = os.path.join(paths.codePath(), 'sslkeys', 'key.pem') self.keyfile = os.path.join(
paths.codePath(), 'sslkeys', 'key.pem')
else: else:
self.keyfile = keyfile self.keyfile = keyfile
self.server_side = server_side self.server_side = server_side
@ -72,20 +75,23 @@ class TLSDispatcher(AdvancedDispatcher):
# pylint: disable=attribute-defined-outside-init # pylint: disable=attribute-defined-outside-init
self.isSSL = True self.isSSL = True
self.tlsStarted = True self.tlsStarted = True
# Once the connection has been established, it's safe to wrap the # Once the connection has been established,
# socket. # it's safe to wrap the socket.
if sys.version_info >= (2, 7, 9): if sys.version_info >= (2, 7, 9):
context = ssl.create_default_context( context = ssl.create_default_context(
purpose=ssl.Purpose.SERVER_AUTH if self.server_side else ssl.Purpose.CLIENT_AUTH) purpose=ssl.Purpose.SERVER_AUTH
if self.server_side else ssl.Purpose.CLIENT_AUTH)
context.set_ciphers(self.ciphers) context.set_ciphers(self.ciphers)
context.set_ecdh_curve("secp256k1") context.set_ecdh_curve("secp256k1")
context.check_hostname = False context.check_hostname = False
context.verify_mode = ssl.CERT_NONE context.verify_mode = ssl.CERT_NONE
# also exclude TLSv1 and TLSv1.1 in the future # also exclude TLSv1 and TLSv1.1 in the future
context.options = ssl.OP_ALL | ssl.OP_NO_SSLv2 |\ context.options = ssl.OP_ALL | ssl.OP_NO_SSLv2 |\
ssl.OP_NO_SSLv3 | ssl.OP_SINGLE_ECDH_USE | ssl.OP_CIPHER_SERVER_PREFERENCE ssl.OP_NO_SSLv3 | ssl.OP_SINGLE_ECDH_USE |\
ssl.OP_CIPHER_SERVER_PREFERENCE
self.sslSocket = context.wrap_socket( self.sslSocket = context.wrap_socket(
self.socket, server_side=self.server_side, do_handshake_on_connect=False) self.socket, server_side=self.server_side,
do_handshake_on_connect=False)
else: else:
self.sslSocket = ssl.wrap_socket( self.sslSocket = ssl.wrap_socket(
self.socket, server_side=self.server_side, self.socket, server_side=self.server_side,
@ -119,12 +125,15 @@ class TLSDispatcher(AdvancedDispatcher):
def readable(self): def readable(self):
"""Handle readable check for TLS-enabled sockets""" """Handle readable check for TLS-enabled sockets"""
try: try:
# during TLS handshake, and after flushing write buffer, return status of last handshake attempt # during TLS handshake, and after flushing write buffer,
# return status of last handshake attempt
if self.tlsStarted and not self.tlsDone and not self.write_buf: if self.tlsStarted and not self.tlsDone and not self.write_buf:
# print "tls readable, %r" % (self.want_read) # print "tls readable, %r" % (self.want_read)
return self.want_read return self.want_read
# prior to TLS handshake, receiveDataThread should emulate synchronous behaviour # prior to TLS handshake,
elif not self.fullyEstablished and (self.expectBytes == 0 or not self.write_buf_empty()): # receiveDataThread should emulate synchronous behaviour
elif not self.fullyEstablished and (
self.expectBytes == 0 or not self.write_buf_empty()):
return False return False
return AdvancedDispatcher.readable(self) return AdvancedDispatcher.readable(self)
except AttributeError: except AttributeError:
@ -139,10 +148,14 @@ class TLSDispatcher(AdvancedDispatcher):
try: try:
# wait for write buffer flush # wait for write buffer flush
if self.tlsStarted and not self.tlsDone and not self.write_buf: if self.tlsStarted and not self.tlsDone and not self.write_buf:
# logger.debug("%s:%i TLS handshaking (read)", self.destination.host, self.destination.port) # logger.debug(
# "%s:%i TLS handshaking (read)", self.destination.host,
# self.destination.port)
self.tls_handshake() self.tls_handshake()
else: else:
# logger.debug("%s:%i Not TLS handshaking (read)", self.destination.host, self.destination.port) # logger.debug(
# "%s:%i Not TLS handshaking (read)", self.destination.host,
# self.destination.port)
return AdvancedDispatcher.handle_read(self) return AdvancedDispatcher.handle_read(self)
except AttributeError: except AttributeError:
return AdvancedDispatcher.handle_read(self) return AdvancedDispatcher.handle_read(self)
@ -165,10 +178,14 @@ class TLSDispatcher(AdvancedDispatcher):
try: try:
# wait for write buffer flush # wait for write buffer flush
if self.tlsStarted and not self.tlsDone and not self.write_buf: if self.tlsStarted and not self.tlsDone and not self.write_buf:
# logger.debug("%s:%i TLS handshaking (write)", self.destination.host, self.destination.port) # logger.debug(
# "%s:%i TLS handshaking (write)", self.destination.host,
# self.destination.port)
self.tls_handshake() self.tls_handshake()
else: else:
# logger.debug("%s:%i Not TLS handshaking (write)", self.destination.host, self.destination.port) # logger.debug(
# "%s:%i Not TLS handshaking (write)", self.destination.host,
# self.destination.port)
return AdvancedDispatcher.handle_write(self) return AdvancedDispatcher.handle_write(self)
except AttributeError: except AttributeError:
return AdvancedDispatcher.handle_write(self) return AdvancedDispatcher.handle_write(self)

18
src/network/udp.py
View File

@ -1,16 +1,16 @@
""" """
src/network/udp.py UDP protocol handler
==================
""" """
import logging import logging
import time
import socket import socket
import time
import state
import protocol import protocol
from network.bmproto import BMProto from network.bmproto import BMProto
from network.objectracker import ObjectTracker from network.objectracker import ObjectTracker
from .node import Peer from .node import Peer
import state
from queues import receiveDataQueue from queues import receiveDataQueue
logger = logging.getLogger('default') logger = logging.getLogger('default')
@ -76,10 +76,9 @@ class UDPSocket(BMProto): # pylint: disable=too-many-instance-attributes
# attacks from random IPs on the internet # attacks from random IPs on the internet
self.local = True self.local = True
remoteport = False remoteport = False
for seenTime, stream, services, ip, port in addresses:
# decodedIP = bool(protocol.checkIPAddress(ip))
decodedIP = False
for seenTime, stream, _, ip, port in addresses:
decodedIP = protocol.checkIPAddress(bytes(ip))
if stream not in state.streamsInWhichIAmParticipating: if stream not in state.streamsInWhichIAmParticipating:
continue continue
if (seenTime < time.time() - self.maxTimeOffset if (seenTime < time.time() - self.maxTimeOffset
@ -95,9 +94,8 @@ class UDPSocket(BMProto): # pylint: disable=too-many-instance-attributes
"received peer discovery from {}:{} (port {}):", "received peer discovery from {}:{} (port {}):",
self.destination.host, self.destination.port, remoteport) self.destination.host, self.destination.port, remoteport)
if self.local: if self.local:
state.discoveredPeers[ state.discoveredPeers[Peer(self.destination.host, remoteport)] = \
Peer(self.destination.host, remoteport) time.time()
] = time.time()
return True return True
def bm_command_portcheck(self): def bm_command_portcheck(self):

5
src/plugins/indicator_libmessaging.py
View File

@ -1,7 +1,6 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
""" """
src/plugins/indicator_libmessaging.py Indicator plugin using libmessaging
=====================================
""" """
import gi import gi
@ -37,7 +36,7 @@ class IndicatorLibmessaging(object):
if self.app: if self.app:
self.app.unregister() self.app.unregister()
def activate(self, app, source): # pylint: disable=unused-argument def activate(self, app, source): # pylint: disable=unused-argument
"""Activate the libmessaging indicator plugin""" """Activate the libmessaging indicator plugin"""
self.form.appIndicatorInbox( self.form.appIndicatorInbox(
self.new_message_item if source == 'messages' self.new_message_item if source == 'messages'

8
src/plugins/menu_qrcode.py
View File

@ -1,8 +1,5 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
""" """
src/plugins/menu_qrcode.py
==========================
A menu plugin showing QR-Code for bitmessage address in modal dialog. A menu plugin showing QR-Code for bitmessage address in modal dialog.
""" """
@ -15,10 +12,11 @@ from pybitmessage.tr import _translate
# http://stackoverflow.com/questions/20452486 # http://stackoverflow.com/questions/20452486
class Image(qrcode.image.base.BaseImage): # pylint: disable=abstract-method class Image(qrcode.image.base.BaseImage): # pylint: disable=abstract-method
"""Image output class for qrcode using QPainter""" """Image output class for qrcode using QPainter"""
def __init__(self, border, width, box_size): # pylint: disable=super-init-not-called def __init__(self, border, width, box_size):
# pylint: disable=super-init-not-called
self.border = border self.border = border
self.width = width self.width = width
self.box_size = box_size self.box_size = box_size

5
src/plugins/notification_notify2.py
View File

@ -1,7 +1,6 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
""" """
src/plugins/notification_notify2.py Notification plugin using notify2
===================================
""" """
import gi import gi
@ -11,7 +10,7 @@ from gi.repository import Notify
Notify.init('pybitmessage') Notify.init('pybitmessage')
def connect_plugin(title, subtitle, category, label, icon): def connect_plugin(title, subtitle, category, _, icon):
"""Plugin for notify2""" """Plugin for notify2"""
if not icon: if not icon:
icon = 'mail-message-new' if category == 2 else 'pybitmessage' icon = 'mail-message-new' if category == 2 else 'pybitmessage'

9
src/plugins/proxyconfig_stem.py
View File

@ -22,9 +22,8 @@ import stem.process
import stem.version import stem.version
class DebugLogger(object): class DebugLogger(object): # pylint: disable=too-few-public-methods
"""Safe logger wrapper for tor and plugin's logs""" """Safe logger wrapper for tor and plugin's logs"""
# pylint: disable=too-few-public-methods
def __init__(self): def __init__(self):
self._logger = logging.getLogger('default') self._logger = logging.getLogger('default')
self._levels = { self._levels = {
@ -108,7 +107,8 @@ def connect_plugin(config): # pylint: disable=too-many-branches
onionhostname = config.safeGet('bitmessagesettings', 'onionhostname') onionhostname = config.safeGet('bitmessagesettings', 'onionhostname')
onionkey = config.safeGet(onionhostname, 'privsigningkey') onionkey = config.safeGet(onionhostname, 'privsigningkey')
if onionhostname and not onionkey: if onionhostname and not onionkey:
logwrite('The hidden service found in config ): %s' % onionhostname) logwrite('The hidden service found in config ): %s' %
onionhostname)
onionkeytype = config.safeGet(onionhostname, 'keytype') onionkeytype = config.safeGet(onionhostname, 'keytype')
response = controller.create_ephemeral_hidden_service( response = controller.create_ephemeral_hidden_service(
@ -124,7 +124,8 @@ def connect_plugin(config): # pylint: disable=too-many-branches
if not onionkey: if not onionkey:
logwrite('Started hidden service %s.onion' % response.service_id) logwrite('Started hidden service %s.onion' % response.service_id)
# only save new service keys if onionhostname was not set previously # only save new service keys
# if onionhostname was not set previously
if not onionhostname: if not onionhostname:
onionhostname = response.service_id + '.onion' onionhostname = response.service_id + '.onion'
config.set( config.set(

5
src/plugins/sound_canberra.py
View File

@ -1,7 +1,6 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
""" """
src/plugins/proxyconfig_stem.py Sound theme plugin using pycanberra
===================================
""" """
from pybitmessage.bitmessageqt import sound from pybitmessage.bitmessageqt import sound
@ -18,7 +17,7 @@ _theme = {
} }
def connect_plugin(category, label=None): # pylint: disable=unused-argument def connect_plugin(category, label=None): # pylint: disable=unused-argument
"""This function implements the entry point.""" """This function implements the entry point."""
try: try:
_canberra.play(0, pycanberra.CA_PROP_EVENT_ID, _theme[category], None) _canberra.play(0, pycanberra.CA_PROP_EVENT_ID, _theme[category], None)

3
src/plugins/sound_gstreamer.py
View File

@ -1,7 +1,6 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
""" """
src/plugins/sound_gstreamer.py Sound notification plugin using gstreamer
===================================
""" """
import gi import gi
gi.require_version('Gst', '1.0') gi.require_version('Gst', '1.0')

5
src/plugins/sound_playfile.py
View File

@ -1,7 +1,6 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
""" """
src/plugins/sound_playfile.py Sound notification plugin using external executable or winsound (on Windows)
===================================
""" """
try: try:
@ -23,7 +22,7 @@ except ImportError:
def connect_plugin(sound_file): def connect_plugin(sound_file):
"""This function implements the entry point.""" """This function implements the entry point."""
global play_cmd # pylint: disable=global-statement global play_cmd # pylint: disable=global-statement
ext = os.path.splitext(sound_file)[-1] ext = os.path.splitext(sound_file)[-1]
try: try:

3
src/proofofwork.py
View File

@ -1,7 +1,6 @@
# pylint: disable=too-many-branches,too-many-statements,protected-access # pylint: disable=too-many-branches,too-many-statements,protected-access
""" """
src/proofofwork.py Proof of work calculation
==================
""" """
import ctypes import ctypes

28
src/pyelliptic/arithmetic.py
View File

@ -1,5 +1,6 @@
# pylint: disable=missing-docstring,too-many-function-args """
Arithmetic Expressions
"""
import hashlib import hashlib
import re import re
@ -11,6 +12,7 @@ G = (Gx, Gy)
def inv(a, n): def inv(a, n):
"""Inversion"""
lm, hm = 1, 0 lm, hm = 1, 0
low, high = a % n, n low, high = a % n, n
while low > 1: while low > 1:
@ -21,6 +23,7 @@ def inv(a, n):
def get_code_string(base): def get_code_string(base):
"""Returns string according to base value"""
if base == 2: if base == 2:
return '01' return '01'
elif base == 10: elif base == 10:
@ -38,6 +41,7 @@ def get_code_string(base):
def encode(val, base, minlen=0): def encode(val, base, minlen=0):
"""Returns the encoded string"""
code_string = get_code_string(base) code_string = get_code_string(base)
result = str.encode('') if type(code_string) is bytes else '' result = str.encode('') if type(code_string) is bytes else ''
while val > 0: while val > 0:
@ -48,6 +52,7 @@ def encode(val, base, minlen=0):
return result return result
def decode(string, base): def decode(string, base):
"""Returns the decoded string"""
code_string = get_code_string(base) code_string = get_code_string(base)
result = 0 result = 0
if base == 16: if base == 16:
@ -60,10 +65,13 @@ def decode(string, base):
def changebase(string, frm, to, minlen=0): def changebase(string, frm, to, minlen=0):
"""Change base of the string"""
return encode(decode(string, frm), to, minlen) return encode(decode(string, frm), to, minlen)
def base10_add(a, b): def base10_add(a, b):
"""Adding the numbers that are of base10"""
# pylint: disable=too-many-function-args
if a is None: if a is None:
return b[0], b[1] return b[0], b[1]
if b is None: if b is None:
@ -79,6 +87,7 @@ def base10_add(a, b):
def base10_double(a): def base10_double(a):
"""Double the numbers that are of base10"""
if a is None: if a is None:
return None return None
m = ((3 * a[0] * a[0] + A) * inv(2 * a[1], P)) % P m = ((3 * a[0] * a[0] + A) * inv(2 * a[1], P)) % P
@ -88,6 +97,7 @@ def base10_double(a):
def base10_multiply(a, n): def base10_multiply(a, n):
"""Multiply the numbers that are of base10"""
if n == 0: if n == 0:
return G return G
if n == 1: if n == 1:
@ -100,28 +110,35 @@ def base10_multiply(a, n):
def hex_to_point(h): def hex_to_point(h):
"""Converting hexadecimal to point value"""
return (decode(h[2:66], 16), decode(h[66:], 16)) return (decode(h[2:66], 16), decode(h[66:], 16))
def point_to_hex(p): def point_to_hex(p):
"""Converting point value to hexadecimal"""
return '04' + encode(p[0], 16, 64) + encode(p[1], 16, 64) return '04' + encode(p[0], 16, 64) + encode(p[1], 16, 64)
def multiply(privkey, pubkey): def multiply(privkey, pubkey):
return point_to_hex(base10_multiply(hex_to_point(pubkey), decode(privkey, 16))) """Multiplying keys"""
return point_to_hex(base10_multiply(
hex_to_point(pubkey), decode(privkey, 16)))
def privtopub(privkey): def privtopub(privkey):
"""Converting key from private to public"""
return point_to_hex(base10_multiply(G, decode(privkey, 16))) return point_to_hex(base10_multiply(G, decode(privkey, 16)))
def add(p1, p2): def add(p1, p2):
"""Adding two public keys"""
if len(p1) == 32: if len(p1) == 32:
return encode(decode(p1, 16) + decode(p2, 16) % P, 16, 32) return encode(decode(p1, 16) + decode(p2, 16) % P, 16, 32)
return point_to_hex(base10_add(hex_to_point(p1), hex_to_point(p2))) return point_to_hex(base10_add(hex_to_point(p1), hex_to_point(p2)))
def hash_160(string): def hash_160(string):
"""Hashed version of public key"""
intermed = hashlib.sha256(string).digest() intermed = hashlib.sha256(string).digest()
ripemd160 = hashlib.new('ripemd160') ripemd160 = hashlib.new('ripemd160')
ripemd160.update(intermed) ripemd160.update(intermed)
@ -129,17 +146,18 @@ def hash_160(string):
def dbl_sha256(string): def dbl_sha256(string):
"""Double hashing (SHA256)"""
return hashlib.sha256(hashlib.sha256(string).digest()).digest() return hashlib.sha256(hashlib.sha256(string).digest()).digest()
def bin_to_b58check(inp): def bin_to_b58check(inp):
"""Convert binary to base58"""
inp_fmtd = '\x00' + inp inp_fmtd = '\x00' + inp
leadingzbytes = len(re.match('^\x00*', inp_fmtd).group(0)) leadingzbytes = len(re.match('^\x00*', inp_fmtd).group(0))
checksum = dbl_sha256(inp_fmtd)[:4] checksum = dbl_sha256(inp_fmtd)[:4]
return '1' * leadingzbytes + changebase(inp_fmtd + checksum, 256, 58) return '1' * leadingzbytes + changebase(inp_fmtd + checksum, 256, 58)
# Convert a public key (in hex) to a Bitcoin address
def pubkey_to_address(pubkey): def pubkey_to_address(pubkey):
"""Convert a public key (in hex) to a Bitcoin address"""
return bin_to_b58check(hash_160(changebase(pubkey, 16, 256))) return bin_to_b58check(hash_160(changebase(pubkey, 16, 256)))

10
src/pyelliptic/cipher.py
View File

@ -1,10 +1,8 @@
#!/usr/bin/env python #!/usr/bin/env python
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
""" """
src/pyelliptic/cipher.py Symmetric Encryption
========================
""" """
# Copyright (C) 2011 Yann GUIBET <yannguibet@gmail.com> # Copyright (C) 2011 Yann GUIBET <yannguibet@gmail.com>
# See LICENSE for details. # See LICENSE for details.
@ -14,7 +12,7 @@ from .openssl import OpenSSL
# pylint: disable=redefined-builtin # pylint: disable=redefined-builtin
class Cipher(object): class Cipher(object):
""" """
Symmetric encryption Main class for encryption
import pyelliptic import pyelliptic
iv = pyelliptic.Cipher.gen_IV('aes-256-cfb') iv = pyelliptic.Cipher.gen_IV('aes-256-cfb')
@ -67,7 +65,7 @@ class Cipher(object):
if OpenSSL.EVP_CipherUpdate(self.ctx, OpenSSL.byref(buffer), if OpenSSL.EVP_CipherUpdate(self.ctx, OpenSSL.byref(buffer),
OpenSSL.byref(i), inp, len(input)) == 0: OpenSSL.byref(i), inp, len(input)) == 0:
raise Exception("[OpenSSL] EVP_CipherUpdate FAIL ...") raise Exception("[OpenSSL] EVP_CipherUpdate FAIL ...")
return buffer.raw[0:i.value] # pylint: disable=invalid-slice-index return buffer.raw[0:i.value] # pylint: disable=invalid-slice-index
def final(self): def final(self):
"""Returning the final value""" """Returning the final value"""
@ -76,7 +74,7 @@ class Cipher(object):
if (OpenSSL.EVP_CipherFinal_ex(self.ctx, OpenSSL.byref(buffer), if (OpenSSL.EVP_CipherFinal_ex(self.ctx, OpenSSL.byref(buffer),
OpenSSL.byref(i))) == 0: OpenSSL.byref(i))) == 0:
raise Exception("[OpenSSL] EVP_CipherFinal_ex FAIL ...") raise Exception("[OpenSSL] EVP_CipherFinal_ex FAIL ...")
return buffer.raw[0:i.value] # pylint: disable=invalid-slice-index return buffer.raw[0:i.value] # pylint: disable=invalid-slice-index
def ciphering(self, input): def ciphering(self, input):
""" """

22
src/pyelliptic/ecc.py
View File

@ -1,11 +1,9 @@
#!/usr/bin/env python #!/usr/bin/env python
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
""" """
src/pyelliptic/ecc.py Asymmetric cryptography using elliptic curves
=====================
""" """
# pylint: disable=protected-access # pylint: disable=protected-access, too-many-branches, too-many-locals
# Copyright (C) 2011 Yann GUIBET <yannguibet@gmail.com> # Copyright (C) 2011 Yann GUIBET <yannguibet@gmail.com>
# See LICENSE for details. # See LICENSE for details.
@ -172,7 +170,8 @@ class ECC(object):
if OpenSSL.EC_POINT_get_affine_coordinates_GFp( if OpenSSL.EC_POINT_get_affine_coordinates_GFp(
group, pub_key, pub_key_x, pub_key_y, 0) == 0: group, pub_key, pub_key_x, pub_key_y, 0) == 0:
raise Exception("[OpenSSL] EC_POINT_get_affine_coordinates_GFp FAIL ...") raise Exception(
"[OpenSSL] EC_POINT_get_affine_coordinates_GFp FAIL ...")
privkey = OpenSSL.malloc(0, OpenSSL.BN_num_bytes(priv_key)) privkey = OpenSSL.malloc(0, OpenSSL.BN_num_bytes(priv_key))
pubkeyx = OpenSSL.malloc(0, OpenSSL.BN_num_bytes(pub_key_x)) pubkeyx = OpenSSL.malloc(0, OpenSSL.BN_num_bytes(pub_key_x))
@ -275,7 +274,6 @@ class ECC(object):
def raw_check_key(self, privkey, pubkey_x, pubkey_y, curve=None): def raw_check_key(self, privkey, pubkey_x, pubkey_y, curve=None):
"""Check key validity, key is supplied as binary data""" """Check key validity, key is supplied as binary data"""
# pylint: disable=too-many-branches
if curve is None: if curve is None:
curve = self.curve curve = self.curve
elif isinstance(curve, str): elif isinstance(curve, str):
@ -322,7 +320,6 @@ class ECC(object):
""" """
Sign the input with ECDSA method and returns the signature Sign the input with ECDSA method and returns the signature
""" """
# pylint: disable=too-many-branches,too-many-locals
try: try:
size = len(inputb) size = len(inputb)
buff = OpenSSL.malloc(inputb, size) buff = OpenSSL.malloc(inputb, size)
@ -392,7 +389,6 @@ class ECC(object):
Verify the signature with the input and the local public key. Verify the signature with the input and the local public key.
Returns a boolean Returns a boolean
""" """
# pylint: disable=too-many-branches
try: try:
bsig = OpenSSL.malloc(sig, len(sig)) bsig = OpenSSL.malloc(sig, len(sig))
binputb = OpenSSL.malloc(inputb, len(inputb)) binputb = OpenSSL.malloc(inputb, len(inputb))
@ -435,10 +431,13 @@ class ECC(object):
0, digest, dgst_len.contents, bsig, len(sig), key) 0, digest, dgst_len.contents, bsig, len(sig), key)
if ret == -1: if ret == -1:
return False # Fail to Check # Fail to Check
return False
if ret == 0: if ret == 0:
return False # Bad signature ! # Bad signature !
return True # Good return False
# Good
return True
finally: finally:
OpenSSL.EC_KEY_free(key) OpenSSL.EC_KEY_free(key)
@ -486,7 +485,6 @@ class ECC(object):
""" """
Decrypt data with ECIES method using the local private key Decrypt data with ECIES method using the local private key
""" """
# pylint: disable=too-many-locals
blocksize = OpenSSL.get_cipher(ciphername).get_blocksize() blocksize = OpenSSL.get_cipher(ciphername).get_blocksize()
iv = data[:blocksize] iv = data[:blocksize]
i = blocksize i = blocksize

10
src/pyelliptic/eccblind.py
View File

@ -1,6 +1,7 @@
#!/usr/bin/env python #!/usr/bin/env python
""" """
ECC blind signature functionality based on "An Efficient Blind Signature Scheme ECC blind signature functionality based on
"An Efficient Blind Signature Scheme
Based on the Elliptic CurveDiscrete Logarithm Problem" by Morteza Nikooghadama Based on the Elliptic CurveDiscrete Logarithm Problem" by Morteza Nikooghadama
<mnikooghadam@sbu.ac.ir> and Ali Zakerolhosseini <a-zaker@sbu.ac.ir>, <mnikooghadam@sbu.ac.ir> and Ali Zakerolhosseini <a-zaker@sbu.ac.ir>,
http://www.isecure-journal.com/article_39171_47f9ec605dd3918c2793565ec21fcd7a.pdf http://www.isecure-journal.com/article_39171_47f9ec605dd3918c2793565ec21fcd7a.pdf
@ -8,7 +9,6 @@ http://www.isecure-journal.com/article_39171_47f9ec605dd3918c2793565ec21fcd7a.pd
# variable names are based on the math in the paper, so they don't conform # variable names are based on the math in the paper, so they don't conform
# to PEP8 # to PEP8
# pylint: disable=invalid-name
from .openssl import OpenSSL from .openssl import OpenSSL
@ -72,8 +72,7 @@ class ECCBlind(object): # pylint: disable=too-many-instance-attributes
# F = (x0, y0) # F = (x0, y0)
x0 = OpenSSL.BN_new() x0 = OpenSSL.BN_new()
y0 = OpenSSL.BN_new() y0 = OpenSSL.BN_new()
OpenSSL.EC_POINT_get_affine_coordinates_GFp(group, F, x0, y0, OpenSSL.EC_POINT_get_affine_coordinates_GFp(group, F, x0, y0, ctx)
ctx)
return x0 return x0
def __init__(self, curve="secp256k1", pubkey=None): def __init__(self, curve="secp256k1", pubkey=None):
@ -82,7 +81,8 @@ class ECCBlind(object): # pylint: disable=too-many-instance-attributes
if pubkey: if pubkey:
self.group, self.G, self.n, self.Q = pubkey self.group, self.G, self.n, self.Q = pubkey
else: else:
self.group = OpenSSL.EC_GROUP_new_by_curve_name(OpenSSL.get_curve(curve)) self.group = OpenSSL.EC_GROUP_new_by_curve_name(
OpenSSL.get_curve(curve))
# Order n # Order n
self.n = OpenSSL.BN_new() self.n = OpenSSL.BN_new()
OpenSSL.EC_GROUP_get_order(self.group, self.n, self.ctx) OpenSSL.EC_GROUP_get_order(self.group, self.n, self.ctx)

125
src/pyelliptic/openssl.py
View File

@ -2,12 +2,11 @@
# See LICENSE for details. # See LICENSE for details.
# #
# Software slightly changed by Jonathan Warren <bitmessage at-symbol jonwarren.org> # Software slightly changed by Jonathan Warren <bitmessage at-symbol jonwarren.org>
# pylint: disable=protected-access, import-error
""" """
This module loads openssl libs with ctypes and incapsulates This module loads openssl libs with ctypes and incapsulates
needed openssl functionality in class _OpenSSL. needed openssl functionality in class _OpenSSL.
""" """
# pylint: disable=protected-access
import sys import sys
import ctypes import ctypes
from kivy.utils import platform from kivy.utils import platform
@ -15,10 +14,9 @@ from kivy.utils import platform
OpenSSL = None OpenSSL = None
class CipherName: class CipherName(object):
"""Class returns cipher name, pointer and blocksize""" """Class returns cipher name, pointer and blocksize"""
# pylint: disable=old-style-class
def __init__(self, name, pointer, blocksize): def __init__(self, name, pointer, blocksize):
self._name = name self._name = name
self._pointer = pointer self._pointer = pointer
@ -74,11 +72,11 @@ def get_version(library):
return (version, hexversion, cflags) return (version, hexversion, cflags)
class _OpenSSL: class _OpenSSL(object):
""" """
Wrapper for OpenSSL using ctypes Wrapper for OpenSSL using ctypes
""" """
# pylint: disable=too-many-statements, too-many-instance-attributes, old-style-class # pylint: disable=too-many-statements, too-many-instance-attributes
def __init__(self, library): def __init__(self, library):
"""Build the wrapper""" """Build the wrapper"""
self._lib = ctypes.CDLL(library) self._lib = ctypes.CDLL(library)
@ -139,7 +137,8 @@ class _OpenSSL:
self.EC_KEY_get0_group.restype = ctypes.c_void_p self.EC_KEY_get0_group.restype = ctypes.c_void_p
self.EC_KEY_get0_group.argtypes = [ctypes.c_void_p] self.EC_KEY_get0_group.argtypes = [ctypes.c_void_p]
self.EC_POINT_get_affine_coordinates_GFp = self._lib.EC_POINT_get_affine_coordinates_GFp self.EC_POINT_get_affine_coordinates_GFp = \
self._lib.EC_POINT_get_affine_coordinates_GFp
self.EC_POINT_get_affine_coordinates_GFp.restype = ctypes.c_int self.EC_POINT_get_affine_coordinates_GFp.restype = ctypes.c_int
self.EC_POINT_get_affine_coordinates_GFp.argtypes = [ctypes.c_void_p, self.EC_POINT_get_affine_coordinates_GFp.argtypes = [ctypes.c_void_p,
ctypes.c_void_p, ctypes.c_void_p,
@ -162,7 +161,8 @@ class _OpenSSL:
self.EC_KEY_set_group.argtypes = [ctypes.c_void_p, self.EC_KEY_set_group.argtypes = [ctypes.c_void_p,
ctypes.c_void_p] ctypes.c_void_p]
self.EC_POINT_set_affine_coordinates_GFp = self._lib.EC_POINT_set_affine_coordinates_GFp self.EC_POINT_set_affine_coordinates_GFp = \
self._lib.EC_POINT_set_affine_coordinates_GFp
self.EC_POINT_set_affine_coordinates_GFp.restype = ctypes.c_int self.EC_POINT_set_affine_coordinates_GFp.restype = ctypes.c_int
self.EC_POINT_set_affine_coordinates_GFp.argtypes = [ctypes.c_void_p, self.EC_POINT_set_affine_coordinates_GFp.argtypes = [ctypes.c_void_p,
ctypes.c_void_p, ctypes.c_void_p,
@ -296,7 +296,8 @@ class _OpenSSL:
self.EVP_CipherUpdate = self._lib.EVP_CipherUpdate self.EVP_CipherUpdate = self._lib.EVP_CipherUpdate
self.EVP_CipherUpdate.restype = ctypes.c_int self.EVP_CipherUpdate.restype = ctypes.c_int
self.EVP_CipherUpdate.argtypes = [ctypes.c_void_p, self.EVP_CipherUpdate.argtypes = [ctypes.c_void_p,
ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int] ctypes.c_void_p, ctypes.c_void_p,
ctypes.c_void_p, ctypes.c_int]
self.EVP_CipherFinal_ex = self._lib.EVP_CipherFinal_ex self.EVP_CipherFinal_ex = self._lib.EVP_CipherFinal_ex
self.EVP_CipherFinal_ex.restype = ctypes.c_int self.EVP_CipherFinal_ex.restype = ctypes.c_int
@ -329,12 +330,14 @@ class _OpenSSL:
self.ECDSA_sign = self._lib.ECDSA_sign self.ECDSA_sign = self._lib.ECDSA_sign
self.ECDSA_sign.restype = ctypes.c_int self.ECDSA_sign.restype = ctypes.c_int
self.ECDSA_sign.argtypes = [ctypes.c_int, ctypes.c_void_p, self.ECDSA_sign.argtypes = [ctypes.c_int, ctypes.c_void_p,
ctypes.c_int, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p] ctypes.c_int, ctypes.c_void_p,
ctypes.c_void_p, ctypes.c_void_p]
self.ECDSA_verify = self._lib.ECDSA_verify self.ECDSA_verify = self._lib.ECDSA_verify
self.ECDSA_verify.restype = ctypes.c_int self.ECDSA_verify.restype = ctypes.c_int
self.ECDSA_verify.argtypes = [ctypes.c_int, ctypes.c_void_p, self.ECDSA_verify.argtypes = [ctypes.c_int, ctypes.c_void_p,
ctypes.c_int, ctypes.c_void_p, ctypes.c_int, ctypes.c_void_p] ctypes.c_int, ctypes.c_void_p,
ctypes.c_int, ctypes.c_void_p]
if self._hexversion >= 0x10100000 and not self._libreSSL: if self._hexversion >= 0x10100000 and not self._libreSSL:
self.EVP_MD_CTX_new = self._lib.EVP_MD_CTX_new self.EVP_MD_CTX_new = self._lib.EVP_MD_CTX_new
@ -392,7 +395,8 @@ class _OpenSSL:
self.HMAC = self._lib.HMAC self.HMAC = self._lib.HMAC
self.HMAC.restype = ctypes.c_void_p self.HMAC.restype = ctypes.c_void_p
self.HMAC.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int, self.HMAC.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int,
ctypes.c_void_p, ctypes.c_int, ctypes.c_void_p, ctypes.c_void_p] ctypes.c_void_p, ctypes.c_int,
ctypes.c_void_p, ctypes.c_void_p]
try: try:
self.PKCS5_PBKDF2_HMAC = self._lib.PKCS5_PBKDF2_HMAC self.PKCS5_PBKDF2_HMAC = self._lib.PKCS5_PBKDF2_HMAC
@ -529,17 +533,29 @@ class _OpenSSL:
def _set_ciphers(self): def _set_ciphers(self):
self.cipher_algo = { self.cipher_algo = {
'aes-128-cbc': CipherName('aes-128-cbc', self.EVP_aes_128_cbc, 16), 'aes-128-cbc': CipherName(
'aes-256-cbc': CipherName('aes-256-cbc', self.EVP_aes_256_cbc, 16), 'aes-128-cbc', self.EVP_aes_128_cbc, 16),
'aes-128-cfb': CipherName('aes-128-cfb', self.EVP_aes_128_cfb128, 16), 'aes-256-cbc': CipherName(
'aes-256-cfb': CipherName('aes-256-cfb', self.EVP_aes_256_cfb128, 16), 'aes-256-cbc', self.EVP_aes_256_cbc, 16),
'aes-128-ofb': CipherName('aes-128-ofb', self._lib.EVP_aes_128_ofb, 16), 'aes-128-cfb': CipherName(
'aes-256-ofb': CipherName('aes-256-ofb', self._lib.EVP_aes_256_ofb, 16), 'aes-128-cfb', self.EVP_aes_128_cfb128, 16),
# 'aes-128-ctr': CipherName('aes-128-ctr', self._lib.EVP_aes_128_ctr, 16), 'aes-256-cfb': CipherName(
# 'aes-256-ctr': CipherName('aes-256-ctr', self._lib.EVP_aes_256_ctr, 16), 'aes-256-cfb', self.EVP_aes_256_cfb128, 16),
'bf-cfb': CipherName('bf-cfb', self.EVP_bf_cfb64, 8), 'aes-128-ofb': CipherName(
'bf-cbc': CipherName('bf-cbc', self.EVP_bf_cbc, 8), 'aes-128-ofb', self._lib.EVP_aes_128_ofb, 16),
'rc4': CipherName('rc4', self.EVP_rc4, 128), # 128 is the initialisation size not block size 'aes-256-ofb': CipherName(
'aes-256-ofb', self._lib.EVP_aes_256_ofb, 16),
# 'aes-128-ctr': CipherName(
# 'aes-128-ctr', self._lib.EVP_aes_128_ctr, 16),
# 'aes-256-ctr': CipherName(
# 'aes-256-ctr', self._lib.EVP_aes_256_ctr, 16),
'bf-cfb': CipherName(
'bf-cfb', self.EVP_bf_cfb64, 8),
'bf-cbc': CipherName(
'bf-cbc', self.EVP_bf_cbc, 8),
# 128 is the initialisation size not block size
'rc4': CipherName(
'rc4', self.EVP_rc4, 128),
} }
def _set_curves(self): def _set_curves(self):
@ -599,14 +615,13 @@ class _OpenSSL:
raise Exception("Unknown curve") raise Exception("Unknown curve")
return self.curves[name] return self.curves[name]
def get_curve_by_id(self, id): def get_curve_by_id(self, id_):
""" """
returns the name of a elliptic curve with his id returns the name of a elliptic curve with his id
""" """
# pylint: disable=redefined-builtin
res = None res = None
for i in self.curves: for i in self.curves:
if self.curves[i] == id: if self.curves[i] == id_:
res = i res = i
break break
if res is None: if res is None:
@ -617,32 +632,31 @@ class _OpenSSL:
""" """
OpenSSL random function OpenSSL random function
""" """
# pylint: disable=redefined-builtin buffer_ = self.malloc(0, size)
buffer = self.malloc(0, size) # This pyelliptic library, by default, didn't check the return value
# This pyelliptic library, by default, didn't check the return value of RAND_bytes. It is # of RAND_bytes. It is evidently possible that it returned an error
# evidently possible that it returned an error and not-actually-random data. However, in # and not-actually-random data. However, in tests on various
# tests on various operating systems, while generating hundreds of gigabytes of random # operating systems, while generating hundreds of gigabytes of random
# strings of various sizes I could not get an error to occur. Also Bitcoin doesn't check # strings of various sizes I could not get an error to occur.
# the return value of RAND_bytes either. # Also Bitcoin doesn't check the return value of RAND_bytes either.
# Fixed in Bitmessage version 0.4.2 (in source code on 2013-10-13) # Fixed in Bitmessage version 0.4.2 (in source code on 2013-10-13)
while self.RAND_bytes(buffer, size) != 1: while self.RAND_bytes(buffer_, size) != 1:
import time import time
time.sleep(1) time.sleep(1)
return buffer.raw return buffer_.raw
def malloc(self, data, size): def malloc(self, data, size):
""" """
returns a create_string_buffer (ctypes) returns a create_string_buffer (ctypes)
""" """
# pylint: disable=redefined-builtin buffer_ = None
buffer = None
if data != 0: if data != 0:
if sys.version_info.major == 3 and isinstance(data, type('')): if sys.version_info.major == 3 and isinstance(data, type('')):
data = data.encode() data = data.encode()
buffer = self.create_string_buffer(data, size) buffer_ = self.create_string_buffer(data, size)
else: else:
buffer = self.create_string_buffer(size) buffer_ = self.create_string_buffer(size)
return buffer return buffer_
def loadOpenSSL(): def loadOpenSSL():
@ -657,12 +671,24 @@ def loadOpenSSL():
if getattr(sys, 'frozen', None): if getattr(sys, 'frozen', None):
if 'darwin' in sys.platform: if 'darwin' in sys.platform:
libdir.extend([ libdir.extend([
path.join(environ['RESOURCEPATH'], '..', 'Frameworks', 'libcrypto.dylib'), path.join(
path.join(environ['RESOURCEPATH'], '..', 'Frameworks', 'libcrypto.1.1.0.dylib'), environ['RESOURCEPATH'], '..',
path.join(environ['RESOURCEPATH'], '..', 'Frameworks', 'libcrypto.1.0.2.dylib'), 'Frameworks', 'libcrypto.dylib'),
path.join(environ['RESOURCEPATH'], '..', 'Frameworks', 'libcrypto.1.0.1.dylib'), path.join(
path.join(environ['RESOURCEPATH'], '..', 'Frameworks', 'libcrypto.1.0.0.dylib'), environ['RESOURCEPATH'], '..',
path.join(environ['RESOURCEPATH'], '..', 'Frameworks', 'libcrypto.0.9.8.dylib'), 'Frameworks', 'libcrypto.1.1.0.dylib'),
path.join(
environ['RESOURCEPATH'], '..',
'Frameworks', 'libcrypto.1.0.2.dylib'),
path.join(
environ['RESOURCEPATH'], '..',
'Frameworks', 'libcrypto.1.0.1.dylib'),
path.join(
environ['RESOURCEPATH'], '..',
'Frameworks', 'libcrypto.1.0.0.dylib'),
path.join(
environ['RESOURCEPATH'], '..',
'Frameworks', 'libcrypto.0.9.8.dylib'),
]) ])
elif 'win32' in sys.platform or 'win64' in sys.platform: elif 'win32' in sys.platform or 'win64' in sys.platform:
libdir.append(path.join(sys._MEIPASS, 'libeay32.dll')) libdir.append(path.join(sys._MEIPASS, 'libeay32.dll'))
@ -682,7 +708,8 @@ def loadOpenSSL():
path.join(sys._MEIPASS, 'libssl.so.0.9.8'), path.join(sys._MEIPASS, 'libssl.so.0.9.8'),
]) ])
if 'darwin' in sys.platform: if 'darwin' in sys.platform:
libdir.extend(['libcrypto.dylib', '/usr/local/opt/openssl/lib/libcrypto.dylib']) libdir.extend([
'libcrypto.dylib', '/usr/local/opt/openssl/lib/libcrypto.dylib'])
elif 'win32' in sys.platform or 'win64' in sys.platform: elif 'win32' in sys.platform or 'win64' in sys.platform:
libdir.append('libeay32.dll') libdir.append('libeay32.dll')
elif platform == "android": elif platform == "android":
@ -701,7 +728,6 @@ def loadOpenSSL():
libdir.append(find_library('ssl')) libdir.append(find_library('ssl'))
except OSError: except OSError:
pass pass
elif 'win32' in sys.platform or 'win64' in sys.platform:
libdir.append(find_library('libeay32')) libdir.append(find_library('libeay32'))
for library in libdir: for library in libdir:
try: try:
@ -709,7 +735,8 @@ def loadOpenSSL():
return return
except: except:
pass pass
raise Exception("Couldn't find and load the OpenSSL library. You must install it.") raise Exception(
"Couldn't find and load the OpenSSL library. You must install it.")
loadOpenSSL() loadOpenSSL()

70
src/storage/filesystem.py
View File

@ -1,6 +1,5 @@
""" """
src/storage/filesystem.py Module for using filesystem (directory with files) for inventory storage
=========================
""" """
from binascii import hexlify, unhexlify from binascii import hexlify, unhexlify
from os import listdir, makedirs, path, remove, rmdir from os import listdir, makedirs, path, remove, rmdir
@ -12,8 +11,9 @@ from paths import lookupAppdataFolder
from storage.storage import InventoryStorage, InventoryItem from storage.storage import InventoryStorage, InventoryItem
class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ancestors, abstract-method class FilesystemInventory(InventoryStorage):
"""Module for using filesystem (directory with files) for inventory storage""" """Filesystem for inventory storage"""
# pylint: disable=too-many-ancestors, abstract-method
topDir = "inventory" topDir = "inventory"
objectDir = "objects" objectDir = "objects"
metadataFilename = "metadata" metadataFilename = "metadata"
@ -21,21 +21,23 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
def __init__(self): def __init__(self):
super(FilesystemInventory, self).__init__() super(FilesystemInventory, self).__init__()
self.baseDir = path.join(lookupAppdataFolder(), FilesystemInventory.topDir) self.baseDir = path.join(
lookupAppdataFolder(), FilesystemInventory.topDir)
for createDir in [self.baseDir, path.join(self.baseDir, "objects")]: for createDir in [self.baseDir, path.join(self.baseDir, "objects")]:
if path.exists(createDir): if path.exists(createDir):
if not path.isdir(createDir): if not path.isdir(createDir):
raise IOError("%s exists but it's not a directory" % (createDir)) raise IOError(
"%s exists but it's not a directory" % createDir)
else: else:
makedirs(createDir) makedirs(createDir)
# Guarantees that two receiveDataThreads don't receive and process the same message # Guarantees that two receiveDataThreads
# don't receive and process the same message
# concurrently (probably sent by a malicious individual) # concurrently (probably sent by a malicious individual)
self.lock = RLock() self.lock = RLock()
self._inventory = {} self._inventory = {}
self._load() self._load()
def __contains__(self, hashval): def __contains__(self, hashval):
retval = False
for streamDict in self._inventory.values(): for streamDict in self._inventory.values():
if hashval in streamDict: if hashval in streamDict:
return True return True
@ -48,7 +50,12 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
except KeyError: except KeyError:
continue continue
if retval.payload is None: if retval.payload is None:
retval = InventoryItem(retval.type, retval.stream, self.getData(hashval), retval.expires, retval.tag) retval = InventoryItem(
retval.type,
retval.stream,
self.getData(hashval),
retval.expires,
retval.tag)
return retval return retval
raise KeyError(hashval) raise KeyError(hashval)
@ -56,7 +63,10 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
with self.lock: with self.lock:
value = InventoryItem(*value) value = InventoryItem(*value)
try: try:
makedirs(path.join(self.baseDir, FilesystemInventory.objectDir, hexlify(hashval))) makedirs(path.join(
self.baseDir,
FilesystemInventory.objectDir,
hexlify(hashval)))
except OSError: except OSError:
pass pass
try: try:
@ -69,7 +79,11 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
), ),
"w", "w",
) as f: ) as f:
f.write("%s,%s,%s,%s," % (value.type, value.stream, value.expires, hexlify(value.tag))) f.write("%s,%s,%s,%s," % (
value.type,
value.stream,
value.expires,
hexlify(value.tag)))
with open( with open(
path.join( path.join(
self.baseDir, self.baseDir,
@ -115,7 +129,10 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
except IOError: except IOError:
pass pass
try: try:
rmdir(path.join(self.baseDir, FilesystemInventory.objectDir, hexlify(hashval))) rmdir(path.join(
self.baseDir,
FilesystemInventory.objectDir,
hexlify(hashval)))
except IOError: except IOError:
pass pass
@ -135,7 +152,8 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
newInventory = {} newInventory = {}
for hashId in self.object_list(): for hashId in self.object_list():
try: try:
objectType, streamNumber, expiresTime, tag = self.getMetadata(hashId) objectType, streamNumber, expiresTime, tag = self.getMetadata(
hashId)
try: try:
newInventory[streamNumber][hashId] = InventoryItem( newInventory[streamNumber][hashId] = InventoryItem(
objectType, streamNumber, None, expiresTime, tag) objectType, streamNumber, None, expiresTime, tag)
@ -155,7 +173,8 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
def object_list(self): def object_list(self):
"""Return inventory vectors (hashes) from a directory""" """Return inventory vectors (hashes) from a directory"""
return [unhexlify(x) for x in listdir(path.join(self.baseDir, FilesystemInventory.objectDir))] return [unhexlify(x) for x in listdir(path.join(
self.baseDir, FilesystemInventory.objectDir))]
def getData(self, hashId): def getData(self, hashId):
"""Get object data""" """Get object data"""
@ -185,15 +204,20 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
), ),
"r", "r",
) as f: ) as f:
objectType, streamNumber, expiresTime, tag, undef = string.split(f.read(), ",", 4) objectType, streamNumber, expiresTime, tag = string.split(
return [int(objectType), int(streamNumber), int(expiresTime), unhexlify(tag)] f.read(), ",", 4)[:4]
return [
int(objectType),
int(streamNumber),
int(expiresTime),
unhexlify(tag)]
except IOError: except IOError:
raise KeyError raise KeyError
def by_type_and_tag(self, objectType, tag): def by_type_and_tag(self, objectType, tag):
"""Get a list of objects filtered by object type and tag""" """Get a list of objects filtered by object type and tag"""
retval = [] retval = []
for stream, streamDict in self._inventory: for streamDict in self._inventory.values():
for hashId, item in streamDict: for hashId, item in streamDict:
if item.type == objectType and item.tag == tag: if item.type == objectType and item.tag == tag:
try: try:
@ -201,7 +225,12 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
item.payload = self.getData(hashId) item.payload = self.getData(hashId)
except IOError: except IOError:
continue continue
retval.append(InventoryItem(item.type, item.stream, item.payload, item.expires, item.tag)) retval.append(InventoryItem(
item.type,
item.stream,
item.payload,
item.expires,
item.tag))
return retval return retval
def hashes_by_stream(self, stream): def hashes_by_stream(self, stream):
@ -215,7 +244,8 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
"""Return unexpired hashes in the inventory for a particular stream""" """Return unexpired hashes in the inventory for a particular stream"""
t = int(time.time()) t = int(time.time())
try: try:
return [x for x, value in self._inventory[stream].items() if value.expires > t] return [x for x, value in self._inventory[stream].items()
if value.expires > t]
except KeyError: except KeyError:
return [] return []
@ -227,7 +257,7 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
"""Clean out old items from the inventory""" """Clean out old items from the inventory"""
minTime = int(time.time()) - (60 * 60 * 30) minTime = int(time.time()) - (60 * 60 * 30)
deletes = [] deletes = []
for stream, streamDict in self._inventory.items(): for streamDict in self._inventory.values():
for hashId, item in streamDict.items(): for hashId, item in streamDict.items():
if item.expires < minTime: if item.expires < minTime:
deletes.append(hashId) deletes.append(hashId)

50
src/storage/sqlite.py
View File

@ -1,6 +1,5 @@
""" """
src/storage/sqlite.py Sqlite Inventory
=========================
""" """
import sqlite3 import sqlite3
import time import time
@ -11,7 +10,7 @@ from helper_sql import sqlQuery, SqlBulkExecute, sqlExecute
from storage.storage import InventoryStorage, InventoryItem from storage.storage import InventoryStorage, InventoryItem
class SqliteInventory(InventoryStorage): # pylint: disable=too-many-ancestors class SqliteInventory(InventoryStorage): # pylint: disable=too-many-ancestors
"""Inventory using SQLite""" """Inventory using SQLite"""
def __init__(self): def __init__(self):
super(SqliteInventory, self).__init__() super(SqliteInventory, self).__init__()
@ -21,9 +20,11 @@ class SqliteInventory(InventoryStorage): # pylint: disable=too-many-ancestors
# cache for existing objects, used for quick lookups if we have an object. # cache for existing objects, used for quick lookups if we have an object.
# This is used for example whenever we receive an inv message from a peer # This is used for example whenever we receive an inv message from a peer
# to check to see what items are new to us. # to check to see what items are new to us.
# We don't delete things out of it; instead, the singleCleaner thread clears and refills it. # We don't delete things out of it; instead,
# the singleCleaner thread clears and refills it.
self._objects = {} self._objects = {}
# Guarantees that two receiveDataThreads don't receive and process the same message concurrently # Guarantees that two receiveDataThreads don't receive
# and process the same message concurrently
# (probably sent by a malicious individual) # (probably sent by a malicious individual)
self.lock = RLock() self.lock = RLock()
@ -32,7 +33,9 @@ class SqliteInventory(InventoryStorage): # pylint: disable=too-many-ancestors
with self.lock: with self.lock:
if hash_ in self._objects: if hash_ in self._objects:
return True return True
rows = sqlQuery('SELECT streamnumber FROM inventory WHERE hash=?', sqlite3.Binary(hash_)) rows = sqlQuery(
'SELECT streamnumber FROM inventory WHERE hash=?',
sqlite3.Binary(hash_))
if not rows: if not rows:
return False return False
self._objects[hash_] = rows[0][0] self._objects[hash_] = rows[0][0]
@ -76,36 +79,49 @@ class SqliteInventory(InventoryStorage): # pylint: disable=too-many-ancestors
def __len__(self): def __len__(self):
print('----------__len__------------------') print('----------__len__------------------')
with self.lock: with self.lock:
return len(self._inventory) + sqlQuery('SELECT count(*) FROM inventory')[0][0] return len(self._inventory) + sqlQuery(
'SELECT count(*) FROM inventory')[0][0]
def by_type_and_tag(self, objectType, tag): def by_type_and_tag(self, objectType, tag):
print('----------by_type_and_tag------------------') """Return objects filtered by object type and tag"""
with self.lock: with self.lock:
values = [value for value in self._inventory.values() if value.type == objectType and value.tag == tag] values = [value for value in self._inventory.values()
if value.type == objectType and value.tag == tag]
values += (InventoryItem(*value) for value in sqlQuery( values += (InventoryItem(*value) for value in sqlQuery(
'SELECT objecttype, streamnumber, payload, expirestime, tag \ 'SELECT objecttype, streamnumber, payload, expirestime, tag'
FROM inventory WHERE objecttype=? AND tag=?', objectType, sqlite3.Binary(tag))) ' FROM inventory WHERE objecttype=? AND tag=?',
objectType, sqlite3.Binary(tag)))
return values return values
def unexpired_hashes_by_stream(self, stream): def unexpired_hashes_by_stream(self, stream):
"""Return unexpired inventory vectors filtered by stream"""
with self.lock: with self.lock:
t = int(time.time()) t = int(time.time())
hashes = [x for x, value in self._inventory.items() if value.stream == stream and value.expires > t] hashes = [x for x, value in self._inventory.items()
hashes += (payload for payload, in sqlQuery( if value.stream == stream and value.expires > t]
'SELECT hash FROM inventory WHERE streamnumber=? AND expirestime>?', stream, t)) hashes += (str(payload) for payload, in sqlQuery(
'SELECT hash FROM inventory WHERE streamnumber=?'
' AND expirestime>?', stream, t))
return hashes return hashes
def flush(self): def flush(self):
"""Flush cache"""
with self.lock: with self.lock:
# If you use both the inventoryLock and the sqlLock, always use the inventoryLock OUTSIDE of the sqlLock. # If you use both the inventoryLock and the sqlLock,
# always use the inventoryLock OUTSIDE of the sqlLock.
with SqlBulkExecute() as sql: with SqlBulkExecute() as sql:
for objectHash, value in self._inventory.items(): for objectHash, value in self._inventory.items():
sql.execute('INSERT INTO inventory VALUES (?, ?, ?, ?, ?, ?)', sqlite3.Binary(objectHash), *value) sql.execute(
'INSERT INTO inventory VALUES (?, ?, ?, ?, ?, ?)',
sqlite3.Binary(objectHash), *value)
self._inventory.clear() self._inventory.clear()
def clean(self): def clean(self):
"""Free memory / perform garbage collection"""
with self.lock: with self.lock:
sqlExecute('DELETE FROM inventory WHERE expirestime<?', int(time.time()) - (60 * 60 * 3)) sqlExecute(
'DELETE FROM inventory WHERE expirestime<?',
int(time.time()) - (60 * 60 * 3))
self._objects.clear() self._objects.clear()
for objectHash, value in self._inventory.items(): for objectHash, value in self._inventory.items():
self._objects[objectHash] = value.stream self._objects[objectHash] = value.stream

35
src/storage/storage.py
View File

@ -1,21 +1,22 @@
""" """
src/storage/storage.py Storing inventory items
======================
""" """
import collections import collections
InventoryItem = collections.namedtuple('InventoryItem', 'type stream payload expires tag') InventoryItem = collections.namedtuple(
'InventoryItem', 'type stream payload expires tag')
class Storage(object): class Storage(object): # pylint: disable=too-few-public-methods
"""Base class for storing inventory (extendable for other items to store)""" """Base class for storing inventory
(extendable for other items to store)"""
pass pass
class InventoryStorage(Storage, collections.MutableMapping): class InventoryStorage(Storage, collections.MutableMapping):
"""Module used for inventory storage""" """Module used for inventory storage"""
def __init__(self):
# pylint: disable=super-init-not-called def __init__(self): # pylint: disable=super-init-not-called
self.numberOfInventoryLookupsPerformed = 0 self.numberOfInventoryLookupsPerformed = 0
def __contains__(self, _): def __contains__(self, _):
@ -53,8 +54,20 @@ class InventoryStorage(Storage, collections.MutableMapping):
raise NotImplementedError raise NotImplementedError
class MailboxStorage(Storage, collections.MutableMapping): # pylint: disable=abstract-method class MailboxStorage(Storage, collections.MutableMapping):
"""Method for storing mails""" """Method for storing mails"""
def __init__(self):
# pylint: disable=super-init-not-called def __delitem__(self, key):
pass raise NotImplementedError
def __getitem__(self, key):
raise NotImplementedError
def __iter__(self):
raise NotImplementedError
def __len__(self):
raise NotImplementedError
def __setitem__(self, key, value):
raise NotImplementedError