Updated origin by upstream
This commit is contained in:
jai.s
commit
6b1abe7048
25
LICENSE
25
LICENSE
|
|
@ -22,7 +22,7 @@ SOFTWARE.
|
|||
|
||||
===== qidenticon.py identicon python implementation with QPixmap output by sendiulo <sendiulo@gmx.net>
|
||||
|
||||
qidenticon.py is Licesensed under FreeBSD License.
|
||||
qidenticon.py is Licensed under FreeBSD License.
|
||||
(http://www.freebsd.org/copyright/freebsd-license.html)
|
||||
|
||||
Copyright 2013 "Sendiulo". All rights reserved.
|
||||
|
|
@ -36,7 +36,7 @@ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER ``AS IS'' AND ANY EXPRESS OR I
|
|||
|
||||
===== based on identicon.py identicon python implementation. by Shin Adachi <shn@glucose.jp>
|
||||
|
||||
identicon.py is Licesensed under FreeBSD License.
|
||||
identicon.py is Licensed under FreeBSD License.
|
||||
(http://www.freebsd.org/copyright/freebsd-license.html)
|
||||
|
||||
Copyright 1994-2009 Shin Adachi. All rights reserved.
|
||||
|
|
@ -47,3 +47,24 @@ Redistribution and use in source and binary forms, with or without modification,
|
|||
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
===== based on asyncore_pollchoose.py asyncore_pollchoose python implementation. by Sam Rushing <rushing@nightmare.com>
|
||||
|
||||
Copyright 1996 by Sam Rushing. All Rights Reserved
|
||||
|
||||
Permission to use, copy, modify, and distribute this software and
|
||||
its documentation for any purpose and without fee is hereby
|
||||
granted, provided that the above copyright notice appear in all
|
||||
copies and that both that copyright notice and this permission
|
||||
notice appear in supporting documentation, and that the name of Sam
|
||||
Rushing not be used in advertising or publicity pertaining to
|
||||
distribution of the software without specific, written prior
|
||||
permission.
|
||||
|
||||
SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
|
||||
INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN
|
||||
NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR
|
||||
CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
|
||||
OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
|
||||
NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
|
||||
CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
PyBitmessage(Android)
|
||||
|
||||
This sample aims to be as close to a real world example of a mobile. It has a more refined design and also provides a practical example of how a mobile app would interact and communicate with its adresses.
|
||||
This sample aims to be as close to a real world example of a mobile. It has a more refined design and also provides a practical example of how a mobile app would interact and communicate with its addresses.
|
||||
|
||||
Steps for trying out this sample:
|
||||
|
||||
|
|
@ -13,7 +13,7 @@ This sample uses the kivy as Kivy is an open source, cross-platform Python frame
|
|||
|
||||
Kivy is written in Python and Cython, supports various input devices and has an extensive widget library. With the same codebase, you can target Windows, OS X, Linux, Android and iOS. All Kivy widgets are built with multitouch support.
|
||||
|
||||
Kivy in support take Buildozer which is a tool that automates the entire build process. It downloads and sets up all the prequisites for python-for-android, including the android SDK and NDK, then builds an apk that can be automatically pushed to the device.
|
||||
Kivy in support take Buildozer which is a tool that automates the entire build process. It downloads and sets up all the prerequisite for python-for-android, including the android SDK and NDK, then builds an apk that can be automatically pushed to the device.
|
||||
|
||||
Buildozer currently works only in Linux, and is an alpha release, but it already works well and can significantly simplify the apk build.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
#!/usr/bin/env python2
|
||||
"""
|
||||
Check dependendies and give recommendations about how to satisfy them
|
||||
Check dependencies and give recommendations about how to satisfy them
|
||||
|
||||
Limitations:
|
||||
|
||||
|
|
|
|||
|
|
@ -21,12 +21,12 @@ If we are to make bold claims about protecting your privacy we should demonstrat
|
|||
- looking to audit
|
||||
- warrant canary
|
||||
|
||||
Digital foootprint
|
||||
Digital footprint
|
||||
------------------
|
||||
|
||||
Your internet use can reveal metadata you wouldn't expect. This can be connected with other information about you if you're not careful.
|
||||
|
||||
* Use separate addresses for different puprose
|
||||
* Use separate addresses for different purposes
|
||||
* Don't make the same mistakes all the time
|
||||
* Your language use is unique. The more you type, the more you fingerprint yourself. The words you know and use often vs the words you don't know or use often.
|
||||
|
||||
|
|
|
|||
|
|
@ -11,17 +11,17 @@ Bitmessage makes use of fabric_ to define tasks such as building documentation o
|
|||
Code style and linters
|
||||
----------------------
|
||||
|
||||
We aim to be PEP8 compliant but we recognise that we have a long way still to go. Currently we have style and lint exceptions specified at the most specific place we can. We are ignoring certain issues project-wide in order to avoid alert-blindess, avoid style and lint regressions and to allow continuous integration to hook into the output from the tools. While it is hoped that all new changes pass the checks, fixing some existing violations are mini-projects in themselves. Current thinking on ignorable violations is reflected in the options and comments in setup.cfg. Module and line-level lint warnings represent refactoring opportunities.
|
||||
We aim to be PEP8 compliant but we recognize that we have a long way still to go. Currently we have style and lint exceptions specified at the most specific place we can. We are ignoring certain issues project-wide in order to avoid alert-blindness, avoid style and lint regressions and to allow continuous integration to hook into the output from the tools. While it is hoped that all new changes pass the checks, fixing some existing violations are mini-projects in themselves. Current thinking on ignorable violations is reflected in the options and comments in setup.cfg. Module and line-level lint warnings represent refactoring opportunities.
|
||||
|
||||
Pull requests
|
||||
-------------
|
||||
|
||||
There is a template at PULL_REQUEST_TEMPLATE.md that appears in the pull-request description. Please replace this text with something appropriate to your changes based off the ideas in the template.
|
||||
There is a template at PULL_REQUEST_TEMPLATE.md that appears in the pull-request description. Please replace this text with something appropriate to your changes based on the ideas in the template.
|
||||
|
||||
Bike-shedding
|
||||
-------------
|
||||
|
||||
Beyond having well-documented, Pythonic code with static analysis tool checks, extensive test coverage and powerful devops tools, what else can we have? Without violating any linters there is room for making arbirary decisions solely for the sake of project consistency. These are the stuff of the pedant's PR comments. Rather than have such conversations in PR comments, we can lay out the result of discussion here.
|
||||
Beyond having well-documented, Pythonic code with static analysis tool checks, extensive test coverage and powerful devops tools, what else can we have? Without violating any linters there is room for making arbitrary decisions solely for the sake of project consistency. These are the stuff of the pedant's PR comments. Rather than have such conversations in PR comments, we can lay out the result of discussion here.
|
||||
|
||||
I'm putting up a strawman for each topic here, mostly based on my memory of reading related Stack Overflow articles etc. If contributors feel strongly (and we don't have anything better to do) then maybe we can convince each other to update this section.
|
||||
|
||||
|
|
@ -49,7 +49,7 @@ British vs American spelling
|
|||
Dependency graph
|
||||
----------------
|
||||
|
||||
These images are not very useful right now but the aim is to tweak the settings of one or more of them to be informative, and/or divide them up into smaller grapghs.
|
||||
These images are not very useful right now but the aim is to tweak the settings of one or more of them to be informative, and/or divide them up into smaller graphs.
|
||||
|
||||
To re-build them, run `fab build_docs:dep_graphs=true`. Note that the dot graph takes a lot of time.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Fabric
|
||||
|
||||
[Fabric](https://www.fabfile.org) is a Python library for performing devops tasks. You can thing of it a bit like a
|
||||
[Fabric](https://www.fabfile.org) is a Python library for performing devops tasks. You can think of it a bit like a
|
||||
makefile on steroids for Python. Its api abstracts away the clunky way you would run shell commands in Python, check
|
||||
return values and manage stdio. Tasks may be targetted at particular hosts or group of hosts.
|
||||
|
||||
|
|
@ -46,7 +46,7 @@ Furthermore, you can use -- to run arbitrary shell commands rather than tasks:
|
|||
|
||||
There are a number of advantages that should benefit us:
|
||||
|
||||
* Common tasks can be writen in Python and executed consistently
|
||||
* Common tasks can be written in Python and executed consistently
|
||||
* Common tasks are now under source control
|
||||
* All developers can run the same commands, if the underlying command sequence for a task changes (after review, obv)
|
||||
the user does not have to care
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ OSX:
|
|||
|
||||
https://github.com/Bitmessage/PyBitmessage/releases
|
||||
|
||||
Wors on OSX 10.7.5 or higher
|
||||
Works on OSX 10.7.5 or higher
|
||||
|
||||
|
||||
Arch linux:
|
||||
|
|
|
|||
|
|
@ -1,8 +1,6 @@
|
|||
"""
|
||||
src/bitmessagecurses/__init__.py
|
||||
================================
|
||||
Bitmessage commandline interface
|
||||
"""
|
||||
|
||||
# Copyright (c) 2014 Luke Montalvo <lukemontalvo@gmail.com>
|
||||
# This file adds a alternative commandline interface, feel free to critique and fork
|
||||
#
|
||||
|
|
@ -20,21 +18,22 @@ import time
|
|||
from textwrap import fill
|
||||
from threading import Timer
|
||||
|
||||
from addresses import addBMIfNotPresent, decodeAddress
|
||||
from bmconfigparser import BMConfigParser
|
||||
from dialog import Dialog
|
||||
from helper_ackPayload import genAckPayload
|
||||
from helper_sql import sqlExecute, sqlQuery
|
||||
from inventory import Inventory
|
||||
import l10n
|
||||
import network.stats
|
||||
from pyelliptic.openssl import OpenSSL
|
||||
import queues
|
||||
import shared
|
||||
import shutdown
|
||||
|
||||
from addresses import addBMIfNotPresent, decodeAddress
|
||||
from bmconfigparser import BMConfigParser
|
||||
from helper_ackPayload import genAckPayload
|
||||
from helper_sql import sqlExecute, sqlQuery
|
||||
from inventory import Inventory
|
||||
# pylint: disable=global-statement
|
||||
|
||||
quit = False # pylint: disable=redefined-builtin
|
||||
|
||||
quit_ = False
|
||||
menutab = 1
|
||||
menu = ["Inbox", "Send", "Sent", "Your Identities", "Subscriptions", "Address Book", "Blacklist", "Network Status"]
|
||||
naptime = 100
|
||||
|
|
@ -61,26 +60,31 @@ bwtype = "black"
|
|||
BROADCAST_STR = "[Broadcast subscribers]"
|
||||
|
||||
|
||||
class printLog: # pylint: disable=no-self-use, no-init, old-style-class
|
||||
class printLog(object):
|
||||
"""Printing logs"""
|
||||
# pylint: disable=no-self-use
|
||||
|
||||
def write(self, output):
|
||||
# pylint: disable=global-statement
|
||||
"""Write logs"""
|
||||
global log
|
||||
log += output
|
||||
|
||||
def flush(self):
|
||||
"""Flush logs"""
|
||||
pass
|
||||
|
||||
|
||||
class errLog: # pylint: disable=no-self-use, no-init, old-style-class
|
||||
class errLog(object):
|
||||
"""Error logs"""
|
||||
# pylint: disable=no-self-use
|
||||
|
||||
def write(self, output):
|
||||
# pylint: disable=global-statement
|
||||
"""Write error logs"""
|
||||
global log
|
||||
log += "!" + output
|
||||
|
||||
def flush(self):
|
||||
"""Flush error logs"""
|
||||
pass
|
||||
|
||||
|
||||
|
|
@ -138,14 +142,15 @@ def scrollbox(d, text, height=None, width=None):
|
|||
|
||||
def resetlookups():
|
||||
"""Reset the Inventory Lookups"""
|
||||
global inventorydata # pylint: disable=global-statement
|
||||
global inventorydata
|
||||
inventorydata = Inventory().numberOfInventoryLookupsPerformed
|
||||
Inventory().numberOfInventoryLookupsPerformed = 0
|
||||
Timer(1, resetlookups, ()).start()
|
||||
|
||||
|
||||
def drawtab(stdscr): # pylint: disable=too-many-branches, too-many-statements
|
||||
def drawtab(stdscr):
|
||||
"""Method for drawing different tabs"""
|
||||
# pylint: disable=too-many-branches, too-many-statements
|
||||
if menutab in range(1, len(menu) + 1):
|
||||
if menutab == 1: # Inbox
|
||||
stdscr.addstr(3, 5, "To", curses.A_BOLD)
|
||||
|
|
@ -282,12 +287,12 @@ def drawtab(stdscr): # pylint: disable=too-many-branches, too-many-statem
|
|||
stdscr.addstr(13, 6, "Log", curses.A_BOLD)
|
||||
n = log.count('\n')
|
||||
if n > 0:
|
||||
l = log.split('\n')
|
||||
lg = log.split('\n')
|
||||
if n > 512:
|
||||
del l[:(n - 256)]
|
||||
del lg[:(n - 256)]
|
||||
logpad.erase()
|
||||
n = len(l)
|
||||
for i, item in enumerate(l):
|
||||
n = len(lg)
|
||||
for i, item in enumerate(lg):
|
||||
a = 0
|
||||
if item and item[0] == '!':
|
||||
a = curses.color_pair(1)
|
||||
|
|
@ -314,7 +319,8 @@ def dialogreset(stdscr):
|
|||
|
||||
# pylint: disable=too-many-branches, too-many-statements
|
||||
def handlech(c, stdscr):
|
||||
# pylint: disable=redefined-outer-name, too-many-nested-blocks, too-many-locals, global-statement
|
||||
"""Handle character given on the command-line interface"""
|
||||
# pylint: disable=redefined-outer-name, too-many-nested-blocks, too-many-locals
|
||||
if c != curses.ERR:
|
||||
global inboxcur, addrcur, sentcur, subcur, abookcur, blackcur
|
||||
if c in range(256):
|
||||
|
|
@ -322,8 +328,8 @@ def handlech(c, stdscr):
|
|||
global menutab
|
||||
menutab = int(chr(c))
|
||||
elif chr(c) == 'q':
|
||||
global quit
|
||||
quit = True
|
||||
global quit_
|
||||
quit_ = True
|
||||
elif chr(c) == '\n':
|
||||
curses.curs_set(1)
|
||||
d = Dialog(dialog="dialog")
|
||||
|
|
@ -877,7 +883,7 @@ def sendMessage(sender="", recv="", broadcast=None, subject="", body="", reply=F
|
|||
10,
|
||||
60)
|
||||
if r != d.DIALOG_OK:
|
||||
global menutab # pylint: disable=global-statement
|
||||
global menutab
|
||||
menutab = 6
|
||||
return
|
||||
recv = t
|
||||
|
|
@ -906,7 +912,7 @@ def sendMessage(sender="", recv="", broadcast=None, subject="", body="", reply=F
|
|||
|
||||
if not broadcast:
|
||||
recvlist = []
|
||||
for i, item in enumerate(recv.replace(",", ";").split(";")):
|
||||
for _, item in enumerate(recv.replace(",", ";").split(";")):
|
||||
recvlist.append(item.strip())
|
||||
list(set(recvlist)) # Remove exact duplicates
|
||||
for addr in recvlist:
|
||||
|
|
@ -1170,7 +1176,7 @@ def loadSubscriptions():
|
|||
|
||||
def loadBlackWhiteList():
|
||||
"""load black/white list"""
|
||||
global bwtype # pylint: disable=global-statement
|
||||
global bwtype
|
||||
bwtype = BMConfigParser().get("bitmessagesettings", "blackwhitelist")
|
||||
if bwtype == "black":
|
||||
ret = sqlQuery("SELECT label, address, enabled FROM blacklist")
|
||||
|
|
@ -1183,10 +1189,10 @@ def loadBlackWhiteList():
|
|||
|
||||
|
||||
def runwrapper():
|
||||
"""Main method"""
|
||||
sys.stdout = printlog
|
||||
# sys.stderr = errlog
|
||||
|
||||
# Load messages from database
|
||||
loadInbox()
|
||||
loadSent()
|
||||
loadAddrBook()
|
||||
|
|
@ -1195,7 +1201,7 @@ def runwrapper():
|
|||
|
||||
stdscr = curses.initscr()
|
||||
|
||||
global logpad # pylint: disable=global-statement
|
||||
global logpad
|
||||
logpad = curses.newpad(1024, curses.COLS)
|
||||
|
||||
stdscr.nodelay(0)
|
||||
|
|
@ -1207,6 +1213,7 @@ def runwrapper():
|
|||
|
||||
|
||||
def run(stdscr):
|
||||
"""Main loop"""
|
||||
# Schedule inventory lookup data
|
||||
resetlookups()
|
||||
|
||||
|
|
@ -1246,7 +1253,7 @@ def run(stdscr):
|
|||
|
||||
stdscr.clear()
|
||||
redraw(stdscr)
|
||||
while quit is False:
|
||||
while quit_ is False:
|
||||
drawtab(stdscr)
|
||||
handlech(stdscr.getch(), stdscr)
|
||||
|
||||
|
|
@ -1259,5 +1266,4 @@ def doShutdown():
|
|||
shutdown.doCleanShutdown()
|
||||
sys.stdout = sys.__stdout__
|
||||
sys.stderr = sys.__stderr__
|
||||
|
||||
os._exit(0) # pylint: disable=protected-access
|
||||
|
|
|
|||
|
|
@ -150,11 +150,14 @@ class objectProcessor(threading.Thread):
|
|||
'ackreceived', int(time.time()), data[readPosition:])
|
||||
queues.UISignalQueue.put((
|
||||
'updateSentItemStatusByAckdata',
|
||||
(data[readPosition:],
|
||||
(
|
||||
data[readPosition:],
|
||||
tr._translate(
|
||||
"MainWindow",
|
||||
"Acknowledgement of the message received %1").arg(
|
||||
l10n.formatTimestamp()))))
|
||||
"Acknowledgement of the message received %1"
|
||||
).arg(l10n.formatTimestamp())
|
||||
)
|
||||
))
|
||||
else:
|
||||
logger.debug('This object is not an acknowledgement bound for me.')
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
"""
|
||||
src/class_smtpDeliver.py
|
||||
========================
|
||||
SMTP client thread for delivering emails
|
||||
"""
|
||||
# pylint: disable=unused-variable
|
||||
import smtplib
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
"""
|
||||
src/multiqueue.py
|
||||
=================
|
||||
A queue with multiple internal subqueues.
|
||||
Elements are added into a random subqueue, and retrieval rotates
|
||||
"""
|
||||
|
||||
import queue as Queue
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ from network.threads import StoppableThread
|
|||
|
||||
|
||||
class AddrThread(StoppableThread):
|
||||
"""(Node) address broadcasting thread"""
|
||||
name = "AddrBroadcaster"
|
||||
|
||||
def run(self):
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
"""
|
||||
src/network/advanceddispatcher.py
|
||||
=================================
|
||||
Improved version of asyncore dispatcher
|
||||
"""
|
||||
# pylint: disable=attribute-defined-outside-init
|
||||
import socket
|
||||
|
|
@ -13,7 +12,8 @@ from network.threads import BusyError, nonBlocking
|
|||
|
||||
|
||||
class ProcessingError(Exception):
|
||||
"""General class for protocol parser exception, use as a base for others."""
|
||||
"""General class for protocol parser exception,
|
||||
use as a base for others."""
|
||||
pass
|
||||
|
||||
|
||||
|
|
@ -23,7 +23,8 @@ class UnknownStateError(ProcessingError):
|
|||
|
||||
|
||||
class AdvancedDispatcher(asyncore.dispatcher):
|
||||
"""Improved version of asyncore dispatcher, with buffers and protocol state."""
|
||||
"""Improved version of asyncore dispatcher,
|
||||
with buffers and protocol state."""
|
||||
# pylint: disable=too-many-instance-attributes
|
||||
_buf_len = 131072 # 128kB
|
||||
|
||||
|
|
@ -74,7 +75,8 @@ class AdvancedDispatcher(asyncore.dispatcher):
|
|||
del self.read_buf[0:length]
|
||||
|
||||
def process(self):
|
||||
"""Process (parse) data that's in the buffer, as long as there is enough data and the connection is open."""
|
||||
"""Process (parse) data that's in the buffer,
|
||||
as long as there is enough data and the connection is open."""
|
||||
while self.connected and not state.shutdown:
|
||||
try:
|
||||
with nonBlocking(self.processingLock):
|
||||
|
|
@ -106,8 +108,9 @@ class AdvancedDispatcher(asyncore.dispatcher):
|
|||
if asyncore.maxUploadRate > 0:
|
||||
self.uploadChunk = int(asyncore.uploadBucket)
|
||||
self.uploadChunk = min(self.uploadChunk, len(self.write_buf))
|
||||
return asyncore.dispatcher.writable(self) and \
|
||||
(self.connecting or (self.connected and self.uploadChunk > 0))
|
||||
return asyncore.dispatcher.writable(self) and (
|
||||
self.connecting or (
|
||||
self.connected and self.uploadChunk > 0))
|
||||
|
||||
def readable(self):
|
||||
"""Is the read buffer ready to accept data from the network?"""
|
||||
|
|
@ -116,13 +119,15 @@ class AdvancedDispatcher(asyncore.dispatcher):
|
|||
self.downloadChunk = int(asyncore.downloadBucket)
|
||||
try:
|
||||
if self.expectBytes > 0 and not self.fullyEstablished:
|
||||
self.downloadChunk = min(self.downloadChunk, self.expectBytes - len(self.read_buf))
|
||||
self.downloadChunk = min(
|
||||
self.downloadChunk, self.expectBytes - len(self.read_buf))
|
||||
if self.downloadChunk < 0:
|
||||
self.downloadChunk = 0
|
||||
except AttributeError:
|
||||
pass
|
||||
return asyncore.dispatcher.readable(self) and \
|
||||
(self.connecting or self.accepting or (self.connected and self.downloadChunk > 0))
|
||||
return asyncore.dispatcher.readable(self) and (
|
||||
self.connecting or self.accepting or (
|
||||
self.connected and self.downloadChunk > 0))
|
||||
|
||||
def handle_read(self):
|
||||
"""Append incoming data to the read buffer."""
|
||||
|
|
@ -146,20 +151,21 @@ class AdvancedDispatcher(asyncore.dispatcher):
|
|||
try:
|
||||
asyncore.dispatcher.handle_connect_event(self)
|
||||
except socket.error as e:
|
||||
if e.args[0] not in asyncore._DISCONNECTED: # pylint: disable=protected-access
|
||||
# pylint: disable=protected-access
|
||||
if e.args[0] not in asyncore._DISCONNECTED:
|
||||
raise
|
||||
|
||||
def handle_connect(self):
|
||||
"""Method for handling connection established implementations."""
|
||||
self.lastTx = time.time()
|
||||
|
||||
def state_close(self):
|
||||
def state_close(self): # pylint: disable=no-self-use
|
||||
"""Signal to the processing loop to end."""
|
||||
# pylint: disable=no-self-use
|
||||
return False
|
||||
|
||||
def handle_close(self):
|
||||
"""Callback for connection being closed, but can also be called directly when you want connection to close."""
|
||||
"""Callback for connection being closed,
|
||||
but can also be called directly when you want connection to close."""
|
||||
with self.readLock:
|
||||
self.read_buf = bytearray()
|
||||
with self.writeLock:
|
||||
|
|
|
|||
|
|
@ -1,8 +1,6 @@
|
|||
"""
|
||||
src/network/announcethread.py
|
||||
=================================
|
||||
Announce myself (node address)
|
||||
"""
|
||||
|
||||
import time
|
||||
|
||||
import state
|
||||
|
|
@ -43,6 +41,7 @@ class AnnounceThread(StoppableThread):
|
|||
# connection.append_write_buf(BMProto.assembleAddr([addr]))
|
||||
Peer(
|
||||
'127.0.0.1',
|
||||
BMConfigParser().safeGetInt('bitmessagesettings', 'port')),
|
||||
BMConfigParser().safeGetInt(
|
||||
'bitmessagesettings', 'port')),
|
||||
time.time())
|
||||
connection.append_write_buf(assemble_addr([addr]))
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
"""
|
||||
Create bitmessage protocol command packets
|
||||
"""
|
||||
|
||||
import struct
|
||||
|
||||
import addresses
|
||||
|
|
@ -15,18 +14,19 @@ def assemble_addr(peerList):
|
|||
if isinstance(peerList, Peer):
|
||||
peerList = [peerList]
|
||||
if not peerList:
|
||||
return b''
|
||||
retval = b''
|
||||
return bytes()
|
||||
retval = bytes()
|
||||
for i in range(0, len(peerList), MAX_ADDR_COUNT):
|
||||
payload = addresses.encodeVarint(
|
||||
len(peerList[i:i + MAX_ADDR_COUNT]))
|
||||
payload = addresses.encodeVarint(len(peerList[i:i + MAX_ADDR_COUNT]))
|
||||
for stream, peer, timestamp in peerList[i:i + MAX_ADDR_COUNT]:
|
||||
payload += struct.pack(
|
||||
'>Q', int(timestamp)) # 64-bit time
|
||||
|
||||
payload += struct.pack('>I', stream)
|
||||
payload += struct.pack(
|
||||
'>q', 1) # service bit flags offered by this node
|
||||
# service bit flags offered by this node
|
||||
payload += struct.pack('>q', 1)
|
||||
payload += encodeHost(peer.host)
|
||||
payload += struct.pack('>H', peer.port) # remote port
|
||||
# remote port
|
||||
payload += struct.pack('>H', peer.port)
|
||||
retval += CreatePacket('addr', payload)
|
||||
return retval
|
||||
|
|
|
|||
|
|
@ -1,56 +1,11 @@
|
|||
"""
|
||||
Basic infrastructure for asynchronous socket service clients and servers.
|
||||
"""
|
||||
# -*- Mode: Python -*-
|
||||
# Id: asyncore.py,v 2.51 2000/09/07 22:29:26 rushing Exp
|
||||
# Author: Sam Rushing <rushing@nightmare.com>
|
||||
# pylint: disable=too-many-statements,too-many-branches,no-self-use,too-many-lines,attribute-defined-outside-init
|
||||
# pylint: disable=global-statement
|
||||
"""
|
||||
src/network/asyncore_pollchoose.py
|
||||
==================================
|
||||
|
||||
# ======================================================================
|
||||
# Copyright 1996 by Sam Rushing
|
||||
#
|
||||
# All Rights Reserved
|
||||
#
|
||||
# Permission to use, copy, modify, and distribute this software and
|
||||
# its documentation for any purpose and without fee is hereby
|
||||
# granted, provided that the above copyright notice appear in all
|
||||
# copies and that both that copyright notice and this permission
|
||||
# notice appear in supporting documentation, and that the name of Sam
|
||||
# Rushing not be used in advertising or publicity pertaining to
|
||||
# distribution of the software without specific, written prior
|
||||
# permission.
|
||||
#
|
||||
# SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
|
||||
# INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN
|
||||
# NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR
|
||||
# CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
|
||||
# OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
|
||||
# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
|
||||
# CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||
# ======================================================================
|
||||
|
||||
Basic infrastructure for asynchronous socket service clients and servers.
|
||||
|
||||
There are only two ways to have a program on a single processor do "more
|
||||
than one thing at a time". Multi-threaded programming is the simplest and
|
||||
most popular way to do it, but there is another very different technique,
|
||||
that lets you have nearly all the advantages of multi-threading, without
|
||||
actually using multiple threads. it's really only practical if your program
|
||||
is largely I/O bound. If your program is CPU bound, then pre-emptive
|
||||
scheduled threads are probably what you really need. Network servers are
|
||||
rarely CPU-bound, however.
|
||||
|
||||
If your operating system supports the select() system call in its I/O
|
||||
library (and nearly all do), then you can use it to juggle multiple
|
||||
communication channels at once; doing other work while your I/O is taking
|
||||
place in the "background." Although this strategy can seem strange and
|
||||
complex, especially at first, it is in many ways easier to understand and
|
||||
control than multi-threaded programming. The module documented here solves
|
||||
many of the difficult problems for you, making the task of building
|
||||
sophisticated high-performance network servers and clients a snap.
|
||||
"""
|
||||
|
||||
# pylint: disable=too-many-branches,too-many-lines,global-statement
|
||||
# pylint: disable=redefined-builtin,no-self-use
|
||||
import os
|
||||
import select
|
||||
import socket
|
||||
|
|
@ -58,8 +13,9 @@ import sys
|
|||
import time
|
||||
import warnings
|
||||
from errno import (
|
||||
EADDRINUSE, EAGAIN, EALREADY, EBADF, ECONNABORTED, ECONNREFUSED, ECONNRESET, EHOSTUNREACH, EINPROGRESS, EINTR,
|
||||
EINVAL, EISCONN, ENETUNREACH, ENOTCONN, ENOTSOCK, EPIPE, ESHUTDOWN, ETIMEDOUT, EWOULDBLOCK, errorcode
|
||||
EADDRINUSE, EAGAIN, EALREADY, EBADF, ECONNABORTED, ECONNREFUSED,
|
||||
ECONNRESET, EHOSTUNREACH, EINPROGRESS, EINTR, EINVAL, EISCONN, ENETUNREACH,
|
||||
ENOTCONN, ENOTSOCK, EPIPE, ESHUTDOWN, ETIMEDOUT, EWOULDBLOCK, errorcode
|
||||
)
|
||||
from threading import current_thread
|
||||
|
||||
|
|
@ -108,7 +64,8 @@ def _strerror(err):
|
|||
|
||||
|
||||
class ExitNow(Exception):
|
||||
"""We don't use directly but may be necessary as we replace asyncore due to some library raising or expecting it"""
|
||||
"""We don't use directly but may be necessary as we replace
|
||||
asyncore due to some library raising or expecting it"""
|
||||
pass
|
||||
|
||||
|
||||
|
|
@ -153,7 +110,8 @@ def write(obj):
|
|||
def set_rates(download, upload):
|
||||
"""Set throttling rates"""
|
||||
|
||||
global maxDownloadRate, maxUploadRate, downloadBucket, uploadBucket, downloadTimestamp, uploadTimestamp
|
||||
global maxDownloadRate, maxUploadRate, downloadBucket
|
||||
global uploadBucket, downloadTimestamp, uploadTimestamp
|
||||
|
||||
maxDownloadRate = float(download) * 1024
|
||||
maxUploadRate = float(upload) * 1024
|
||||
|
|
@ -183,7 +141,8 @@ def update_received(download=0):
|
|||
currentTimestamp = time.time()
|
||||
receivedBytes += download
|
||||
if maxDownloadRate > 0:
|
||||
bucketIncrease = maxDownloadRate * (currentTimestamp - downloadTimestamp)
|
||||
bucketIncrease = \
|
||||
maxDownloadRate * (currentTimestamp - downloadTimestamp)
|
||||
downloadBucket += bucketIncrease
|
||||
if downloadBucket > maxDownloadRate:
|
||||
downloadBucket = int(maxDownloadRate)
|
||||
|
|
@ -243,7 +202,6 @@ def readwrite(obj, flags):
|
|||
|
||||
def select_poller(timeout=0.0, map=None):
|
||||
"""A poller which uses select(), available on most platforms."""
|
||||
# pylint: disable=redefined-builtin
|
||||
|
||||
if map is None:
|
||||
map = socket_map
|
||||
|
|
@ -298,7 +256,6 @@ def select_poller(timeout=0.0, map=None):
|
|||
|
||||
def poll_poller(timeout=0.0, map=None):
|
||||
"""A poller which uses poll(), available on most UNIXen."""
|
||||
# pylint: disable=redefined-builtin
|
||||
|
||||
if map is None:
|
||||
map = socket_map
|
||||
|
|
@ -356,7 +313,6 @@ poll2 = poll3 = poll_poller
|
|||
|
||||
def epoll_poller(timeout=0.0, map=None):
|
||||
"""A poller which uses epoll(), supported on Linux 2.5.44 and newer."""
|
||||
# pylint: disable=redefined-builtin
|
||||
|
||||
if map is None:
|
||||
map = socket_map
|
||||
|
|
@ -412,7 +368,7 @@ def epoll_poller(timeout=0.0, map=None):
|
|||
|
||||
def kqueue_poller(timeout=0.0, map=None):
|
||||
"""A poller which uses kqueue(), BSD specific."""
|
||||
# pylint: disable=redefined-builtin,no-member
|
||||
# pylint: disable=no-member,too-many-statements
|
||||
|
||||
if map is None:
|
||||
map = socket_map
|
||||
|
|
@ -440,14 +396,20 @@ def kqueue_poller(timeout=0.0, map=None):
|
|||
poller_flags |= select.KQ_EV_ENABLE
|
||||
else:
|
||||
poller_flags |= select.KQ_EV_DISABLE
|
||||
updates.append(select.kevent(fd, filter=select.KQ_FILTER_READ, flags=poller_flags))
|
||||
updates.append(
|
||||
select.kevent(
|
||||
fd, filter=select.KQ_FILTER_READ,
|
||||
flags=poller_flags))
|
||||
if kq_filter & 2 != obj.poller_filter & 2:
|
||||
poller_flags = select.KQ_EV_ADD
|
||||
if kq_filter & 2:
|
||||
poller_flags |= select.KQ_EV_ENABLE
|
||||
else:
|
||||
poller_flags |= select.KQ_EV_DISABLE
|
||||
updates.append(select.kevent(fd, filter=select.KQ_FILTER_WRITE, flags=poller_flags))
|
||||
updates.append(
|
||||
select.kevent(
|
||||
fd, filter=select.KQ_FILTER_WRITE,
|
||||
flags=poller_flags))
|
||||
obj.poller_filter = kq_filter
|
||||
|
||||
if not selectables:
|
||||
|
|
@ -481,7 +443,6 @@ def kqueue_poller(timeout=0.0, map=None):
|
|||
|
||||
def loop(timeout=30.0, _=False, map=None, count=None, poller=None):
|
||||
"""Poll in a loop, until count or timeout is reached"""
|
||||
# pylint: disable=redefined-builtin
|
||||
|
||||
if map is None:
|
||||
map = socket_map
|
||||
|
|
@ -522,7 +483,7 @@ def loop(timeout=30.0, _=False, map=None, count=None, poller=None):
|
|||
|
||||
class dispatcher(object):
|
||||
"""Dispatcher for socket objects"""
|
||||
# pylint: disable=too-many-public-methods,too-many-instance-attributes,old-style-class
|
||||
# pylint: disable=too-many-public-methods,too-many-instance-attributes
|
||||
|
||||
debug = False
|
||||
connected = False
|
||||
|
|
@ -537,7 +498,6 @@ class dispatcher(object):
|
|||
minTx = 1500
|
||||
|
||||
def __init__(self, sock=None, map=None):
|
||||
# pylint: disable=redefined-builtin
|
||||
if map is None:
|
||||
self._map = socket_map
|
||||
else:
|
||||
|
|
@ -586,8 +546,7 @@ class dispatcher(object):
|
|||
|
||||
def add_channel(self, map=None):
|
||||
"""Add a channel"""
|
||||
# pylint: disable=redefined-builtin
|
||||
|
||||
# pylint: disable=attribute-defined-outside-init
|
||||
if map is None:
|
||||
map = self._map
|
||||
map[self._fileno] = self
|
||||
|
|
@ -596,8 +555,6 @@ class dispatcher(object):
|
|||
|
||||
def del_channel(self, map=None):
|
||||
"""Delete a channel"""
|
||||
# pylint: disable=redefined-builtin
|
||||
|
||||
fd = self._fileno
|
||||
if map is None:
|
||||
map = self._map
|
||||
|
|
@ -605,11 +562,13 @@ class dispatcher(object):
|
|||
del map[fd]
|
||||
if self._fileno:
|
||||
try:
|
||||
kqueue_poller.pollster.control([select.kevent(fd, select.KQ_FILTER_READ, select.KQ_EV_DELETE)], 0)
|
||||
kqueue_poller.pollster.control([select.kevent(
|
||||
fd, select.KQ_FILTER_READ, select.KQ_EV_DELETE)], 0)
|
||||
except(AttributeError, KeyError, TypeError, IOError, OSError):
|
||||
pass
|
||||
try:
|
||||
kqueue_poller.pollster.control([select.kevent(fd, select.KQ_FILTER_WRITE, select.KQ_EV_DELETE)], 0)
|
||||
kqueue_poller.pollster.control([select.kevent(
|
||||
fd, select.KQ_FILTER_WRITE, select.KQ_EV_DELETE)], 0)
|
||||
except(AttributeError, KeyError, TypeError, IOError, OSError):
|
||||
pass
|
||||
try:
|
||||
|
|
@ -627,8 +586,10 @@ class dispatcher(object):
|
|||
self.poller_filter = 0
|
||||
self.poller_registered = False
|
||||
|
||||
def create_socket(self, family=socket.AF_INET, socket_type=socket.SOCK_STREAM):
|
||||
def create_socket(
|
||||
self, family=socket.AF_INET, socket_type=socket.SOCK_STREAM):
|
||||
"""Create a socket"""
|
||||
# pylint: disable=attribute-defined-outside-init
|
||||
self.family_and_type = family, socket_type
|
||||
sock = socket.socket(family, socket_type)
|
||||
sock.setblocking(0)
|
||||
|
|
@ -636,20 +597,16 @@ class dispatcher(object):
|
|||
|
||||
def set_socket(self, sock, map=None):
|
||||
"""Set socket"""
|
||||
# pylint: disable=redefined-builtin
|
||||
|
||||
self.socket = sock
|
||||
self._fileno = sock.fileno()
|
||||
self.add_channel(map)
|
||||
|
||||
def set_reuse_addr(self):
|
||||
"""try to re-use a server port if possible"""
|
||||
|
||||
try:
|
||||
self.socket.setsockopt(
|
||||
socket.SOL_SOCKET, socket.SO_REUSEADDR,
|
||||
self.socket.getsockopt(socket.SOL_SOCKET,
|
||||
socket.SO_REUSEADDR) | 1
|
||||
socket.SOL_SOCKET, socket.SO_REUSEADDR, self.socket.getsockopt(
|
||||
socket.SOL_SOCKET, socket.SO_REUSEADDR) | 1
|
||||
)
|
||||
except socket.error:
|
||||
pass
|
||||
|
|
@ -704,13 +661,16 @@ class dispatcher(object):
|
|||
raise socket.error(err, errorcode[err])
|
||||
|
||||
def accept(self):
|
||||
"""Accept incoming connections. Returns either an address pair or None."""
|
||||
"""Accept incoming connections.
|
||||
Returns either an address pair or None."""
|
||||
try:
|
||||
conn, addr = self.socket.accept()
|
||||
except TypeError:
|
||||
return None
|
||||
except socket.error as why:
|
||||
if why.args[0] in (EWOULDBLOCK, WSAEWOULDBLOCK, ECONNABORTED, EAGAIN, ENOTCONN):
|
||||
if why.args[0] in (
|
||||
EWOULDBLOCK, WSAEWOULDBLOCK, ECONNABORTED,
|
||||
EAGAIN, ENOTCONN):
|
||||
return None
|
||||
else:
|
||||
raise
|
||||
|
|
@ -771,6 +731,7 @@ class dispatcher(object):
|
|||
except AttributeError:
|
||||
raise AttributeError("{} instance has no attribute {}"
|
||||
.format(self.__class__.__name__, attr))
|
||||
|
||||
else:
|
||||
msg = "%(me)s.%(attr)s is deprecated; use %(me)s.socket.%(attr)s"\
|
||||
" instead" % {'me': self.__class__.__name__, 'attr': attr}
|
||||
|
|
@ -855,13 +816,8 @@ class dispatcher(object):
|
|||
|
||||
self.log_info(
|
||||
'uncaptured python exception, closing channel %s (%s:%s %s)' % (
|
||||
self_repr,
|
||||
t,
|
||||
v,
|
||||
tbinfo
|
||||
),
|
||||
'error'
|
||||
)
|
||||
self_repr, t, v, tbinfo),
|
||||
'error')
|
||||
self.handle_close()
|
||||
|
||||
def handle_accept(self):
|
||||
|
|
@ -902,11 +858,8 @@ class dispatcher_with_send(dispatcher):
|
|||
adds simple buffered output capability, useful for simple clients.
|
||||
[for more sophisticated usage use asynchat.async_chat]
|
||||
"""
|
||||
# pylint: disable=redefined-builtin
|
||||
|
||||
def __init__(self, sock=None, map=None):
|
||||
# pylint: disable=redefined-builtin
|
||||
|
||||
dispatcher.__init__(self, sock, map)
|
||||
self.out_buffer = b''
|
||||
|
||||
|
|
@ -941,7 +894,8 @@ def compact_traceback():
|
|||
"""Return a compact traceback"""
|
||||
t, v, tb = sys.exc_info()
|
||||
tbinfo = []
|
||||
if not tb: # Must have a traceback
|
||||
# Must have a traceback
|
||||
if not tb:
|
||||
raise AssertionError("traceback does not exist")
|
||||
while tb:
|
||||
tbinfo.append((
|
||||
|
|
@ -961,7 +915,6 @@ def compact_traceback():
|
|||
|
||||
def close_all(map=None, ignore_all=False):
|
||||
"""Close all connections"""
|
||||
# pylint: disable=redefined-builtin
|
||||
|
||||
if map is None:
|
||||
map = socket_map
|
||||
|
|
@ -998,13 +951,13 @@ def close_all(map=None, ignore_all=False):
|
|||
if os.name == 'posix':
|
||||
import fcntl
|
||||
|
||||
class file_wrapper:
|
||||
class file_wrapper: # pylint: disable=old-style-class
|
||||
"""
|
||||
Here we override just enough to make a file look like a socket for the purposes of asyncore.
|
||||
Here we override just enough to make a file look
|
||||
like a socket for the purposes of asyncore.
|
||||
|
||||
The passed fd is automatically os.dup()'d
|
||||
"""
|
||||
# pylint: disable=old-style-class
|
||||
|
||||
def __init__(self, fd):
|
||||
self.fd = os.dup(fd)
|
||||
|
|
@ -1019,12 +972,11 @@ if os.name == 'posix':
|
|||
|
||||
def getsockopt(self, level, optname, buflen=None):
|
||||
"""Fake getsockopt()"""
|
||||
if (level == socket.SOL_SOCKET and
|
||||
optname == socket.SO_ERROR and
|
||||
if (level == socket.SOL_SOCKET and optname == socket.SO_ERROR and
|
||||
not buflen):
|
||||
return 0
|
||||
raise NotImplementedError("Only asyncore specific behaviour "
|
||||
"implemented.")
|
||||
raise NotImplementedError(
|
||||
"Only asyncore specific behaviour implemented.")
|
||||
|
||||
read = recv
|
||||
write = send
|
||||
|
|
@ -1041,8 +993,6 @@ if os.name == 'posix':
|
|||
"""A dispatcher for file_wrapper objects"""
|
||||
|
||||
def __init__(self, fd, map=None):
|
||||
# pylint: disable=redefined-builtin
|
||||
|
||||
dispatcher.__init__(self, None, map)
|
||||
self.connected = True
|
||||
try:
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
"""
|
||||
BMObject and it's exceptions.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import time
|
||||
|
||||
|
|
@ -15,12 +14,14 @@ logger = logging.getLogger('default')
|
|||
|
||||
|
||||
class BMObjectInsufficientPOWError(Exception):
|
||||
"""Exception indicating the object doesn't have sufficient proof of work."""
|
||||
"""Exception indicating the object
|
||||
doesn't have sufficient proof of work."""
|
||||
errorCodes = ("Insufficient proof of work")
|
||||
|
||||
|
||||
class BMObjectInvalidDataError(Exception):
|
||||
"""Exception indicating the data being parsed does not match the specification."""
|
||||
"""Exception indicating the data being parsed
|
||||
does not match the specification."""
|
||||
errorCodes = ("Data invalid")
|
||||
|
||||
|
||||
|
|
@ -30,7 +31,8 @@ class BMObjectExpiredError(Exception):
|
|||
|
||||
|
||||
class BMObjectUnwantedStreamError(Exception):
|
||||
"""Exception indicating the object is in a stream we didn't advertise as being interested in."""
|
||||
"""Exception indicating the object is in a stream
|
||||
we didn't advertise as being interested in."""
|
||||
errorCodes = ("Object in unwanted stream")
|
||||
|
||||
|
||||
|
|
@ -44,9 +46,8 @@ class BMObjectAlreadyHaveError(Exception):
|
|||
errorCodes = ("Already have this object")
|
||||
|
||||
|
||||
class BMObject(object):
|
||||
class BMObject(object): # pylint: disable=too-many-instance-attributes
|
||||
"""Bitmessage Object as a class."""
|
||||
# pylint: disable=too-many-instance-attributes
|
||||
|
||||
# max TTL, 28 days and 3 hours
|
||||
maxTTL = 28 * 24 * 60 * 60 + 10800
|
||||
|
|
@ -81,31 +82,36 @@ class BMObject(object):
|
|||
raise BMObjectInsufficientPOWError()
|
||||
|
||||
def checkEOLSanity(self):
|
||||
"""Check if object's lifetime isn't ridiculously far in the past or future."""
|
||||
"""Check if object's lifetime
|
||||
isn't ridiculously far in the past or future."""
|
||||
# EOL sanity check
|
||||
if self.expiresTime - int(time.time()) > BMObject.maxTTL:
|
||||
logger.info(
|
||||
'This object\'s End of Life time is too far in the future. Ignoring it. Time is %i',
|
||||
self.expiresTime)
|
||||
'This object\'s End of Life time is too far in the future.'
|
||||
' Ignoring it. Time is %i', self.expiresTime)
|
||||
# .. todo:: remove from download queue
|
||||
raise BMObjectExpiredError()
|
||||
|
||||
if self.expiresTime - int(time.time()) < BMObject.minTTL:
|
||||
logger.info(
|
||||
'This object\'s End of Life time was too long ago. Ignoring the object. Time is %i',
|
||||
self.expiresTime)
|
||||
'This object\'s End of Life time was too long ago.'
|
||||
' Ignoring the object. Time is %i', self.expiresTime)
|
||||
# .. todo:: remove from download queue
|
||||
raise BMObjectExpiredError()
|
||||
|
||||
def checkStream(self):
|
||||
"""Check if object's stream matches streams we are interested in"""
|
||||
if self.streamNumber not in state.streamsInWhichIAmParticipating:
|
||||
logger.debug('The streamNumber %i isn\'t one we are interested in.', self.streamNumber)
|
||||
logger.debug(
|
||||
'The streamNumber %i isn\'t one we are interested in.',
|
||||
self.streamNumber)
|
||||
raise BMObjectUnwantedStreamError()
|
||||
|
||||
def checkAlreadyHave(self):
|
||||
"""
|
||||
Check if we already have the object (so that we don't duplicate it in inventory or advertise it unnecessarily)
|
||||
Check if we already have the object
|
||||
(so that we don't duplicate it in inventory
|
||||
or advertise it unnecessarily)
|
||||
"""
|
||||
# if it's a stem duplicate, pretend we don't have it
|
||||
if Dandelion().hasHash(self.inventoryHash):
|
||||
|
|
@ -114,7 +120,8 @@ class BMObject(object):
|
|||
raise BMObjectAlreadyHaveError()
|
||||
|
||||
def checkObjectByType(self):
|
||||
"""Call a object type specific check (objects can have additional checks based on their types)"""
|
||||
"""Call a object type specific check
|
||||
(objects can have additional checks based on their types)"""
|
||||
if self.objectType == protocol.OBJECT_GETPUBKEY:
|
||||
self.checkGetpubkey()
|
||||
elif self.objectType == protocol.OBJECT_PUBKEY:
|
||||
|
|
@ -125,20 +132,21 @@ class BMObject(object):
|
|||
self.checkBroadcast()
|
||||
# other objects don't require other types of tests
|
||||
|
||||
def checkMessage(self):
|
||||
def checkMessage(self): # pylint: disable=no-self-use
|
||||
""""Message" object type checks."""
|
||||
# pylint: disable=no-self-use
|
||||
return
|
||||
|
||||
def checkGetpubkey(self):
|
||||
""""Getpubkey" object type checks."""
|
||||
if len(self.data) < 42:
|
||||
logger.info('getpubkey message doesn\'t contain enough data. Ignoring.')
|
||||
logger.info(
|
||||
'getpubkey message doesn\'t contain enough data. Ignoring.')
|
||||
raise BMObjectInvalidError()
|
||||
|
||||
def checkPubkey(self):
|
||||
""""Pubkey" object type checks."""
|
||||
if len(self.data) < 146 or len(self.data) > 440: # sanity check
|
||||
# sanity check
|
||||
if len(self.data) < 146 or len(self.data) > 440:
|
||||
logger.info('pubkey object too short or too long. Ignoring.')
|
||||
raise BMObjectInvalidError()
|
||||
|
||||
|
|
@ -146,8 +154,9 @@ class BMObject(object):
|
|||
""""Broadcast" object type checks."""
|
||||
if len(self.data) < 180:
|
||||
logger.debug(
|
||||
'The payload length of this broadcast packet is unreasonably low.'
|
||||
' Someone is probably trying funny business. Ignoring message.')
|
||||
'The payload length of this broadcast'
|
||||
' packet is unreasonably low. Someone is probably'
|
||||
' trying funny business. Ignoring message.')
|
||||
raise BMObjectInvalidError()
|
||||
|
||||
# this isn't supported anymore
|
||||
|
|
|
|||
|
|
@ -1,8 +1,7 @@
|
|||
"""
|
||||
src/network/bmproto.py
|
||||
==================================
|
||||
Bitmessage Protocol
|
||||
"""
|
||||
# pylint: disable=attribute-defined-outside-init
|
||||
# pylint: disable=attribute-defined-outside-init, too-few-public-methods
|
||||
import base64
|
||||
import hashlib
|
||||
import logging
|
||||
|
|
@ -19,17 +18,16 @@ import state
|
|||
from bmconfigparser import BMConfigParser
|
||||
from inventory import Inventory
|
||||
from network.advanceddispatcher import AdvancedDispatcher
|
||||
from network.constants import (
|
||||
ADDRESS_ALIVE,
|
||||
MAX_MESSAGE_SIZE,
|
||||
MAX_OBJECT_COUNT,
|
||||
MAX_OBJECT_PAYLOAD_SIZE,
|
||||
MAX_TIME_OFFSET)
|
||||
from network.dandelion import Dandelion
|
||||
from network.bmobject import (
|
||||
BMObject, BMObjectInsufficientPOWError, BMObjectInvalidDataError,
|
||||
BMObjectExpiredError, BMObjectUnwantedStreamError,
|
||||
BMObjectInvalidError, BMObjectAlreadyHaveError)
|
||||
BMObjectInvalidError, BMObjectAlreadyHaveError
|
||||
)
|
||||
from network.constants import (
|
||||
ADDRESS_ALIVE, MAX_MESSAGE_SIZE, MAX_OBJECT_COUNT,
|
||||
MAX_OBJECT_PAYLOAD_SIZE, MAX_TIME_OFFSET
|
||||
)
|
||||
from network.dandelion import Dandelion
|
||||
from network.proxy import ProxyError
|
||||
from network.objectracker import missingObjects, ObjectTracker
|
||||
from network.node import Node, Peer
|
||||
|
|
@ -73,7 +71,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
|
|||
# pylint: disable=too-many-instance-attributes, too-many-public-methods
|
||||
timeOffsetWrongCount = 0
|
||||
|
||||
def __init__(self, address=None, sock=None): # pylint: disable=unused-argument, super-init-not-called
|
||||
def __init__(self, address=None, sock=None):
|
||||
# pylint: disable=unused-argument, super-init-not-called
|
||||
AdvancedDispatcher.__init__(self, sock)
|
||||
self.isOutbound = False
|
||||
# packet/connection from a local IP
|
||||
|
|
@ -190,7 +189,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
|
|||
|
||||
def decode_payload_varint(self):
|
||||
"""Decode a varint from the payload"""
|
||||
value, offset = addresses.decodeVarint(self.payload[self.payloadOffset:])
|
||||
value, offset = addresses.decodeVarint(
|
||||
self.payload[self.payloadOffset:])
|
||||
self.payloadOffset += offset
|
||||
return value
|
||||
|
||||
|
|
@ -212,8 +212,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
|
|||
|
||||
return Node(services, host, port)
|
||||
|
||||
def decode_payload_content(self, pattern="v"): # pylint: disable=too-many-branches, too-many-statements
|
||||
|
||||
# pylint: disable=too-many-branches, too-many-statements
|
||||
def decode_payload_content(self, pattern="v"):
|
||||
"""
|
||||
Decode the payload depending on pattern:
|
||||
|
||||
|
|
@ -229,7 +229,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
|
|||
, = end of array
|
||||
"""
|
||||
|
||||
def decode_simple(self, char="v"): # pylint: disable=inconsistent-return-statements
|
||||
# pylint: disable=inconsistent-return-statements
|
||||
def decode_simple(self, char="v"):
|
||||
"""Decode the payload using one char pattern"""
|
||||
if char == "v":
|
||||
return self.decode_payload_varint()
|
||||
|
|
@ -442,6 +443,7 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
|
|||
if self.object.inventoryHash in Inventory() and Dandelion().hasHash(self.object.inventoryHash):
|
||||
Dandelion().removeHash(self.object.inventoryHash, "cycle detection")
|
||||
[self.object.inventoryHash] = (
|
||||
|
||||
self.object.objectType, self.object.streamNumber,
|
||||
memoryview(self.payload[objectOffset:]), self.object.expiresTime,
|
||||
memoryview(self.object.tag)
|
||||
|
|
@ -463,6 +465,7 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
|
|||
for i in addresses:
|
||||
seenTime, stream, _, ip, port = i
|
||||
decodedIP = protocol.checkIPAddress(bytes(ip))
|
||||
|
||||
if stream not in state.streamsInWhichIAmParticipating:
|
||||
continue
|
||||
if (
|
||||
|
|
@ -472,14 +475,16 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
|
|||
):
|
||||
peer = Peer(decodedIP, port)
|
||||
try:
|
||||
if knownnodes.knownNodes[stream][peer]["lastseen"] > seenTime:
|
||||
if knownnodes.knownNodes[stream][peer]["lastseen"] > \
|
||||
seenTime:
|
||||
continue
|
||||
except KeyError:
|
||||
pass
|
||||
if len(knownnodes.knownNodes[stream]) < int(BMConfigParser().safeGet("knownnodes", "maxnodes")):
|
||||
with knownnodes.knownNodesLock:
|
||||
try:
|
||||
knownnodes.knownNodes[stream][peer]["lastseen"] = seenTime
|
||||
knownnodes.knownNodes[stream][peer]["lastseen"] = \
|
||||
seenTime
|
||||
except (TypeError, KeyError):
|
||||
knownnodes.knownNodes[stream][peer] = {
|
||||
"lastseen": seenTime,
|
||||
|
|
@ -573,7 +578,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
|
|||
length=self.payloadLength, expectBytes=0)
|
||||
return False
|
||||
|
||||
def peerValidityChecks(self): # pylint: disable=too-many-return-statements
|
||||
# pylint: disable=too-many-return-statements
|
||||
def peerValidityChecks(self):
|
||||
"""Check the validity of the peer"""
|
||||
if self.remoteProtocolVersion < 3:
|
||||
self.append_write_buf(protocol.assembleErrorMessage(
|
||||
|
|
@ -585,8 +591,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
|
|||
return False
|
||||
if self.timeOffset > MAX_TIME_OFFSET:
|
||||
self.append_write_buf(protocol.assembleErrorMessage(
|
||||
errorText="Your time is too far in the future compared to mine."
|
||||
" Closing connection.", fatal=2))
|
||||
errorText="Your time is too far in the future"
|
||||
" compared to mine. Closing connection.", fatal=2))
|
||||
logger.info(
|
||||
"%s's time is too far in the future (%s seconds)."
|
||||
" Closing connection to it.", self.destination, self.timeOffset)
|
||||
|
|
@ -608,8 +614,8 @@ class BMProto(AdvancedDispatcher, ObjectTracker):
|
|||
errorText="We don't have shared stream interests."
|
||||
" Closing connection.", fatal=2))
|
||||
logger.debug(
|
||||
'Closed connection to %s because there is no overlapping interest'
|
||||
' in streams.', self.destination)
|
||||
'Closed connection to %s because there is no overlapping'
|
||||
' interest in streams.', self.destination)
|
||||
return False
|
||||
if self.destination in connectionpool.BMConnectionPool().inboundConnections:
|
||||
try:
|
||||
|
|
|
|||
|
|
@ -1,3 +1,6 @@
|
|||
"""
|
||||
Select which node to connect to
|
||||
"""
|
||||
# pylint: disable=too-many-branches
|
||||
import logging
|
||||
import random # nosec
|
||||
|
|
@ -12,6 +15,7 @@ logger = logging.getLogger('default')
|
|||
|
||||
|
||||
def getDiscoveredPeer():
|
||||
"""Get a peer from the local peer discovery list"""
|
||||
try:
|
||||
peer = random.choice([key for key in state.discoveredPeers.keys()])
|
||||
except (IndexError, KeyError):
|
||||
|
|
@ -24,6 +28,7 @@ def getDiscoveredPeer():
|
|||
|
||||
|
||||
def chooseConnection(stream):
|
||||
"""Returns an appropriate connection"""
|
||||
haveOnion = BMConfigParser().safeGet(
|
||||
"bitmessagesettings", "socksproxytype")[0:5] == 'SOCKS'
|
||||
onionOnly = BMConfigParser().safeGetBoolean(
|
||||
|
|
@ -49,7 +54,8 @@ def chooseConnection(stream):
|
|||
logger.warning('Error in {}'.format(peer))
|
||||
rating = 0
|
||||
if haveOnion:
|
||||
# do not connect to raw IP addresses--keep all traffic within Tor overlay
|
||||
# do not connect to raw IP addresses
|
||||
# --keep all traffic within Tor overlay
|
||||
if onionOnly and not peer.host.endswith('.onion'):
|
||||
continue
|
||||
# onion addresses have a higher priority when SOCKS
|
||||
|
|
|
|||
|
|
@ -3,9 +3,15 @@ Network protocol constants
|
|||
"""
|
||||
|
||||
|
||||
ADDRESS_ALIVE = 10800 #: address is online if online less than this many seconds ago
|
||||
MAX_ADDR_COUNT = 1000 #: protocol specification says max 1000 addresses in one addr command
|
||||
MAX_MESSAGE_SIZE = 1600100 #: ~1.6 MB which is the maximum possible size of an inv message.
|
||||
MAX_OBJECT_PAYLOAD_SIZE = 2**18 #: 2**18 = 256kB is the maximum size of an object payload
|
||||
MAX_OBJECT_COUNT = 50000 #: protocol specification says max 50000 objects in one inv command
|
||||
MAX_TIME_OFFSET = 3600 #: maximum time offset
|
||||
#: address is online if online less than this many seconds ago
|
||||
ADDRESS_ALIVE = 10800
|
||||
#: protocol specification says max 1000 addresses in one addr command
|
||||
MAX_ADDR_COUNT = 1000
|
||||
#: ~1.6 MB which is the maximum possible size of an inv message.
|
||||
MAX_MESSAGE_SIZE = 1600100
|
||||
#: 2**18 = 256kB is the maximum size of an object payload
|
||||
MAX_OBJECT_PAYLOAD_SIZE = 2**18
|
||||
#: protocol specification says max 50000 objects in one inv command
|
||||
MAX_OBJECT_COUNT = 50000
|
||||
#: maximum time offset
|
||||
MAX_TIME_OFFSET = 3600
|
||||
|
|
|
|||
|
|
@ -1,10 +1,9 @@
|
|||
"""
|
||||
src/network/dandelion.py
|
||||
========================
|
||||
Dandelion class definition, tracks stages
|
||||
"""
|
||||
import logging
|
||||
from collections import namedtuple
|
||||
from random import choice, sample, expovariate
|
||||
from random import choice, expovariate, sample
|
||||
from threading import RLock
|
||||
from time import time
|
||||
|
||||
|
|
@ -131,6 +130,8 @@ class Dandelion(object):
|
|||
|
||||
for k in (
|
||||
k for k, v in iter(self.nodeMap.items()) if v == connection
|
||||
# k for k, v in self.nodeMap.iteritems()
|
||||
# if v == connection
|
||||
):
|
||||
self.nodeMap[k] = None
|
||||
for k, v in {
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
"""
|
||||
`DownloadThread` class definition
|
||||
"""
|
||||
|
||||
import time
|
||||
|
||||
import addresses
|
||||
|
|
@ -31,6 +30,9 @@ class DownloadThread(StoppableThread):
|
|||
deadline = time.time() - self.requestExpires
|
||||
try:
|
||||
toDelete = [k for k, v in iter(missingObjects.items()) if v < deadline]
|
||||
# toDelete = [
|
||||
# k for k, v in missingObjects.iteritems()
|
||||
# if v < deadline]
|
||||
except RuntimeError:
|
||||
pass
|
||||
else:
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
"""
|
||||
src/network/invthread.py
|
||||
========================
|
||||
Thread to send inv annoucements
|
||||
"""
|
||||
import queue as Queue
|
||||
import random
|
||||
|
|
@ -34,7 +33,7 @@ def handleExpiredDandelion(expired):
|
|||
|
||||
|
||||
class InvThread(StoppableThread):
|
||||
"""A thread to send inv annoucements."""
|
||||
"""Main thread that sends inv annoucements"""
|
||||
|
||||
name = "InvBroadcaster"
|
||||
|
||||
|
|
@ -43,7 +42,8 @@ class InvThread(StoppableThread):
|
|||
"""Locally generated inventory items require special handling"""
|
||||
Dandelion().addHash(hashId, stream=stream)
|
||||
for connection in BMConnectionPool().connections():
|
||||
if state.dandelion and connection != Dandelion().objectChildStem(hashId):
|
||||
if state.dandelion and connection != \
|
||||
Dandelion().objectChildStem(hashId):
|
||||
continue
|
||||
connection.objectsNewToThem[hashId] = time()
|
||||
|
||||
|
|
@ -92,15 +92,17 @@ class InvThread(StoppableThread):
|
|||
random.shuffle(fluffs)
|
||||
connection.append_write_buf(protocol.CreatePacket(
|
||||
'inv',
|
||||
addresses.encodeVarint(len(fluffs)) + ''.join(fluffs)))
|
||||
addresses.encodeVarint(
|
||||
len(fluffs)) + ''.join(fluffs)))
|
||||
if stems:
|
||||
random.shuffle(stems)
|
||||
connection.append_write_buf(protocol.CreatePacket(
|
||||
'dinv',
|
||||
addresses.encodeVarint(len(stems)) + ''.join(stems)))
|
||||
addresses.encodeVarint(
|
||||
len(stems)) + ''.join(stems)))
|
||||
|
||||
invQueue.iterate()
|
||||
for i in range(len(chunk)):
|
||||
for _ in range(len(chunk)):
|
||||
invQueue.task_done()
|
||||
|
||||
if Dandelion().refresh < time():
|
||||
|
|
|
|||
|
|
@ -1,3 +1,6 @@
|
|||
"""
|
||||
A thread to handle network concerns
|
||||
"""
|
||||
import network.asyncore_pollchoose as asyncore
|
||||
import state
|
||||
from network.connectionpool import BMConnectionPool
|
||||
|
|
@ -6,7 +9,7 @@ from network.threads import StoppableThread
|
|||
|
||||
|
||||
class BMNetworkThread(StoppableThread):
|
||||
"""A thread to handle network concerns"""
|
||||
"""Main network thread"""
|
||||
name = "Asyncore"
|
||||
|
||||
def run(self):
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
"""
|
||||
src/network/objectracker.py
|
||||
===========================
|
||||
Module for tracking objects
|
||||
"""
|
||||
import time
|
||||
from threading import RLock
|
||||
|
|
@ -50,14 +49,17 @@ class ObjectTracker(object):
|
|||
"""Init bloom filter for tracking. WIP."""
|
||||
if haveBloom:
|
||||
# lock?
|
||||
self.invBloom = BloomFilter(capacity=ObjectTracker.invInitialCapacity,
|
||||
self.invBloom = BloomFilter(
|
||||
capacity=ObjectTracker.invInitialCapacity,
|
||||
error_rate=ObjectTracker.invErrorRate)
|
||||
|
||||
def initAddrBloom(self):
|
||||
"""Init bloom filter for tracking addrs, WIP. This either needs to be moved to addrthread.py or removed."""
|
||||
"""Init bloom filter for tracking addrs, WIP.
|
||||
This either needs to be moved to addrthread.py or removed."""
|
||||
if haveBloom:
|
||||
# lock?
|
||||
self.addrBloom = BloomFilter(capacity=ObjectTracker.invInitialCapacity,
|
||||
self.addrBloom = BloomFilter(
|
||||
capacity=ObjectTracker.invInitialCapacity,
|
||||
error_rate=ObjectTracker.invErrorRate)
|
||||
|
||||
def clean(self):
|
||||
|
|
@ -72,6 +74,10 @@ class ObjectTracker(object):
|
|||
deadline = time.time() - ObjectTracker.trackingExpires
|
||||
with self.objectsNewToThemLock:
|
||||
self.objectsNewToThem = {k: v for k, v in iter(self.objectsNewToThem.items()) if v >= deadline}
|
||||
# self.objectsNewToThem = {
|
||||
# k: v
|
||||
# for k, v in self.objectsNewToThem.iteritems()
|
||||
# if v >= deadline}
|
||||
self.lastCleaned = time.time()
|
||||
|
||||
def hasObj(self, hashid):
|
||||
|
|
@ -102,10 +108,12 @@ class ObjectTracker(object):
|
|||
del i.objectsNewToMe[hashid]
|
||||
except KeyError:
|
||||
if streamNumber in i.streams and (
|
||||
not Dandelion().hasHash(hashid) or Dandelion().objectChildStem(hashid) == i):
|
||||
not Dandelion().hasHash(hashid) or
|
||||
Dandelion().objectChildStem(hashid) == i):
|
||||
with i.objectsNewToThemLock:
|
||||
i.objectsNewToThem[hashid] = time.time()
|
||||
# update stream number, which we didn't have when we just received the dinv
|
||||
# update stream number,
|
||||
# which we didn't have when we just received the dinv
|
||||
# also resets expiration of the stem mode
|
||||
Dandelion().setHashStream(hashid, streamNumber)
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
"""
|
||||
src/network/proxy.py
|
||||
====================
|
||||
Set proxy if avaiable otherwise exception
|
||||
"""
|
||||
# pylint: disable=protected-access
|
||||
import logging
|
||||
|
|
@ -123,8 +122,7 @@ class Proxy(AdvancedDispatcher):
|
|||
BMConfigParser().safeGet(
|
||||
"bitmessagesettings", "socksusername"),
|
||||
BMConfigParser().safeGet(
|
||||
"bitmessagesettings", "sockspassword")
|
||||
)
|
||||
"bitmessagesettings", "sockspassword"))
|
||||
else:
|
||||
self.auth = None
|
||||
self.connect(
|
||||
|
|
|
|||
|
|
@ -1,8 +1,6 @@
|
|||
"""
|
||||
src/randomtrackingdict.py
|
||||
=========================
|
||||
Track randomize ordered dict
|
||||
"""
|
||||
|
||||
import random
|
||||
from threading import RLock
|
||||
from time import time
|
||||
|
|
@ -14,12 +12,12 @@ class RandomTrackingDict(object):
|
|||
"""
|
||||
Dict with randomised order and tracking.
|
||||
|
||||
Keeps a track of how many items have been requested from the dict, and timeouts.
|
||||
Resets after all objects have been retrieved and timed out.
|
||||
Keeps a track of how many items have been requested from the dict,
|
||||
and timeouts. Resets after all objects have been retrieved and timed out.
|
||||
The main purpose of this isn't as much putting related code together
|
||||
as performance optimisation and anonymisation of downloading of objects from other peers.
|
||||
If done using a standard dict or array, it takes too much CPU (and looks convoluted).
|
||||
Randomisation helps with anonymity.
|
||||
as performance optimisation and anonymisation of downloading of objects
|
||||
from other peers. If done using a standard dict or array, it takes
|
||||
too much CPU (and looks convoluted). Randomisation helps with anonymity.
|
||||
"""
|
||||
# pylint: disable=too-many-instance-attributes
|
||||
maxPending = 10
|
||||
|
|
@ -87,13 +85,14 @@ class RandomTrackingDict(object):
|
|||
|
||||
def setMaxPending(self, maxPending):
|
||||
"""
|
||||
Sets maximum number of objects that can be retrieved from the class simultaneously as long as there is no
|
||||
timeout
|
||||
Sets maximum number of objects that can be retrieved from the class
|
||||
simultaneously as long as there is no timeout
|
||||
"""
|
||||
self.maxPending = maxPending
|
||||
|
||||
def setPendingTimeout(self, pendingTimeout):
|
||||
"""Sets how long to wait for a timeout if max pending is reached (or all objects have been retrieved)"""
|
||||
"""Sets how long to wait for a timeout if max pending is reached
|
||||
(or all objects have been retrieved)"""
|
||||
self.pendingTimeout = pendingTimeout
|
||||
|
||||
def setLastObject(self):
|
||||
|
|
@ -101,7 +100,8 @@ class RandomTrackingDict(object):
|
|||
self.lastObject = time()
|
||||
|
||||
def randomKeys(self, count=1):
|
||||
"""Retrieve count random keys from the dict that haven't already been retrieved"""
|
||||
"""Retrieve count random keys from the dict
|
||||
that haven't already been retrieved"""
|
||||
if self.len == 0 or ((self.pendingLen >= self.maxPending or
|
||||
self.pendingLen == self.len) and self.lastPoll +
|
||||
self.pendingTimeout > time()):
|
||||
|
|
@ -111,13 +111,15 @@ class RandomTrackingDict(object):
|
|||
with self.lock:
|
||||
# reset if we've requested all
|
||||
# and if last object received too long time ago
|
||||
if self.pendingLen == self.len and self.lastObject + self.pendingTimeout < time():
|
||||
if self.pendingLen == self.len and self.lastObject + \
|
||||
self.pendingTimeout < time():
|
||||
self.pendingLen = 0
|
||||
self.setLastObject()
|
||||
available = self.len - self.pendingLen
|
||||
if count > available:
|
||||
count = available
|
||||
randomIndex = helper_random.randomsample(range(self.len - self.pendingLen), count)
|
||||
randomIndex = helper_random.randomsample(
|
||||
range(self.len - self.pendingLen), count)
|
||||
retval = [self.indexDict[i] for i in randomIndex]
|
||||
|
||||
for i in sorted(randomIndex, reverse=True):
|
||||
|
|
|
|||
|
|
@ -1,3 +1,6 @@
|
|||
"""
|
||||
Process data incoming from network
|
||||
"""
|
||||
import errno
|
||||
import queue as Queue
|
||||
import socket
|
||||
|
|
@ -10,6 +13,8 @@ from network.threads import StoppableThread
|
|||
|
||||
|
||||
class ReceiveQueueThread(StoppableThread):
|
||||
"""This thread processes data received from the network
|
||||
(which is done by the asyncore thread)"""
|
||||
def __init__(self, num=0):
|
||||
super(ReceiveQueueThread, self).__init__(name="ReceiveQueue_%i" % num)
|
||||
|
||||
|
|
@ -32,12 +37,14 @@ class ReceiveQueueThread(StoppableThread):
|
|||
|
||||
try:
|
||||
connection = BMConnectionPool().getConnectionByAddr(dest)
|
||||
except KeyError: # connection object not found
|
||||
# connection object not found
|
||||
except KeyError:
|
||||
receiveDataQueue.task_done()
|
||||
continue
|
||||
try:
|
||||
connection.process()
|
||||
except UnknownStateError: # state isn't implemented
|
||||
# state isn't implemented
|
||||
except UnknownStateError:
|
||||
pass
|
||||
except socket.error as err:
|
||||
if err.errno == errno.EBADF:
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
"""
|
||||
src/network/socks4a.py
|
||||
=================================
|
||||
SOCKS4a proxy module
|
||||
"""
|
||||
# pylint: disable=attribute-defined-outside-init
|
||||
import socket
|
||||
|
|
|
|||
|
|
@ -1,7 +1,5 @@
|
|||
"""
|
||||
src/network/socks5.py
|
||||
=====================
|
||||
|
||||
SOCKS5 proxy module
|
||||
"""
|
||||
# pylint: disable=attribute-defined-outside-init
|
||||
|
||||
|
|
@ -157,7 +155,8 @@ class Socks5(Proxy):
|
|||
return True
|
||||
|
||||
def proxy_sock_name(self):
|
||||
"""Handle return value when using SOCKS5 for DNS resolving instead of connecting."""
|
||||
"""Handle return value when using SOCKS5
|
||||
for DNS resolving instead of connecting."""
|
||||
return socket.inet_ntoa(self.__proxysockname[0])
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
"""
|
||||
src/network/stats.py
|
||||
====================
|
||||
Network statistics
|
||||
"""
|
||||
import time
|
||||
|
||||
|
|
@ -34,7 +33,9 @@ def uploadSpeed():
|
|||
currentTimestamp = time.time()
|
||||
if int(lastSentTimestamp) < int(currentTimestamp):
|
||||
currentSentBytes = asyncore.sentBytes
|
||||
currentSentSpeed = int((currentSentBytes - lastSentBytes) / (currentTimestamp - lastSentTimestamp))
|
||||
currentSentSpeed = int(
|
||||
(currentSentBytes - lastSentBytes) / (
|
||||
currentTimestamp - lastSentTimestamp))
|
||||
lastSentBytes = currentSentBytes
|
||||
lastSentTimestamp = currentTimestamp
|
||||
return currentSentSpeed
|
||||
|
|
@ -53,7 +54,8 @@ def downloadSpeed():
|
|||
if int(lastReceivedTimestamp) < int(currentTimestamp):
|
||||
currentReceivedBytes = asyncore.receivedBytes
|
||||
currentReceivedSpeed = int(
|
||||
(currentReceivedBytes - lastReceivedBytes) / (currentTimestamp - lastReceivedTimestamp))
|
||||
(currentReceivedBytes - lastReceivedBytes) / (
|
||||
currentTimestamp - lastReceivedTimestamp))
|
||||
lastReceivedBytes = currentReceivedBytes
|
||||
lastReceivedTimestamp = currentTimestamp
|
||||
return currentReceivedSpeed
|
||||
|
|
|
|||
|
|
@ -1,9 +1,7 @@
|
|||
"""
|
||||
TCP protocol handler
|
||||
"""
|
||||
# pylint: disable=too-many-ancestors
|
||||
"""
|
||||
src/network/tcp.py
|
||||
==================
|
||||
"""
|
||||
|
||||
import logging
|
||||
import math
|
||||
import random
|
||||
|
|
@ -33,13 +31,13 @@ from network.tls import TLSDispatcher
|
|||
from .node import Peer
|
||||
from queues import UISignalQueue, invQueue, receiveDataQueue
|
||||
|
||||
|
||||
logger = logging.getLogger('default')
|
||||
|
||||
|
||||
class TCPConnection(BMProto, TLSDispatcher):
|
||||
# pylint: disable=too-many-instance-attributes
|
||||
"""
|
||||
|
||||
.. todo:: Look to understand and/or fix the non-parent-init-called
|
||||
"""
|
||||
|
||||
|
|
@ -90,7 +88,8 @@ class TCPConnection(BMProto, TLSDispatcher):
|
|||
not protocol.checkSocksIP(self.destination.host)
|
||||
)
|
||||
except socket.error:
|
||||
pass # it's probably a hostname
|
||||
# it's probably a hostname
|
||||
pass
|
||||
self.network_group = protocol.network_group(self.destination.host)
|
||||
ObjectTracker.__init__(self) # pylint: disable=non-parent-init-called
|
||||
self.bm_proto_reset()
|
||||
|
|
@ -142,12 +141,12 @@ class TCPConnection(BMProto, TLSDispatcher):
|
|||
|
||||
def set_connection_fully_established(self):
|
||||
"""Initiate inventory synchronisation."""
|
||||
if not self.isOutbound and not self.local:
|
||||
shared.clientHasReceivedIncomingConnections = True
|
||||
UISignalQueue.put(('setStatusIcon', 'green'))
|
||||
UISignalQueue.put((
|
||||
'updateNetworkStatusTab',
|
||||
(self.isOutbound, True, self.destination)
|
||||
))
|
||||
UISignalQueue.put(
|
||||
('updateNetworkStatusTab', (
|
||||
self.isOutbound, True, self.destination)))
|
||||
self.antiIntersectionDelay(True)
|
||||
self.fullyEstablished = True
|
||||
if self.isOutbound:
|
||||
|
|
@ -221,6 +220,7 @@ class TCPConnection(BMProto, TLSDispatcher):
|
|||
payload = bytes()
|
||||
# Now let us start appending all of these hashes together. They will be
|
||||
# sent out in a big inv message to our new peer.
|
||||
|
||||
for obj_hash, _ in bigInvList.items():
|
||||
payload += obj_hash
|
||||
objectCount += 1
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ import sys
|
|||
|
||||
from network.advanceddispatcher import AdvancedDispatcher
|
||||
import network.asyncore_pollchoose as asyncore
|
||||
|
||||
from queues import receiveDataQueue
|
||||
import paths
|
||||
|
||||
|
|
@ -28,7 +29,8 @@ if sys.version_info >= (2, 7, 13):
|
|||
except AttributeError:
|
||||
sslProtocolVersion = ssl.PROTOCOL_SSLv23
|
||||
elif sys.version_info >= (2, 7, 9):
|
||||
# this means any SSL/TLS. SSLv2 and 3 are excluded with an option after context is created
|
||||
# this means any SSL/TLS.
|
||||
# SSLv2 and 3 are excluded with an option after context is created
|
||||
sslProtocolVersion = ssl.PROTOCOL_SSLv23
|
||||
else:
|
||||
# this means TLSv1, there is no way to set "TLSv1 or higher" or
|
||||
|
|
@ -37,7 +39,8 @@ else:
|
|||
|
||||
|
||||
# ciphers
|
||||
if ssl.OPENSSL_VERSION_NUMBER >= 0x10100000 and not ssl.OPENSSL_VERSION.startswith("LibreSSL"):
|
||||
if ssl.OPENSSL_VERSION_NUMBER >= 0x10100000 and not \
|
||||
ssl.OPENSSL_VERSION.startswith("LibreSSL"):
|
||||
sslProtocolCiphers = "AECDH-AES256-SHA@SECLEVEL=0"
|
||||
else:
|
||||
sslProtocolCiphers = "AECDH-AES256-SHA"
|
||||
|
|
@ -45,19 +48,19 @@ else:
|
|||
|
||||
class TLSDispatcher(AdvancedDispatcher):
|
||||
"""TLS functionality for classes derived from AdvancedDispatcher"""
|
||||
# pylint: disable=too-many-instance-attributes
|
||||
# pylint: disable=too-many-arguments,super-init-not-called,unused-argument
|
||||
def __init__(
|
||||
self, address=None, sock=None, certfile=None, keyfile=None,
|
||||
server_side=False, ciphers=sslProtocolCiphers
|
||||
):
|
||||
# pylint: disable=too-many-instance-attributes, too-many-arguments
|
||||
# pylint: disable=super-init-not-called
|
||||
def __init__(self, _=None, sock=None, certfile=None, keyfile=None,
|
||||
server_side=False, ciphers=sslProtocolCiphers):
|
||||
self.want_read = self.want_write = True
|
||||
if certfile is None:
|
||||
self.certfile = os.path.join(paths.codePath(), 'sslkeys', 'cert.pem')
|
||||
self.certfile = os.path.join(
|
||||
paths.codePath(), 'sslkeys', 'cert.pem')
|
||||
else:
|
||||
self.certfile = certfile
|
||||
if keyfile is None:
|
||||
self.keyfile = os.path.join(paths.codePath(), 'sslkeys', 'key.pem')
|
||||
self.keyfile = os.path.join(
|
||||
paths.codePath(), 'sslkeys', 'key.pem')
|
||||
else:
|
||||
self.keyfile = keyfile
|
||||
self.server_side = server_side
|
||||
|
|
@ -72,20 +75,23 @@ class TLSDispatcher(AdvancedDispatcher):
|
|||
# pylint: disable=attribute-defined-outside-init
|
||||
self.isSSL = True
|
||||
self.tlsStarted = True
|
||||
# Once the connection has been established, it's safe to wrap the
|
||||
# socket.
|
||||
# Once the connection has been established,
|
||||
# it's safe to wrap the socket.
|
||||
if sys.version_info >= (2, 7, 9):
|
||||
context = ssl.create_default_context(
|
||||
purpose=ssl.Purpose.SERVER_AUTH if self.server_side else ssl.Purpose.CLIENT_AUTH)
|
||||
purpose=ssl.Purpose.SERVER_AUTH
|
||||
if self.server_side else ssl.Purpose.CLIENT_AUTH)
|
||||
context.set_ciphers(self.ciphers)
|
||||
context.set_ecdh_curve("secp256k1")
|
||||
context.check_hostname = False
|
||||
context.verify_mode = ssl.CERT_NONE
|
||||
# also exclude TLSv1 and TLSv1.1 in the future
|
||||
context.options = ssl.OP_ALL | ssl.OP_NO_SSLv2 |\
|
||||
ssl.OP_NO_SSLv3 | ssl.OP_SINGLE_ECDH_USE | ssl.OP_CIPHER_SERVER_PREFERENCE
|
||||
ssl.OP_NO_SSLv3 | ssl.OP_SINGLE_ECDH_USE |\
|
||||
ssl.OP_CIPHER_SERVER_PREFERENCE
|
||||
self.sslSocket = context.wrap_socket(
|
||||
self.socket, server_side=self.server_side, do_handshake_on_connect=False)
|
||||
self.socket, server_side=self.server_side,
|
||||
do_handshake_on_connect=False)
|
||||
else:
|
||||
self.sslSocket = ssl.wrap_socket(
|
||||
self.socket, server_side=self.server_side,
|
||||
|
|
@ -119,12 +125,15 @@ class TLSDispatcher(AdvancedDispatcher):
|
|||
def readable(self):
|
||||
"""Handle readable check for TLS-enabled sockets"""
|
||||
try:
|
||||
# during TLS handshake, and after flushing write buffer, return status of last handshake attempt
|
||||
# during TLS handshake, and after flushing write buffer,
|
||||
# return status of last handshake attempt
|
||||
if self.tlsStarted and not self.tlsDone and not self.write_buf:
|
||||
# print "tls readable, %r" % (self.want_read)
|
||||
return self.want_read
|
||||
# prior to TLS handshake, receiveDataThread should emulate synchronous behaviour
|
||||
elif not self.fullyEstablished and (self.expectBytes == 0 or not self.write_buf_empty()):
|
||||
# prior to TLS handshake,
|
||||
# receiveDataThread should emulate synchronous behaviour
|
||||
elif not self.fullyEstablished and (
|
||||
self.expectBytes == 0 or not self.write_buf_empty()):
|
||||
return False
|
||||
return AdvancedDispatcher.readable(self)
|
||||
except AttributeError:
|
||||
|
|
@ -139,10 +148,14 @@ class TLSDispatcher(AdvancedDispatcher):
|
|||
try:
|
||||
# wait for write buffer flush
|
||||
if self.tlsStarted and not self.tlsDone and not self.write_buf:
|
||||
# logger.debug("%s:%i TLS handshaking (read)", self.destination.host, self.destination.port)
|
||||
# logger.debug(
|
||||
# "%s:%i TLS handshaking (read)", self.destination.host,
|
||||
# self.destination.port)
|
||||
self.tls_handshake()
|
||||
else:
|
||||
# logger.debug("%s:%i Not TLS handshaking (read)", self.destination.host, self.destination.port)
|
||||
# logger.debug(
|
||||
# "%s:%i Not TLS handshaking (read)", self.destination.host,
|
||||
# self.destination.port)
|
||||
return AdvancedDispatcher.handle_read(self)
|
||||
except AttributeError:
|
||||
return AdvancedDispatcher.handle_read(self)
|
||||
|
|
@ -165,10 +178,14 @@ class TLSDispatcher(AdvancedDispatcher):
|
|||
try:
|
||||
# wait for write buffer flush
|
||||
if self.tlsStarted and not self.tlsDone and not self.write_buf:
|
||||
# logger.debug("%s:%i TLS handshaking (write)", self.destination.host, self.destination.port)
|
||||
# logger.debug(
|
||||
# "%s:%i TLS handshaking (write)", self.destination.host,
|
||||
# self.destination.port)
|
||||
self.tls_handshake()
|
||||
else:
|
||||
# logger.debug("%s:%i Not TLS handshaking (write)", self.destination.host, self.destination.port)
|
||||
# logger.debug(
|
||||
# "%s:%i Not TLS handshaking (write)", self.destination.host,
|
||||
# self.destination.port)
|
||||
return AdvancedDispatcher.handle_write(self)
|
||||
except AttributeError:
|
||||
return AdvancedDispatcher.handle_write(self)
|
||||
|
|
|
|||
|
|
@ -1,16 +1,16 @@
|
|||
"""
|
||||
src/network/udp.py
|
||||
==================
|
||||
UDP protocol handler
|
||||
"""
|
||||
import logging
|
||||
import time
|
||||
import socket
|
||||
import time
|
||||
|
||||
import state
|
||||
import protocol
|
||||
from network.bmproto import BMProto
|
||||
from network.objectracker import ObjectTracker
|
||||
from .node import Peer
|
||||
import state
|
||||
|
||||
from queues import receiveDataQueue
|
||||
|
||||
logger = logging.getLogger('default')
|
||||
|
|
@ -76,10 +76,9 @@ class UDPSocket(BMProto): # pylint: disable=too-many-instance-attributes
|
|||
# attacks from random IPs on the internet
|
||||
self.local = True
|
||||
remoteport = False
|
||||
for seenTime, stream, services, ip, port in addresses:
|
||||
# decodedIP = bool(protocol.checkIPAddress(ip))
|
||||
decodedIP = False
|
||||
|
||||
for seenTime, stream, _, ip, port in addresses:
|
||||
decodedIP = protocol.checkIPAddress(bytes(ip))
|
||||
if stream not in state.streamsInWhichIAmParticipating:
|
||||
continue
|
||||
if (seenTime < time.time() - self.maxTimeOffset
|
||||
|
|
@ -95,9 +94,8 @@ class UDPSocket(BMProto): # pylint: disable=too-many-instance-attributes
|
|||
"received peer discovery from {}:{} (port {}):",
|
||||
self.destination.host, self.destination.port, remoteport)
|
||||
if self.local:
|
||||
state.discoveredPeers[
|
||||
Peer(self.destination.host, remoteport)
|
||||
] = time.time()
|
||||
state.discoveredPeers[Peer(self.destination.host, remoteport)] = \
|
||||
time.time()
|
||||
return True
|
||||
|
||||
def bm_command_portcheck(self):
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
src/plugins/indicator_libmessaging.py
|
||||
=====================================
|
||||
Indicator plugin using libmessaging
|
||||
"""
|
||||
|
||||
import gi
|
||||
|
|
|
|||
|
|
@ -1,8 +1,5 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
src/plugins/menu_qrcode.py
|
||||
==========================
|
||||
|
||||
A menu plugin showing QR-Code for bitmessage address in modal dialog.
|
||||
"""
|
||||
|
||||
|
|
@ -18,7 +15,8 @@ from pybitmessage.tr import _translate
|
|||
class Image(qrcode.image.base.BaseImage): # pylint: disable=abstract-method
|
||||
"""Image output class for qrcode using QPainter"""
|
||||
|
||||
def __init__(self, border, width, box_size): # pylint: disable=super-init-not-called
|
||||
def __init__(self, border, width, box_size):
|
||||
# pylint: disable=super-init-not-called
|
||||
self.border = border
|
||||
self.width = width
|
||||
self.box_size = box_size
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
src/plugins/notification_notify2.py
|
||||
===================================
|
||||
Notification plugin using notify2
|
||||
"""
|
||||
|
||||
import gi
|
||||
|
|
@ -11,7 +10,7 @@ from gi.repository import Notify
|
|||
Notify.init('pybitmessage')
|
||||
|
||||
|
||||
def connect_plugin(title, subtitle, category, label, icon):
|
||||
def connect_plugin(title, subtitle, category, _, icon):
|
||||
"""Plugin for notify2"""
|
||||
if not icon:
|
||||
icon = 'mail-message-new' if category == 2 else 'pybitmessage'
|
||||
|
|
|
|||
|
|
@ -22,9 +22,8 @@ import stem.process
|
|||
import stem.version
|
||||
|
||||
|
||||
class DebugLogger(object):
|
||||
class DebugLogger(object): # pylint: disable=too-few-public-methods
|
||||
"""Safe logger wrapper for tor and plugin's logs"""
|
||||
# pylint: disable=too-few-public-methods
|
||||
def __init__(self):
|
||||
self._logger = logging.getLogger('default')
|
||||
self._levels = {
|
||||
|
|
@ -108,7 +107,8 @@ def connect_plugin(config): # pylint: disable=too-many-branches
|
|||
onionhostname = config.safeGet('bitmessagesettings', 'onionhostname')
|
||||
onionkey = config.safeGet(onionhostname, 'privsigningkey')
|
||||
if onionhostname and not onionkey:
|
||||
logwrite('The hidden service found in config ): %s' % onionhostname)
|
||||
logwrite('The hidden service found in config ): %s' %
|
||||
onionhostname)
|
||||
onionkeytype = config.safeGet(onionhostname, 'keytype')
|
||||
|
||||
response = controller.create_ephemeral_hidden_service(
|
||||
|
|
@ -124,7 +124,8 @@ def connect_plugin(config): # pylint: disable=too-many-branches
|
|||
|
||||
if not onionkey:
|
||||
logwrite('Started hidden service %s.onion' % response.service_id)
|
||||
# only save new service keys if onionhostname was not set previously
|
||||
# only save new service keys
|
||||
# if onionhostname was not set previously
|
||||
if not onionhostname:
|
||||
onionhostname = response.service_id + '.onion'
|
||||
config.set(
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
src/plugins/proxyconfig_stem.py
|
||||
===================================
|
||||
Sound theme plugin using pycanberra
|
||||
"""
|
||||
|
||||
from pybitmessage.bitmessageqt import sound
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
src/plugins/sound_gstreamer.py
|
||||
===================================
|
||||
Sound notification plugin using gstreamer
|
||||
"""
|
||||
import gi
|
||||
gi.require_version('Gst', '1.0')
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
src/plugins/sound_playfile.py
|
||||
===================================
|
||||
Sound notification plugin using external executable or winsound (on Windows)
|
||||
"""
|
||||
|
||||
try:
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
# pylint: disable=too-many-branches,too-many-statements,protected-access
|
||||
"""
|
||||
src/proofofwork.py
|
||||
==================
|
||||
Proof of work calculation
|
||||
"""
|
||||
|
||||
import ctypes
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
# pylint: disable=missing-docstring,too-many-function-args
|
||||
|
||||
"""
|
||||
Arithmetic Expressions
|
||||
"""
|
||||
import hashlib
|
||||
import re
|
||||
|
||||
|
|
@ -11,6 +12,7 @@ G = (Gx, Gy)
|
|||
|
||||
|
||||
def inv(a, n):
|
||||
"""Inversion"""
|
||||
lm, hm = 1, 0
|
||||
low, high = a % n, n
|
||||
while low > 1:
|
||||
|
|
@ -21,6 +23,7 @@ def inv(a, n):
|
|||
|
||||
|
||||
def get_code_string(base):
|
||||
"""Returns string according to base value"""
|
||||
if base == 2:
|
||||
return '01'
|
||||
elif base == 10:
|
||||
|
|
@ -38,6 +41,7 @@ def get_code_string(base):
|
|||
|
||||
|
||||
def encode(val, base, minlen=0):
|
||||
"""Returns the encoded string"""
|
||||
code_string = get_code_string(base)
|
||||
result = str.encode('') if type(code_string) is bytes else ''
|
||||
while val > 0:
|
||||
|
|
@ -48,6 +52,7 @@ def encode(val, base, minlen=0):
|
|||
return result
|
||||
|
||||
def decode(string, base):
|
||||
"""Returns the decoded string"""
|
||||
code_string = get_code_string(base)
|
||||
result = 0
|
||||
if base == 16:
|
||||
|
|
@ -60,10 +65,13 @@ def decode(string, base):
|
|||
|
||||
|
||||
def changebase(string, frm, to, minlen=0):
|
||||
"""Change base of the string"""
|
||||
return encode(decode(string, frm), to, minlen)
|
||||
|
||||
|
||||
def base10_add(a, b):
|
||||
"""Adding the numbers that are of base10"""
|
||||
# pylint: disable=too-many-function-args
|
||||
if a is None:
|
||||
return b[0], b[1]
|
||||
if b is None:
|
||||
|
|
@ -79,6 +87,7 @@ def base10_add(a, b):
|
|||
|
||||
|
||||
def base10_double(a):
|
||||
"""Double the numbers that are of base10"""
|
||||
if a is None:
|
||||
return None
|
||||
m = ((3 * a[0] * a[0] + A) * inv(2 * a[1], P)) % P
|
||||
|
|
@ -88,6 +97,7 @@ def base10_double(a):
|
|||
|
||||
|
||||
def base10_multiply(a, n):
|
||||
"""Multiply the numbers that are of base10"""
|
||||
if n == 0:
|
||||
return G
|
||||
if n == 1:
|
||||
|
|
@ -100,28 +110,35 @@ def base10_multiply(a, n):
|
|||
|
||||
|
||||
def hex_to_point(h):
|
||||
"""Converting hexadecimal to point value"""
|
||||
return (decode(h[2:66], 16), decode(h[66:], 16))
|
||||
|
||||
|
||||
def point_to_hex(p):
|
||||
"""Converting point value to hexadecimal"""
|
||||
return '04' + encode(p[0], 16, 64) + encode(p[1], 16, 64)
|
||||
|
||||
|
||||
def multiply(privkey, pubkey):
|
||||
return point_to_hex(base10_multiply(hex_to_point(pubkey), decode(privkey, 16)))
|
||||
"""Multiplying keys"""
|
||||
return point_to_hex(base10_multiply(
|
||||
hex_to_point(pubkey), decode(privkey, 16)))
|
||||
|
||||
|
||||
def privtopub(privkey):
|
||||
"""Converting key from private to public"""
|
||||
return point_to_hex(base10_multiply(G, decode(privkey, 16)))
|
||||
|
||||
|
||||
def add(p1, p2):
|
||||
"""Adding two public keys"""
|
||||
if len(p1) == 32:
|
||||
return encode(decode(p1, 16) + decode(p2, 16) % P, 16, 32)
|
||||
return point_to_hex(base10_add(hex_to_point(p1), hex_to_point(p2)))
|
||||
|
||||
|
||||
def hash_160(string):
|
||||
"""Hashed version of public key"""
|
||||
intermed = hashlib.sha256(string).digest()
|
||||
ripemd160 = hashlib.new('ripemd160')
|
||||
ripemd160.update(intermed)
|
||||
|
|
@ -129,17 +146,18 @@ def hash_160(string):
|
|||
|
||||
|
||||
def dbl_sha256(string):
|
||||
"""Double hashing (SHA256)"""
|
||||
return hashlib.sha256(hashlib.sha256(string).digest()).digest()
|
||||
|
||||
|
||||
def bin_to_b58check(inp):
|
||||
"""Convert binary to base58"""
|
||||
inp_fmtd = '\x00' + inp
|
||||
leadingzbytes = len(re.match('^\x00*', inp_fmtd).group(0))
|
||||
checksum = dbl_sha256(inp_fmtd)[:4]
|
||||
return '1' * leadingzbytes + changebase(inp_fmtd + checksum, 256, 58)
|
||||
|
||||
# Convert a public key (in hex) to a Bitcoin address
|
||||
|
||||
|
||||
def pubkey_to_address(pubkey):
|
||||
"""Convert a public key (in hex) to a Bitcoin address"""
|
||||
return bin_to_b58check(hash_160(changebase(pubkey, 16, 256)))
|
||||
|
|
|
|||
|
|
@ -1,10 +1,8 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
src/pyelliptic/cipher.py
|
||||
========================
|
||||
Symmetric Encryption
|
||||
"""
|
||||
|
||||
# Copyright (C) 2011 Yann GUIBET <yannguibet@gmail.com>
|
||||
# See LICENSE for details.
|
||||
|
||||
|
|
@ -14,7 +12,7 @@ from .openssl import OpenSSL
|
|||
# pylint: disable=redefined-builtin
|
||||
class Cipher(object):
|
||||
"""
|
||||
Symmetric encryption
|
||||
Main class for encryption
|
||||
|
||||
import pyelliptic
|
||||
iv = pyelliptic.Cipher.gen_IV('aes-256-cfb')
|
||||
|
|
|
|||
|
|
@ -1,11 +1,9 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
src/pyelliptic/ecc.py
|
||||
=====================
|
||||
Asymmetric cryptography using elliptic curves
|
||||
"""
|
||||
# pylint: disable=protected-access
|
||||
|
||||
# pylint: disable=protected-access, too-many-branches, too-many-locals
|
||||
# Copyright (C) 2011 Yann GUIBET <yannguibet@gmail.com>
|
||||
# See LICENSE for details.
|
||||
|
||||
|
|
@ -172,7 +170,8 @@ class ECC(object):
|
|||
|
||||
if OpenSSL.EC_POINT_get_affine_coordinates_GFp(
|
||||
group, pub_key, pub_key_x, pub_key_y, 0) == 0:
|
||||
raise Exception("[OpenSSL] EC_POINT_get_affine_coordinates_GFp FAIL ...")
|
||||
raise Exception(
|
||||
"[OpenSSL] EC_POINT_get_affine_coordinates_GFp FAIL ...")
|
||||
|
||||
privkey = OpenSSL.malloc(0, OpenSSL.BN_num_bytes(priv_key))
|
||||
pubkeyx = OpenSSL.malloc(0, OpenSSL.BN_num_bytes(pub_key_x))
|
||||
|
|
@ -275,7 +274,6 @@ class ECC(object):
|
|||
|
||||
def raw_check_key(self, privkey, pubkey_x, pubkey_y, curve=None):
|
||||
"""Check key validity, key is supplied as binary data"""
|
||||
# pylint: disable=too-many-branches
|
||||
if curve is None:
|
||||
curve = self.curve
|
||||
elif isinstance(curve, str):
|
||||
|
|
@ -322,7 +320,6 @@ class ECC(object):
|
|||
"""
|
||||
Sign the input with ECDSA method and returns the signature
|
||||
"""
|
||||
# pylint: disable=too-many-branches,too-many-locals
|
||||
try:
|
||||
size = len(inputb)
|
||||
buff = OpenSSL.malloc(inputb, size)
|
||||
|
|
@ -392,7 +389,6 @@ class ECC(object):
|
|||
Verify the signature with the input and the local public key.
|
||||
Returns a boolean
|
||||
"""
|
||||
# pylint: disable=too-many-branches
|
||||
try:
|
||||
bsig = OpenSSL.malloc(sig, len(sig))
|
||||
binputb = OpenSSL.malloc(inputb, len(inputb))
|
||||
|
|
@ -435,10 +431,13 @@ class ECC(object):
|
|||
0, digest, dgst_len.contents, bsig, len(sig), key)
|
||||
|
||||
if ret == -1:
|
||||
return False # Fail to Check
|
||||
# Fail to Check
|
||||
return False
|
||||
if ret == 0:
|
||||
return False # Bad signature !
|
||||
return True # Good
|
||||
# Bad signature !
|
||||
return False
|
||||
# Good
|
||||
return True
|
||||
|
||||
finally:
|
||||
OpenSSL.EC_KEY_free(key)
|
||||
|
|
@ -486,7 +485,6 @@ class ECC(object):
|
|||
"""
|
||||
Decrypt data with ECIES method using the local private key
|
||||
"""
|
||||
# pylint: disable=too-many-locals
|
||||
blocksize = OpenSSL.get_cipher(ciphername).get_blocksize()
|
||||
iv = data[:blocksize]
|
||||
i = blocksize
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
#!/usr/bin/env python
|
||||
"""
|
||||
ECC blind signature functionality based on "An Efficient Blind Signature Scheme
|
||||
ECC blind signature functionality based on
|
||||
"An Efficient Blind Signature Scheme
|
||||
Based on the Elliptic CurveDiscrete Logarithm Problem" by Morteza Nikooghadama
|
||||
<mnikooghadam@sbu.ac.ir> and Ali Zakerolhosseini <a-zaker@sbu.ac.ir>,
|
||||
http://www.isecure-journal.com/article_39171_47f9ec605dd3918c2793565ec21fcd7a.pdf
|
||||
|
|
@ -8,7 +9,6 @@ http://www.isecure-journal.com/article_39171_47f9ec605dd3918c2793565ec21fcd7a.pd
|
|||
|
||||
# variable names are based on the math in the paper, so they don't conform
|
||||
# to PEP8
|
||||
# pylint: disable=invalid-name
|
||||
|
||||
from .openssl import OpenSSL
|
||||
|
||||
|
|
@ -72,8 +72,7 @@ class ECCBlind(object): # pylint: disable=too-many-instance-attributes
|
|||
# F = (x0, y0)
|
||||
x0 = OpenSSL.BN_new()
|
||||
y0 = OpenSSL.BN_new()
|
||||
OpenSSL.EC_POINT_get_affine_coordinates_GFp(group, F, x0, y0,
|
||||
ctx)
|
||||
OpenSSL.EC_POINT_get_affine_coordinates_GFp(group, F, x0, y0, ctx)
|
||||
return x0
|
||||
|
||||
def __init__(self, curve="secp256k1", pubkey=None):
|
||||
|
|
@ -82,7 +81,8 @@ class ECCBlind(object): # pylint: disable=too-many-instance-attributes
|
|||
if pubkey:
|
||||
self.group, self.G, self.n, self.Q = pubkey
|
||||
else:
|
||||
self.group = OpenSSL.EC_GROUP_new_by_curve_name(OpenSSL.get_curve(curve))
|
||||
self.group = OpenSSL.EC_GROUP_new_by_curve_name(
|
||||
OpenSSL.get_curve(curve))
|
||||
# Order n
|
||||
self.n = OpenSSL.BN_new()
|
||||
OpenSSL.EC_GROUP_get_order(self.group, self.n, self.ctx)
|
||||
|
|
|
|||
|
|
@ -2,12 +2,11 @@
|
|||
# See LICENSE for details.
|
||||
#
|
||||
# Software slightly changed by Jonathan Warren <bitmessage at-symbol jonwarren.org>
|
||||
# pylint: disable=protected-access, import-error
|
||||
"""
|
||||
This module loads openssl libs with ctypes and incapsulates
|
||||
needed openssl functionality in class _OpenSSL.
|
||||
"""
|
||||
|
||||
# pylint: disable=protected-access
|
||||
import sys
|
||||
import ctypes
|
||||
from kivy.utils import platform
|
||||
|
|
@ -15,10 +14,9 @@ from kivy.utils import platform
|
|||
OpenSSL = None
|
||||
|
||||
|
||||
class CipherName:
|
||||
class CipherName(object):
|
||||
"""Class returns cipher name, pointer and blocksize"""
|
||||
|
||||
# pylint: disable=old-style-class
|
||||
def __init__(self, name, pointer, blocksize):
|
||||
self._name = name
|
||||
self._pointer = pointer
|
||||
|
|
@ -74,11 +72,11 @@ def get_version(library):
|
|||
return (version, hexversion, cflags)
|
||||
|
||||
|
||||
class _OpenSSL:
|
||||
class _OpenSSL(object):
|
||||
"""
|
||||
Wrapper for OpenSSL using ctypes
|
||||
"""
|
||||
# pylint: disable=too-many-statements, too-many-instance-attributes, old-style-class
|
||||
# pylint: disable=too-many-statements, too-many-instance-attributes
|
||||
def __init__(self, library):
|
||||
"""Build the wrapper"""
|
||||
self._lib = ctypes.CDLL(library)
|
||||
|
|
@ -139,7 +137,8 @@ class _OpenSSL:
|
|||
self.EC_KEY_get0_group.restype = ctypes.c_void_p
|
||||
self.EC_KEY_get0_group.argtypes = [ctypes.c_void_p]
|
||||
|
||||
self.EC_POINT_get_affine_coordinates_GFp = self._lib.EC_POINT_get_affine_coordinates_GFp
|
||||
self.EC_POINT_get_affine_coordinates_GFp = \
|
||||
self._lib.EC_POINT_get_affine_coordinates_GFp
|
||||
self.EC_POINT_get_affine_coordinates_GFp.restype = ctypes.c_int
|
||||
self.EC_POINT_get_affine_coordinates_GFp.argtypes = [ctypes.c_void_p,
|
||||
ctypes.c_void_p,
|
||||
|
|
@ -162,7 +161,8 @@ class _OpenSSL:
|
|||
self.EC_KEY_set_group.argtypes = [ctypes.c_void_p,
|
||||
ctypes.c_void_p]
|
||||
|
||||
self.EC_POINT_set_affine_coordinates_GFp = self._lib.EC_POINT_set_affine_coordinates_GFp
|
||||
self.EC_POINT_set_affine_coordinates_GFp = \
|
||||
self._lib.EC_POINT_set_affine_coordinates_GFp
|
||||
self.EC_POINT_set_affine_coordinates_GFp.restype = ctypes.c_int
|
||||
self.EC_POINT_set_affine_coordinates_GFp.argtypes = [ctypes.c_void_p,
|
||||
ctypes.c_void_p,
|
||||
|
|
@ -296,7 +296,8 @@ class _OpenSSL:
|
|||
self.EVP_CipherUpdate = self._lib.EVP_CipherUpdate
|
||||
self.EVP_CipherUpdate.restype = ctypes.c_int
|
||||
self.EVP_CipherUpdate.argtypes = [ctypes.c_void_p,
|
||||
ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int]
|
||||
ctypes.c_void_p, ctypes.c_void_p,
|
||||
ctypes.c_void_p, ctypes.c_int]
|
||||
|
||||
self.EVP_CipherFinal_ex = self._lib.EVP_CipherFinal_ex
|
||||
self.EVP_CipherFinal_ex.restype = ctypes.c_int
|
||||
|
|
@ -329,12 +330,14 @@ class _OpenSSL:
|
|||
self.ECDSA_sign = self._lib.ECDSA_sign
|
||||
self.ECDSA_sign.restype = ctypes.c_int
|
||||
self.ECDSA_sign.argtypes = [ctypes.c_int, ctypes.c_void_p,
|
||||
ctypes.c_int, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p]
|
||||
ctypes.c_int, ctypes.c_void_p,
|
||||
ctypes.c_void_p, ctypes.c_void_p]
|
||||
|
||||
self.ECDSA_verify = self._lib.ECDSA_verify
|
||||
self.ECDSA_verify.restype = ctypes.c_int
|
||||
self.ECDSA_verify.argtypes = [ctypes.c_int, ctypes.c_void_p,
|
||||
ctypes.c_int, ctypes.c_void_p, ctypes.c_int, ctypes.c_void_p]
|
||||
ctypes.c_int, ctypes.c_void_p,
|
||||
ctypes.c_int, ctypes.c_void_p]
|
||||
|
||||
if self._hexversion >= 0x10100000 and not self._libreSSL:
|
||||
self.EVP_MD_CTX_new = self._lib.EVP_MD_CTX_new
|
||||
|
|
@ -392,7 +395,8 @@ class _OpenSSL:
|
|||
self.HMAC = self._lib.HMAC
|
||||
self.HMAC.restype = ctypes.c_void_p
|
||||
self.HMAC.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_int,
|
||||
ctypes.c_void_p, ctypes.c_int, ctypes.c_void_p, ctypes.c_void_p]
|
||||
ctypes.c_void_p, ctypes.c_int,
|
||||
ctypes.c_void_p, ctypes.c_void_p]
|
||||
|
||||
try:
|
||||
self.PKCS5_PBKDF2_HMAC = self._lib.PKCS5_PBKDF2_HMAC
|
||||
|
|
@ -529,17 +533,29 @@ class _OpenSSL:
|
|||
|
||||
def _set_ciphers(self):
|
||||
self.cipher_algo = {
|
||||
'aes-128-cbc': CipherName('aes-128-cbc', self.EVP_aes_128_cbc, 16),
|
||||
'aes-256-cbc': CipherName('aes-256-cbc', self.EVP_aes_256_cbc, 16),
|
||||
'aes-128-cfb': CipherName('aes-128-cfb', self.EVP_aes_128_cfb128, 16),
|
||||
'aes-256-cfb': CipherName('aes-256-cfb', self.EVP_aes_256_cfb128, 16),
|
||||
'aes-128-ofb': CipherName('aes-128-ofb', self._lib.EVP_aes_128_ofb, 16),
|
||||
'aes-256-ofb': CipherName('aes-256-ofb', self._lib.EVP_aes_256_ofb, 16),
|
||||
# 'aes-128-ctr': CipherName('aes-128-ctr', self._lib.EVP_aes_128_ctr, 16),
|
||||
# 'aes-256-ctr': CipherName('aes-256-ctr', self._lib.EVP_aes_256_ctr, 16),
|
||||
'bf-cfb': CipherName('bf-cfb', self.EVP_bf_cfb64, 8),
|
||||
'bf-cbc': CipherName('bf-cbc', self.EVP_bf_cbc, 8),
|
||||
'rc4': CipherName('rc4', self.EVP_rc4, 128), # 128 is the initialisation size not block size
|
||||
'aes-128-cbc': CipherName(
|
||||
'aes-128-cbc', self.EVP_aes_128_cbc, 16),
|
||||
'aes-256-cbc': CipherName(
|
||||
'aes-256-cbc', self.EVP_aes_256_cbc, 16),
|
||||
'aes-128-cfb': CipherName(
|
||||
'aes-128-cfb', self.EVP_aes_128_cfb128, 16),
|
||||
'aes-256-cfb': CipherName(
|
||||
'aes-256-cfb', self.EVP_aes_256_cfb128, 16),
|
||||
'aes-128-ofb': CipherName(
|
||||
'aes-128-ofb', self._lib.EVP_aes_128_ofb, 16),
|
||||
'aes-256-ofb': CipherName(
|
||||
'aes-256-ofb', self._lib.EVP_aes_256_ofb, 16),
|
||||
# 'aes-128-ctr': CipherName(
|
||||
# 'aes-128-ctr', self._lib.EVP_aes_128_ctr, 16),
|
||||
# 'aes-256-ctr': CipherName(
|
||||
# 'aes-256-ctr', self._lib.EVP_aes_256_ctr, 16),
|
||||
'bf-cfb': CipherName(
|
||||
'bf-cfb', self.EVP_bf_cfb64, 8),
|
||||
'bf-cbc': CipherName(
|
||||
'bf-cbc', self.EVP_bf_cbc, 8),
|
||||
# 128 is the initialisation size not block size
|
||||
'rc4': CipherName(
|
||||
'rc4', self.EVP_rc4, 128),
|
||||
}
|
||||
|
||||
def _set_curves(self):
|
||||
|
|
@ -599,14 +615,13 @@ class _OpenSSL:
|
|||
raise Exception("Unknown curve")
|
||||
return self.curves[name]
|
||||
|
||||
def get_curve_by_id(self, id):
|
||||
def get_curve_by_id(self, id_):
|
||||
"""
|
||||
returns the name of a elliptic curve with his id
|
||||
"""
|
||||
# pylint: disable=redefined-builtin
|
||||
res = None
|
||||
for i in self.curves:
|
||||
if self.curves[i] == id:
|
||||
if self.curves[i] == id_:
|
||||
res = i
|
||||
break
|
||||
if res is None:
|
||||
|
|
@ -617,32 +632,31 @@ class _OpenSSL:
|
|||
"""
|
||||
OpenSSL random function
|
||||
"""
|
||||
# pylint: disable=redefined-builtin
|
||||
buffer = self.malloc(0, size)
|
||||
# This pyelliptic library, by default, didn't check the return value of RAND_bytes. It is
|
||||
# evidently possible that it returned an error and not-actually-random data. However, in
|
||||
# tests on various operating systems, while generating hundreds of gigabytes of random
|
||||
# strings of various sizes I could not get an error to occur. Also Bitcoin doesn't check
|
||||
# the return value of RAND_bytes either.
|
||||
buffer_ = self.malloc(0, size)
|
||||
# This pyelliptic library, by default, didn't check the return value
|
||||
# of RAND_bytes. It is evidently possible that it returned an error
|
||||
# and not-actually-random data. However, in tests on various
|
||||
# operating systems, while generating hundreds of gigabytes of random
|
||||
# strings of various sizes I could not get an error to occur.
|
||||
# Also Bitcoin doesn't check the return value of RAND_bytes either.
|
||||
# Fixed in Bitmessage version 0.4.2 (in source code on 2013-10-13)
|
||||
while self.RAND_bytes(buffer, size) != 1:
|
||||
while self.RAND_bytes(buffer_, size) != 1:
|
||||
import time
|
||||
time.sleep(1)
|
||||
return buffer.raw
|
||||
return buffer_.raw
|
||||
|
||||
def malloc(self, data, size):
|
||||
"""
|
||||
returns a create_string_buffer (ctypes)
|
||||
"""
|
||||
# pylint: disable=redefined-builtin
|
||||
buffer = None
|
||||
buffer_ = None
|
||||
if data != 0:
|
||||
if sys.version_info.major == 3 and isinstance(data, type('')):
|
||||
data = data.encode()
|
||||
buffer = self.create_string_buffer(data, size)
|
||||
buffer_ = self.create_string_buffer(data, size)
|
||||
else:
|
||||
buffer = self.create_string_buffer(size)
|
||||
return buffer
|
||||
buffer_ = self.create_string_buffer(size)
|
||||
return buffer_
|
||||
|
||||
|
||||
def loadOpenSSL():
|
||||
|
|
@ -657,12 +671,24 @@ def loadOpenSSL():
|
|||
if getattr(sys, 'frozen', None):
|
||||
if 'darwin' in sys.platform:
|
||||
libdir.extend([
|
||||
path.join(environ['RESOURCEPATH'], '..', 'Frameworks', 'libcrypto.dylib'),
|
||||
path.join(environ['RESOURCEPATH'], '..', 'Frameworks', 'libcrypto.1.1.0.dylib'),
|
||||
path.join(environ['RESOURCEPATH'], '..', 'Frameworks', 'libcrypto.1.0.2.dylib'),
|
||||
path.join(environ['RESOURCEPATH'], '..', 'Frameworks', 'libcrypto.1.0.1.dylib'),
|
||||
path.join(environ['RESOURCEPATH'], '..', 'Frameworks', 'libcrypto.1.0.0.dylib'),
|
||||
path.join(environ['RESOURCEPATH'], '..', 'Frameworks', 'libcrypto.0.9.8.dylib'),
|
||||
path.join(
|
||||
environ['RESOURCEPATH'], '..',
|
||||
'Frameworks', 'libcrypto.dylib'),
|
||||
path.join(
|
||||
environ['RESOURCEPATH'], '..',
|
||||
'Frameworks', 'libcrypto.1.1.0.dylib'),
|
||||
path.join(
|
||||
environ['RESOURCEPATH'], '..',
|
||||
'Frameworks', 'libcrypto.1.0.2.dylib'),
|
||||
path.join(
|
||||
environ['RESOURCEPATH'], '..',
|
||||
'Frameworks', 'libcrypto.1.0.1.dylib'),
|
||||
path.join(
|
||||
environ['RESOURCEPATH'], '..',
|
||||
'Frameworks', 'libcrypto.1.0.0.dylib'),
|
||||
path.join(
|
||||
environ['RESOURCEPATH'], '..',
|
||||
'Frameworks', 'libcrypto.0.9.8.dylib'),
|
||||
])
|
||||
elif 'win32' in sys.platform or 'win64' in sys.platform:
|
||||
libdir.append(path.join(sys._MEIPASS, 'libeay32.dll'))
|
||||
|
|
@ -682,7 +708,8 @@ def loadOpenSSL():
|
|||
path.join(sys._MEIPASS, 'libssl.so.0.9.8'),
|
||||
])
|
||||
if 'darwin' in sys.platform:
|
||||
libdir.extend(['libcrypto.dylib', '/usr/local/opt/openssl/lib/libcrypto.dylib'])
|
||||
libdir.extend([
|
||||
'libcrypto.dylib', '/usr/local/opt/openssl/lib/libcrypto.dylib'])
|
||||
elif 'win32' in sys.platform or 'win64' in sys.platform:
|
||||
libdir.append('libeay32.dll')
|
||||
elif platform == "android":
|
||||
|
|
@ -701,7 +728,6 @@ def loadOpenSSL():
|
|||
libdir.append(find_library('ssl'))
|
||||
except OSError:
|
||||
pass
|
||||
elif 'win32' in sys.platform or 'win64' in sys.platform:
|
||||
libdir.append(find_library('libeay32'))
|
||||
for library in libdir:
|
||||
try:
|
||||
|
|
@ -709,7 +735,8 @@ def loadOpenSSL():
|
|||
return
|
||||
except:
|
||||
pass
|
||||
raise Exception("Couldn't find and load the OpenSSL library. You must install it.")
|
||||
raise Exception(
|
||||
"Couldn't find and load the OpenSSL library. You must install it.")
|
||||
|
||||
|
||||
loadOpenSSL()
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
"""
|
||||
src/storage/filesystem.py
|
||||
=========================
|
||||
Module for using filesystem (directory with files) for inventory storage
|
||||
"""
|
||||
from binascii import hexlify, unhexlify
|
||||
from os import listdir, makedirs, path, remove, rmdir
|
||||
|
|
@ -12,8 +11,9 @@ from paths import lookupAppdataFolder
|
|||
from storage.storage import InventoryStorage, InventoryItem
|
||||
|
||||
|
||||
class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ancestors, abstract-method
|
||||
"""Module for using filesystem (directory with files) for inventory storage"""
|
||||
class FilesystemInventory(InventoryStorage):
|
||||
"""Filesystem for inventory storage"""
|
||||
# pylint: disable=too-many-ancestors, abstract-method
|
||||
topDir = "inventory"
|
||||
objectDir = "objects"
|
||||
metadataFilename = "metadata"
|
||||
|
|
@ -21,21 +21,23 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
|
|||
|
||||
def __init__(self):
|
||||
super(FilesystemInventory, self).__init__()
|
||||
self.baseDir = path.join(lookupAppdataFolder(), FilesystemInventory.topDir)
|
||||
self.baseDir = path.join(
|
||||
lookupAppdataFolder(), FilesystemInventory.topDir)
|
||||
for createDir in [self.baseDir, path.join(self.baseDir, "objects")]:
|
||||
if path.exists(createDir):
|
||||
if not path.isdir(createDir):
|
||||
raise IOError("%s exists but it's not a directory" % (createDir))
|
||||
raise IOError(
|
||||
"%s exists but it's not a directory" % createDir)
|
||||
else:
|
||||
makedirs(createDir)
|
||||
# Guarantees that two receiveDataThreads don't receive and process the same message
|
||||
# Guarantees that two receiveDataThreads
|
||||
# don't receive and process the same message
|
||||
# concurrently (probably sent by a malicious individual)
|
||||
self.lock = RLock()
|
||||
self._inventory = {}
|
||||
self._load()
|
||||
|
||||
def __contains__(self, hashval):
|
||||
retval = False
|
||||
for streamDict in self._inventory.values():
|
||||
if hashval in streamDict:
|
||||
return True
|
||||
|
|
@ -48,7 +50,12 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
|
|||
except KeyError:
|
||||
continue
|
||||
if retval.payload is None:
|
||||
retval = InventoryItem(retval.type, retval.stream, self.getData(hashval), retval.expires, retval.tag)
|
||||
retval = InventoryItem(
|
||||
retval.type,
|
||||
retval.stream,
|
||||
self.getData(hashval),
|
||||
retval.expires,
|
||||
retval.tag)
|
||||
return retval
|
||||
raise KeyError(hashval)
|
||||
|
||||
|
|
@ -56,7 +63,10 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
|
|||
with self.lock:
|
||||
value = InventoryItem(*value)
|
||||
try:
|
||||
makedirs(path.join(self.baseDir, FilesystemInventory.objectDir, hexlify(hashval)))
|
||||
makedirs(path.join(
|
||||
self.baseDir,
|
||||
FilesystemInventory.objectDir,
|
||||
hexlify(hashval)))
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
|
|
@ -69,7 +79,11 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
|
|||
),
|
||||
"w",
|
||||
) as f:
|
||||
f.write("%s,%s,%s,%s," % (value.type, value.stream, value.expires, hexlify(value.tag)))
|
||||
f.write("%s,%s,%s,%s," % (
|
||||
value.type,
|
||||
value.stream,
|
||||
value.expires,
|
||||
hexlify(value.tag)))
|
||||
with open(
|
||||
path.join(
|
||||
self.baseDir,
|
||||
|
|
@ -115,7 +129,10 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
|
|||
except IOError:
|
||||
pass
|
||||
try:
|
||||
rmdir(path.join(self.baseDir, FilesystemInventory.objectDir, hexlify(hashval)))
|
||||
rmdir(path.join(
|
||||
self.baseDir,
|
||||
FilesystemInventory.objectDir,
|
||||
hexlify(hashval)))
|
||||
except IOError:
|
||||
pass
|
||||
|
||||
|
|
@ -135,7 +152,8 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
|
|||
newInventory = {}
|
||||
for hashId in self.object_list():
|
||||
try:
|
||||
objectType, streamNumber, expiresTime, tag = self.getMetadata(hashId)
|
||||
objectType, streamNumber, expiresTime, tag = self.getMetadata(
|
||||
hashId)
|
||||
try:
|
||||
newInventory[streamNumber][hashId] = InventoryItem(
|
||||
objectType, streamNumber, None, expiresTime, tag)
|
||||
|
|
@ -155,7 +173,8 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
|
|||
|
||||
def object_list(self):
|
||||
"""Return inventory vectors (hashes) from a directory"""
|
||||
return [unhexlify(x) for x in listdir(path.join(self.baseDir, FilesystemInventory.objectDir))]
|
||||
return [unhexlify(x) for x in listdir(path.join(
|
||||
self.baseDir, FilesystemInventory.objectDir))]
|
||||
|
||||
def getData(self, hashId):
|
||||
"""Get object data"""
|
||||
|
|
@ -185,15 +204,20 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
|
|||
),
|
||||
"r",
|
||||
) as f:
|
||||
objectType, streamNumber, expiresTime, tag, undef = string.split(f.read(), ",", 4)
|
||||
return [int(objectType), int(streamNumber), int(expiresTime), unhexlify(tag)]
|
||||
objectType, streamNumber, expiresTime, tag = string.split(
|
||||
f.read(), ",", 4)[:4]
|
||||
return [
|
||||
int(objectType),
|
||||
int(streamNumber),
|
||||
int(expiresTime),
|
||||
unhexlify(tag)]
|
||||
except IOError:
|
||||
raise KeyError
|
||||
|
||||
def by_type_and_tag(self, objectType, tag):
|
||||
"""Get a list of objects filtered by object type and tag"""
|
||||
retval = []
|
||||
for stream, streamDict in self._inventory:
|
||||
for streamDict in self._inventory.values():
|
||||
for hashId, item in streamDict:
|
||||
if item.type == objectType and item.tag == tag:
|
||||
try:
|
||||
|
|
@ -201,7 +225,12 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
|
|||
item.payload = self.getData(hashId)
|
||||
except IOError:
|
||||
continue
|
||||
retval.append(InventoryItem(item.type, item.stream, item.payload, item.expires, item.tag))
|
||||
retval.append(InventoryItem(
|
||||
item.type,
|
||||
item.stream,
|
||||
item.payload,
|
||||
item.expires,
|
||||
item.tag))
|
||||
return retval
|
||||
|
||||
def hashes_by_stream(self, stream):
|
||||
|
|
@ -215,7 +244,8 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
|
|||
"""Return unexpired hashes in the inventory for a particular stream"""
|
||||
t = int(time.time())
|
||||
try:
|
||||
return [x for x, value in self._inventory[stream].items() if value.expires > t]
|
||||
return [x for x, value in self._inventory[stream].items()
|
||||
if value.expires > t]
|
||||
except KeyError:
|
||||
return []
|
||||
|
||||
|
|
@ -227,7 +257,7 @@ class FilesystemInventory(InventoryStorage): # pylint: disable=too-many-ances
|
|||
"""Clean out old items from the inventory"""
|
||||
minTime = int(time.time()) - (60 * 60 * 30)
|
||||
deletes = []
|
||||
for stream, streamDict in self._inventory.items():
|
||||
for streamDict in self._inventory.values():
|
||||
for hashId, item in streamDict.items():
|
||||
if item.expires < minTime:
|
||||
deletes.append(hashId)
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
"""
|
||||
src/storage/sqlite.py
|
||||
=========================
|
||||
Sqlite Inventory
|
||||
"""
|
||||
import sqlite3
|
||||
import time
|
||||
|
|
@ -21,9 +20,11 @@ class SqliteInventory(InventoryStorage): # pylint: disable=too-many-ancestors
|
|||
# cache for existing objects, used for quick lookups if we have an object.
|
||||
# This is used for example whenever we receive an inv message from a peer
|
||||
# to check to see what items are new to us.
|
||||
# We don't delete things out of it; instead, the singleCleaner thread clears and refills it.
|
||||
# We don't delete things out of it; instead,
|
||||
# the singleCleaner thread clears and refills it.
|
||||
self._objects = {}
|
||||
# Guarantees that two receiveDataThreads don't receive and process the same message concurrently
|
||||
# Guarantees that two receiveDataThreads don't receive
|
||||
# and process the same message concurrently
|
||||
# (probably sent by a malicious individual)
|
||||
self.lock = RLock()
|
||||
|
||||
|
|
@ -32,7 +33,9 @@ class SqliteInventory(InventoryStorage): # pylint: disable=too-many-ancestors
|
|||
with self.lock:
|
||||
if hash_ in self._objects:
|
||||
return True
|
||||
rows = sqlQuery('SELECT streamnumber FROM inventory WHERE hash=?', sqlite3.Binary(hash_))
|
||||
rows = sqlQuery(
|
||||
'SELECT streamnumber FROM inventory WHERE hash=?',
|
||||
sqlite3.Binary(hash_))
|
||||
if not rows:
|
||||
return False
|
||||
self._objects[hash_] = rows[0][0]
|
||||
|
|
@ -76,36 +79,49 @@ class SqliteInventory(InventoryStorage): # pylint: disable=too-many-ancestors
|
|||
def __len__(self):
|
||||
print('----------__len__------------------')
|
||||
with self.lock:
|
||||
return len(self._inventory) + sqlQuery('SELECT count(*) FROM inventory')[0][0]
|
||||
return len(self._inventory) + sqlQuery(
|
||||
'SELECT count(*) FROM inventory')[0][0]
|
||||
|
||||
def by_type_and_tag(self, objectType, tag):
|
||||
print('----------by_type_and_tag------------------')
|
||||
"""Return objects filtered by object type and tag"""
|
||||
with self.lock:
|
||||
values = [value for value in self._inventory.values() if value.type == objectType and value.tag == tag]
|
||||
values = [value for value in self._inventory.values()
|
||||
if value.type == objectType and value.tag == tag]
|
||||
values += (InventoryItem(*value) for value in sqlQuery(
|
||||
'SELECT objecttype, streamnumber, payload, expirestime, tag \
|
||||
FROM inventory WHERE objecttype=? AND tag=?', objectType, sqlite3.Binary(tag)))
|
||||
'SELECT objecttype, streamnumber, payload, expirestime, tag'
|
||||
' FROM inventory WHERE objecttype=? AND tag=?',
|
||||
objectType, sqlite3.Binary(tag)))
|
||||
return values
|
||||
|
||||
def unexpired_hashes_by_stream(self, stream):
|
||||
"""Return unexpired inventory vectors filtered by stream"""
|
||||
with self.lock:
|
||||
t = int(time.time())
|
||||
hashes = [x for x, value in self._inventory.items() if value.stream == stream and value.expires > t]
|
||||
hashes += (payload for payload, in sqlQuery(
|
||||
'SELECT hash FROM inventory WHERE streamnumber=? AND expirestime>?', stream, t))
|
||||
hashes = [x for x, value in self._inventory.items()
|
||||
if value.stream == stream and value.expires > t]
|
||||
hashes += (str(payload) for payload, in sqlQuery(
|
||||
'SELECT hash FROM inventory WHERE streamnumber=?'
|
||||
' AND expirestime>?', stream, t))
|
||||
return hashes
|
||||
|
||||
def flush(self):
|
||||
"""Flush cache"""
|
||||
with self.lock:
|
||||
# If you use both the inventoryLock and the sqlLock, always use the inventoryLock OUTSIDE of the sqlLock.
|
||||
# If you use both the inventoryLock and the sqlLock,
|
||||
# always use the inventoryLock OUTSIDE of the sqlLock.
|
||||
with SqlBulkExecute() as sql:
|
||||
for objectHash, value in self._inventory.items():
|
||||
sql.execute('INSERT INTO inventory VALUES (?, ?, ?, ?, ?, ?)', sqlite3.Binary(objectHash), *value)
|
||||
sql.execute(
|
||||
'INSERT INTO inventory VALUES (?, ?, ?, ?, ?, ?)',
|
||||
sqlite3.Binary(objectHash), *value)
|
||||
self._inventory.clear()
|
||||
|
||||
def clean(self):
|
||||
"""Free memory / perform garbage collection"""
|
||||
with self.lock:
|
||||
sqlExecute('DELETE FROM inventory WHERE expirestime<?', int(time.time()) - (60 * 60 * 3))
|
||||
sqlExecute(
|
||||
'DELETE FROM inventory WHERE expirestime<?',
|
||||
int(time.time()) - (60 * 60 * 3))
|
||||
self._objects.clear()
|
||||
for objectHash, value in self._inventory.items():
|
||||
self._objects[objectHash] = value.stream
|
||||
|
|
|
|||
|
|
@ -1,21 +1,22 @@
|
|||
"""
|
||||
src/storage/storage.py
|
||||
======================
|
||||
Storing inventory items
|
||||
"""
|
||||
import collections
|
||||
|
||||
InventoryItem = collections.namedtuple('InventoryItem', 'type stream payload expires tag')
|
||||
InventoryItem = collections.namedtuple(
|
||||
'InventoryItem', 'type stream payload expires tag')
|
||||
|
||||
|
||||
class Storage(object):
|
||||
"""Base class for storing inventory (extendable for other items to store)"""
|
||||
class Storage(object): # pylint: disable=too-few-public-methods
|
||||
"""Base class for storing inventory
|
||||
(extendable for other items to store)"""
|
||||
pass
|
||||
|
||||
|
||||
class InventoryStorage(Storage, collections.MutableMapping):
|
||||
"""Module used for inventory storage"""
|
||||
def __init__(self):
|
||||
# pylint: disable=super-init-not-called
|
||||
|
||||
def __init__(self): # pylint: disable=super-init-not-called
|
||||
self.numberOfInventoryLookupsPerformed = 0
|
||||
|
||||
def __contains__(self, _):
|
||||
|
|
@ -53,8 +54,20 @@ class InventoryStorage(Storage, collections.MutableMapping):
|
|||
raise NotImplementedError
|
||||
|
||||
|
||||
class MailboxStorage(Storage, collections.MutableMapping): # pylint: disable=abstract-method
|
||||
class MailboxStorage(Storage, collections.MutableMapping):
|
||||
"""Method for storing mails"""
|
||||
def __init__(self):
|
||||
# pylint: disable=super-init-not-called
|
||||
pass
|
||||
|
||||
def __delitem__(self, key):
|
||||
raise NotImplementedError
|
||||
|
||||
def __getitem__(self, key):
|
||||
raise NotImplementedError
|
||||
|
||||
def __iter__(self):
|
||||
raise NotImplementedError
|
||||
|
||||
def __len__(self):
|
||||
raise NotImplementedError
|
||||
|
||||
def __setitem__(self, key, value):
|
||||
raise NotImplementedError
|
||||
|
|
|
|||
Reference in New Issue
Block a user