Commit Graph

32 Commits (b6a81f1252531ff6ebe5b6563e6f17778f905eba)

Author SHA1 Message Date
Dmitri Bogomolov 581c8ee087
Style fixes and pylint hint in inventory 5 years ago
Peter Šurda af2bb95601
Remove unused imports 6 years ago
Peter Šurda 1e4400a207
Remove obsolete code
- PendingDownload and PendingUpload in are obsolete
- the replacement functions only provide lengths and are in
6 years ago
Peter Šurda 07149c73c8
Get rid of evals
- eval is evil. Get rid of the remaining evals. They were not assessed
  as dangerous but it's matter of policy now.
6 years ago
Peter Šurda e522f015a8
Network status updates
- only update processed numbers once every 2 seconds
- moved inventory lookups to the main inventory so now all
storage modules work with it
7 years ago
Peter Šurda 36b5e2c04f
Inventory storage abstraction
- can have multiple storage types for inventory
- sqlite is the old one, filesystem is a new available
7 years ago
Peter Šurda 913b401dd0
PendingDownloadQueue updates
- track pending hashId more accurately
- add timeout and a cleanup so that the download queues don't
get stuck and memory is freed
- randomise download order (only works for inv commands with
more than 1 entry)
7 years ago
Peter Šurda 1af49a0165
Download tracking refactoring
- replace PendingDownload singleton dict with a Queue
- total memory and CPU requirements should be reduced
- get rid of somObjectsOfWhichThisRemoteNodeIsAlearedyAware. It has very
little practicle effect and only uses memory
7 years ago
Peter Šurda 405a06c08a
Indentation 7 years ago
Peter Šurda 15077c9388
More accurate PendingUpload tracking
- works correctly when starting offline
- stops tracking after after 60 seconds but only if at least 1
  successful upload
7 years ago
Peter Šurda e434825bb2
Pending download counter fix
- handles expired objects better
- counts objects that failed download more accurately
7 years ago
Peter Šurda c979481564
Unhandled missing peer 7 years ago
Peter Šurda ba130e03e5
Network subsystem freezing fixes
- queues were too short
- some error handling was missing
- remove nonblocking repeats in receive data thread
- singleCleaner shouldn't wait unnecessarily
7 years ago
Peter Šurda f3849eeb48
Unused imports, problematic variable names 7 years ago
Peter Šurda 5ae1327edc
Download/upload shutdown fixes
- Missing renamed to PendingDownload
- PendingDownload now only retries 3 times rather than 6 to dowload an
- Added PendingUpload, replacing invQueueSize
- PendingUpload has both the "len" method (number of objects not
  uploaded) as well as "progress" method, which is a float from 0
  (nothing done) to 1 (all uploaded) which considers not only objects
  but also how many nodes they are uploaded to
- PendingUpload tracks when the object is successfully uploaded to the
  remote node instead of just adding an arbitrary time after they have
  been send the corresponding "inv"
- Network status tab's "Objects to be synced" shows the sum of
  PendingUpload and PendingDownload sizes
7 years ago
Peter Šurda 5708a38be0
Fix sending messages crashing 7 years ago
Peter Šurda 9197c425d2
Object requesting fix 7 years ago
Peter Šurda 4f920fe641
Fix infinite loop 7 years ago
Peter Šurda d652dc864d
Downloading fixes
- able to request more objects with one command
- fixes to logic and error handling
7 years ago
Peter Šurda 9f89df6d1c
Expire objects that we're unable to receive
- sometimes a node would send an "inv" about an object but then not
  provide it when requested. This could be that it expired in the
  meantime or it was an attack or a bug. This patch will forget that the
  object exists if was requested too many times and not received.
7 years ago
Peter Šurda ca6bc9981c
Better tracking in downloading objects
- remember what was requested from which node
- remember if it was received
- re-request object if we haven't received any new object for more than
  a minute
7 years ago
Peter Šurda 8dfa0faca9
More objects to be downloaded fixes 7 years ago
Peter Šurda 12205ee7ba
More fixes in objects to be downloaded 7 years ago
Peter Šurda bd1aead46e
More fixes for objects to be downloaded 7 years ago
Peter Šurda d04c0e78e4
Better handling of pending downloading objects 7 years ago
Peter Šurda b750e67bfb
Handle exception in iterator 7 years ago
Peter Šurda 805f72e098
Make objects to be send stoppable if not empty 7 years ago
Peter Šurda 94f0bdc731
Objects to be downloaded optimising
- treat requested but never arrived objects as expired. This is how it
  worked before the refactoring. Without this, the list won't go to
7 years ago
Peter Šurda 79893fdc23
Performance tuning objects to be downloaded
- rely on dict quasi-random order instead of an additional shuffle
- request an object once per minute
- stop check after count objects have been found
7 years ago
Peter Šurda dbe15d0b99
Objects to be downloaded fixes
- tries to avoid calling senddata it it would block receiveDataThread,
  allowing fore more asynchronous operation
- request objects in chunks of 100 (CPU performance optimisation)
7 years ago
Peter Šurda f079ff5b99
Refactor objects to be downloaded
- moved logic into a Missing singleton
- shouldn't try to download duplicates anymore, only requests a hash
  once every 5 minutes and not from the same host
- removed obsoleted variables
- the "Objects to be synced" in the Network tab should now be correct
- removed some checks which aren't necessary anymore in my opinion
- fix missing self in Throttle (thanks
7 years ago
Peter Šurda e84b19613e
Inventory refactoring
- minor refactoring, made it into singleton instead of a shared global
  variable. This makes it a little bit cleaner and moves the class into
a separate file
- removed duplicate inventory locking
- renamed to (this is the code that
  ensures only one instance of PyBitmessage runs at the same time)
7 years ago