You are on page 1of 55

seedbox

Release

July 28, 2014


Contents

1 Getting started 3
1.1 General Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Resources 5

3 Contents 7
3.1 CHANGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2 Project To Do List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.3 Source Code Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.4 Sample Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

4 Indices and tables 49

Python Module Index 51

i
ii
seedbox, Release

SeedboxManager is an automated task manager for for synchronizing files from a seedbox to your home library.

Contents 1
seedbox, Release

2 Contents
CHAPTER 1

Getting started

Create virutalenv
virtualenv ~/seedbox/

Start virutalenv
cd ~/seedbox
source bin/activate

Install SeedboxManager in the virtualenv:


mkdir etc
pip install SeedboxManager

Running SeedboxManager:
seedmgr

Note: As long as virtualenv is running (source ~/seedbox/bin/activate); you will be able to execute seedmgr from
anywhere.

Running SeedboxManager from crontab:


crontab -e
@hourly /home/USER/seedbox/bin/seedmgr >> /home/USER/seedbox/etc/seedbox/cron-sync.log 2>&1

Note: As part of installing in virtualenv the sample configuration files will be installed into the ~/seedbox/etc/seedbox
folder.

Starting Admin UI and REST API:


dbadmin sqlite:////home/USER/.seedbox/torrent.db >> /home/USER/seedbox/etc/seedbox/admin.log 2>&1

1.1 General Information

Phases and Built-in Tasks

3
seedbox, Release

Phase Task Description


prepare filecopy copy supported media files related to torrents from download directory to sync directory
fileunrar decompress rar media files related to torrents from download directory to sync directory
activate filesync rsync files in sync directory to remote server location
complete filedelete delete media files from sync directory after successful sync to remote server location
Congiguration
Possible configuration file locations (General to specific):
/etc
/etc/seedbox
# if virtualenv used
~/seedbox/etc
~/seedbox/etc/seedbox
~
~/.seedbox
<current working directory>

Note: configuration filename: seedbox.conf


virtualenv approach is the recommended approach. Multiple configuration files are supported such that each supported
folder is checked for a configuration file and loaded from most general to more specific. Each successive file will
override values from the previous.
The folder of the most specific configuration file found will be considered the resource folder where all log files are
stored by default.

Command line interface:


usage: seedmgr [-h] [--config-dir DIR] [--config-file PATH]
[--logconfig LOG_CONFIG] [--logfile LOG_FILE]
[--loglevel LOG_LEVEL] [--version]

optional arguments:
-h, --help show this help message and exit
--config-dir DIR Path to a config directory to pull *.conf files from.
This file set is sorted, so as to provide a
predictable parse order if individual options are
over-ridden. The set is parsed after the file(s)
specified via previous --config-file, arguments hence
over-ridden options in the directory take precedence.
--config-file PATH Path to a config file to use. Multiple config files
can be specified, with values in later files taking
precedence. The default files used are: None
--logconfig LOG_CONFIG
specific path and filename of logging configuration
(override defaults)
--logfile LOG_FILE specify name of log file (location is resource path)
--loglevel LOG_LEVEL specify logging level to log messages at
--version show program’s version number and exit

Sample Configuration A generated configuration file that contains each option a designation for required, a help
message, default value, and associated type.

4 Chapter 1. Getting started


CHAPTER 2

Resources

• Package on PyPI: https://pypi.python.org/pypi/SeedboxManager


• Repository and issues on GitHub: http://github.com/shad7/seedbox
• Doc on http://seedboxmanager.readthedocs.org/

5
seedbox, Release

6 Chapter 2. Resources
CHAPTER 3

Contents

3.1 CHANGES

3.1.1 2.3.7

• Update to version v2.3.7


• fixed format of message; py26
• updated to support py26
• applied updates to support py26
• updated test cases to cover usage scenarios; added new test class to support generating sample config
• removed files from coverage that are entry point executables or copied from another library to provide coverage
based on actual code from this code base
• fixed pep8 issue
• removed unnecessary docstrings in empty files
• fix ChangeLog by fixing which commit was associated with tag 2.2.4

3.1.2 2.3.6

• Update to version v2.3.6


• test_save still causing an issue; commenting a line for now until periodic issue can be resolved

3.1.3 2.3.5

• Update to version v2.3.5


• removed custom configuration generation code base and script
• fixed db save that fails randomly when two instances with different values not successful when asserting not
equal
• fix pep8 issues
• add generation of sample config via oslo.config instead of custom version copied from oslo.incubator project
• undo travis configuration

7
seedbox, Release

• updating travis configuration


• updating travis configuration
• removing copy of config fixture; leverage instance within oslo.config
• replaced extra imports for the purpose of making config options available; leverage library approach for import-
ing options from other modules
• moved to using mockpatch from oslotest instead of local one
• simplified testing dependencies by leveraging oslotest which handles the setting up the base test class

3.1.4 2.3.4

• Update to version v2.3.4


• removed unnecessary assignment of instance to variable; corrected docstring
• updated parameters to match actual abstract
• fixed TODO docstring
• updated year to 2014
• adding fetch origin as part of finish release

3.1.5 2.3.3

• Update to version v2.3.3


• updated to tox 1.7.2 to fix posarg issue so updated tox.ini to use setup.py directly and allowed for removal of
run_test.sh
• added delete tag after building with pbr; then let git flow handle tagging, merging, and pushing to git
• added support for wheel; and fix related to lazy loading of pbr
• fixed docstring causing warning from sphinx
• Update Changelog

3.1.6 2.3.2

• Update to version v2.3.2


• fixing release process script

3.1.7 2.3.1

• fixing up ChangeLog within docs


• fixing up ChangeLog within docs

8 Chapter 3. Contents
seedbox, Release

3.1.8 2.3.0

• perm change; ignore


• Update to version v2.3.0
• removed deploy section; update_version.sh does this now as part of the tagging process
• updated documentation such that sphinx would format it correctly
• updated docstrings
• added classifiers for python 3.3 and 3.4
• added version badge to README
• added items related to vagrant and pycharm
• added debug to make sure check for mediafiles is working as expected
• added python34 and python33 to extend supported versions
• pep8 fix
• updated to be compliant with pyhton 3
• added TODO
• copied test code for bencode to make sure there is code coverage
• copied bencode into project to support python 3, and then updated custom parser to also support python 3
• removed pypi bencode from requirements
• copied and updated version to support python 3
• updated missing or incomplete docstrings
• moved private method to subclass where instance variable is defined
• removed unncessary priority and comparison methods
• added missing docstrings
• refactored setting log level for library code, and added missing docstrings
• removed unused parameter
• removed test case related to deleted code
• updated missing docstrings
• permissions got changed as part of move to vagrant
• corrected spelling of words in documentation
• refactored the writing of the stdout and stderr to further simplify
• replaced static method containing a single function call by calling function directly
• updated abstract to synchronize the signatures
• removed unncessary definition of variable to None
• updated process for logging stdout and stderr from external rsync process that works for both python2/python3
• leverage different name of torrents to avoid conflict between test threads
• fixed import order for pep8
• removed sorted() call as list is not sortable because the list holds classes and not instances

3.1. CHANGES 9
seedbox, Release

• removed unncessary call to getLevelName as setLevel already handles accepting named level
• include default values when accessing values from dict to support python2/python3
• updated access to dict to work for both python2 and python3
• leverage six iteritems for dict to support python2 and python3
• replaced warn with warning for python3 support
• replaced the use of map method with simple list extend method to suppot python3
• removed extra parentheses
• method expected a string, replaced None with empty string
• fixed class name
• converted print builtin calls to print function calls to be python3 compatiable
• fixed class name
• fixed naming of class
• fixed naming of class
• removed extra parentheses
• simplified variable comparison against 2 values
• updated method parameter name to be different than local variable used within the outter method
• converted print builtin calls to print function calls to be python3 compatiable
• converted print builtin calls to print function calls to be python3 compatiable
• converted print builtin calls to print function calls to be python3 compatiable
• changed name of parameter to avoid collision with outter method parameter
• removed extra parentheses
• moved global variable from inside function to module level
• changed local variable that shadows builtin to more descriptive variable name

3.1.9 2.2.4

• Update to version v2.2.4


• get_torrents_active did not take into account that the torrent contained media_files but to keep from processing
the torrent and marking done and to keep query simple, added simple check if torrent contains media_files
before adding to workflow list
• added script to handle packaging a new release, pushing to pypi, and releasing updates to remote git
• Fixed adding link to ChangeLog[C
• updated ChangeLog files

3.1.10 2.2.3

• Updating version to 2.2.3


• added link to ChangeLog to published documentation

10 Chapter 3. Contents
seedbox, Release

3.1.11 2.2.2

• updated ChangeLog and tag release


• fixed the generated ChangeLog to represent the entire history Updated version to 2.2.2 as that is the next release
strting
• updated requirements file to include sandman and click
• fixed bug where a downloaded file was within a subdirectory, and the subdirectory name is included in the
filename cache attribute. Moved the subdirectory name to the filepath
• minor update to documentation configuration

3.1.12 2.2.1

• updated changelog
• bumped to dot release
• test on travis-ci seemed to fail when writing to /tmp/<dir> so added in logic to create the <dir> under /tmp before
trying to create files within <dir>

3.1.13 2.2

• changes added to ChangeLog


• added auto release of successfully built versions that include a tag
• mades changes such that seedbox works as package name but pypi still sees it as SeedboxManager
• updated sample configuration based on changes in the code and the removal of many unncessary options
• updated documentation to reflect the changes
• updated requirements based on changes from sqlobjects to sqlalchemy etc
• generally replaced or rewrote majority of the modules to simplify for the long run
• added test cases for cli to provide coverage
• replaced manager with cli to better refelct the purpose of the module and remove unncessary code
• added constants to hold flow states that are leveraged in multiple places throughout
• removed workaround for name of package causing issues with version; solved by update from pbr
• updated tests for common components
• updated the common components to remove unncessary code, apply minor fixes
• updated test cases to support changes within loader
• updated torrent loader to leverage new database implementation and models, and minor clean up. minor tweaks
to parser
• added test cases to support the new process flow
• simplified the process/workflow to get rid of the elaborate steps to find out which task(s) to execute next and in
what order. Instead of only being able to execute sync in parallel, now all tasks are executed in parallel
• added test cases for all tasks (plugins)

3.1. CHANGES 11
seedbox, Release

• simplified plugin model by migrating to stevedore and providing a simple abstract base task such that a task
must only implement execute() and/or the optional is_actionable() method
• added testing to support new implementation based on sqlalchemy
• replaced SQLObjects with sqlalchemy to lay the foundation for supporting multi-threaded/multi-process with
databases that include actual multiple concurrent requests. Added public object model for interacting with data
without resulting in direct database interactions (fetch, create, update). Included sqlalchemy-migration to handle
version the database schema
• moved gen_config to the tools directory to keep main directory clean
• added other details to setup.cfg
• PBR added ‘–use-mailcap’ in the call to git log to load AUTHORS which is causing it to no longer be found as
git log does not recognize the option. So removing the use of AUTHORS for a while
• added sample config generation anoptoin within tox
• some cleanup activities to reduce clutter and noise. Also small patch to version since my install library PBR
seems to struggle with app name being different from package name
• updated README to remove a badge
• another fix to publish coverage results
• updated README
• tweaks for coverage
• trying something
• update settings for travis-ci
• minor tweaks
• documentation updates
• updating changelog and increasing version

3.1.14 2.1

• updating changelog and increasing version


• code fixes and cleanup
• made several updates to clean up code and added significant amount of test code to finally reach ~75% code
coverage
• added test cases for options module
• Added more test cases and removed extraneous lines of code
• Added test cases for common/timeutil
• regenerated ChangeLog and generated sample configuration file
• significant refactor to simplify and become more DRY. Also reshaping the structure to align to future plans to
replace the entire workflow approach currently leveraged
• Updated reference to travis ci
• Updating import from __future__ entries
• Instead of printing to stderr when the lockfile is there, simply write to the standard log to avoid having to check
logs in multiple locations. Also a few pep8 updates

12 Chapter 3. Contents
seedbox, Release

• Seems a variable named errno was used which took over the namespace of the imported errno module. Needed
to remove the local variable to avoid clash
• Moved to leverage six instead of doing manual checks for PY2/PY3 and fixed some basic pep8 issues
• Removed old code left over as part of pssh
• MANIFEST.in had a missing ‘c’ so it was excluding all .py files instead of .pyc

3.1.15 2.0

• updated README
• added reference to travis-cli
• travis still
• fixing travis
• small change to get travis to work
• updated travis config and coverage config
• pep8 compliance integration with travis-ci
• fixed bug that cause version to stop working from cli
• added cli option –gen-sample so that generation of sample configuration can be accomplished via seedmgr as
well as shell script stored with project
• documentation config update
• updatd documentation configuration
• Updated release info and started work on making sample config generation a cli option instead of through a shell
script
• fixed setup.cfg to support upload into PyPi; ChangeLog automatic
• see previous commit with details. Moving to version 2.0
• Changed approach for configuration to simplify code and setup. Included is a generator to create a sample
configuration file with help, all available options, their type, default value, and what is required

3.1.16 0.1.20

• added ability for user to specifiy filetypes in configuration file to reduce hardcoding of filetypes. The initial
values are still supported by default

3.1.17 0.1.19

• fixed typo

3.1.18 0.1.18

• reved to next release version


• added logging of stacktrace in plugins

3.1. CHANGES 13
seedbox, Release

3.1.19 0.1.17

• bug fix: format(

3.1.20 0.1.16

• bug fix: forgot to escape sql input when doing select statement

3.1.21 0.1.15

• Reduced excess info logging to avoid growing logs while in cron mode. Added validate_phase plugin. The
new plugin will make sure all torrents are in the proper state before allowing them to continue to the next
phase. Optimized torrentparser; added dependency on Bittorrent-bencode as after performance testing it was
substantially more efficient but also stricter. Therefore it will work 98% of the time and the remaiing 2% of the
time we’ll leverage the custom parser to extract files associated with torrent

3.1.22 0.1.14

• bumping version for next upload


• added patch to make sure using loglevel option was case insensitive
• added unittest for action module and resulted in bug fixes

3.1.23 0.1.13

• lockfile on pypi is out of date by nearly 2 years. I pulled from GitHub to get latest version. No longered required
extension to lockfile
• updated README
• updated README
• updated README

3.1.24 0.1.12

• bugfix: date calculation to determine when to perform db back


• updated README
• updated README
• updated README
• updated README
• updated README
• updated README info

3.1.25 0.1.11

• fixed bug related to how frequently to do backups of db

14 Chapter 3. Contents
seedbox, Release

3.1.26 0.1.10

• added lockfile support to make sure that when running as a cron that multiple instances do not run at the same
time

3.1.27 0.1.9

• undo change to filesync

3.1.28 0.1.8

• general cleanup
• removed commented code

3.1.29 0.1.7

• bug fix: variable name changed but didn’t change all locations

3.1.30 0.1.6

• updated the backup db routine to work similar to RotatingLogFiles


• renamed test folder to tests
• added shutilwhich to setup.py as dependency

3.1.31 0.1.5

• moved purge from plugin to internals of datamanager. Deleted actual filepurge.py as it is no longer needed. Also
added in dependency on shutilwhich since running as a cron made it difficult to find unrar

3.1.32 0.1.4

• bumped the rev


• updated logext to default to user folder/directory if available; else cwd
• updated README; added new required attribute to configfile

3.1.33 0.1.3

• found bug where if torrent was still downloading it would be marked as missing and then purged without ever
doing sync. Added check to make sure it waits and checks again later. (automation will really help you find
issues

3.1. CHANGES 15
seedbox, Release

3.1.34 0.1.1

• forgot to include ez_setup.py in distribution


• no longer needed with setuptools model
• Changes to support packaging and distribution
• added LICENSE; MIT
• removed logfile option from filesync given it is now redundant since logging from the subprocess is now directly
supported
• subprocess module will output to stdout/stderr but I wanted everything consistently going to logging so it can
be properly controlled. So I added an extension to subprocess to handle attaching logging to the child process
and created unit testing for the new module. Then updated filesync which uses rsync for remote syncing of files
to leverage new module
• updated distribution related files
• deleted old test files since they are replaced will real unit test modules
• added sample config files to support testing
• Added another test set related to processing options and command line
• Started using unittest module for doing proper testing and converting some of the scripts I had been using to do
proper unit testing. This was the first one
• updated readme
• updated readme
• Updated readme
• made adjustment so that filecopy works the same as fileunrar, after copy create a new entry for syncing. And
then within delete, simply ignore any file that does not exist to avoid any exceptions
• Bug fixes related to database backup, some extra logging

3.1.35 0.1

• adding setup/distribution related files


• Initial creation
• Initial commit

3.2 Project To Do List

Todo
Investigate merging logic from custom parser into bencode code base such that a single unit of code supports parsing.

(The original entry is located in /var/build/user_builds/seedboxmanager/checkouts/2.3.7/seedbox/torrent/parser.py:docstring


of seedbox.torrent.parser.TorrentParser, line 18.)

16 Chapter 3. Contents
seedbox, Release

3.3 Source Code Index

3.3.1 Seedbox

The seedbox.version Module

Access to the version as configured as part of installation package. Leverages pbr (Python Build Reasonableness)
seedbox.version.release_string()
Provide a string representing the release of the project.
Returns project release
Return type string
seedbox.version.version_string()
Provide a string representing the version of the project.
Returns project version
Return type string

The seedbox.logext Module

Configures logging module for the entire application, setting default log levels for library code.
seedbox.logext.configure()
configure all logging for this execution

The seedbox Module

The seedbox.cli Module

The main program that is the entry point for the SeedboxManager application. Provides the ability to configure and
start up processing.
seedbox.cli.main()
Entry point for seedmgr

The seedbox.options Module

Defines all the common configurations for the application, and then manages loading all Options for system.
seedbox.options.initialize(args)
Handles finding and loading configuration options for the entire system. Searches for configuration files in the
following locations:
VIRTUAL_ENV
defined when virtualenv is started
source bin/activation
•/etc/
•/etc/seedbox/
•~/VIRTUAL_ENV/etc/
•~/VIRTUAL_ENV/etc/seedbox/

3.3. Source Code Index 17


seedbox, Release

•~/
•~/.seedbox/
•./ (current working directory)

Parameters args (list) – command line inputs

seedbox.options.list_opts()
Returns a list of oslo.config options available in the library.
The returned list includes all oslo.config options which may be registered at runtime by the library.
Each element of the list is a tuple. The first element is the name of the group under which the list of elements in
the second element will be registered. A group name of None corresponds to the [DEFAULT] group in config
files.
The purpose of this is to allow tools like the Oslo sample config file generator to discover the options exposed
to users by this library.
Returns a list of (group_name, opts) tuples

The seedbox.constants Module

Constants needed throughout application.

The seedbox.common.tools Module

Holds a group of tools useful by all modules.


seedbox.common.tools.format_file_ext(filetypes)
verifies that each item in the list of filetypes is a string and starts with a ‘.’
Parameters filetypes (list) – a list of extensions representing types of files
seedbox.common.tools.make_opt_list(opts, group)
Generate a list of tuple containing group, options
Parameters
• opts (list) – option lists associated with a group
• group (str) – name of an option group
Returns a list of (group_name, opts) tuples
Return type list
seedbox.common.tools.verify_path(path_entry)
verify a path, if it exists make sure it is an absolute path and return. else None
Parameters path_entry – path to file or directory on host

The seedbox.common Module

The seedbox.common.timeutil Module

Time related utilities and helper functions.

18 Chapter 3. Contents
seedbox, Release

class seedbox.common.timeutil.AfterDelta(method)
Bases: object
@AfterDelta
A decorator that has state and handles checking if period of time
(delta) has passed before executing the function. That way if you
need to control whether a piece of code executes in a loop, the delta
is applied before execution otherwise NOOP
DEFAULT_DELTA = 60
get_delta()
Retrieves the delta (period of time to wait between executions)

Note: To change the delta simply subclass and override this one method, and use the subclass name as
the decorator

Returns period of time (seconds)


Return type int

seedbox.common.timeutil.advance_time_delta(dt, timedelta)
Advances a datetime by a datetime.timedelta
Parameters
• dt (datetime) – a specified date time
• timedelta (datetime.timedelta) – time offset (delta)
Returns a datetime incremented by delta
Return type datetime
seedbox.common.timeutil.advance_time_seconds(dt, seconds)
Advances a datetime by a seconds
Parameters
• dt (datetime) – a specified date time
• seconds (int) – seconds (delta)
Returns a datetime incremented by seconds
Return type datetime
seedbox.common.timeutil.delta_seconds(before, after)
Return the difference between two timing objects.
Compute the difference in seconds between two date, time, or datetime objects (as a float, to microsecond
resolution).
Parameters
• before – date, datetime, time object
• after – date, datetime, time object
Returns difference in seconds
Return type int
seedbox.common.timeutil.is_newer_than(after, seconds)
Checks if a datetime is newer than seconds

3.3. Source Code Index 19


seedbox, Release

Parameters
• after (datetime) – a datetime to check
• seconds (int) – seconds (delta)
Returns True if before is newer than seconds else False
Return type bool
seedbox.common.timeutil.is_older_than(before, seconds)
Checks if a datetime is older than seconds
Parameters
• before (datetime) – a datetime to check
• seconds (int) – seconds (delta)
Returns True if before is older than seconds else False
Return type bool
seedbox.common.timeutil.is_soon(dt, window)
Determines if time is going to happen in the next window seconds.
Parameters
• dt – the time
• window – minimum seconds to remain to consider the time not soon
Returns True if expiration is within the given duration
seedbox.common.timeutil.nvl_date(dt, default=None)
Determines if the provided date, time, or datetime has a value, and returns the provided value back or the value
of default (current time)
Parameters
• dt – an instance of a date, time, or datetime
• default – value to return if provided dt has no value
Returns date, time, or datetime provided or default
Return type datetime
seedbox.common.timeutil.time_delta_seconds(seconds)
Retrieves a timedelta
Parameters seconds (int) – delta of seconds
Returns timedelta by seconds
Return type timedelta
seedbox.common.timeutil.timed(method=None, logger=None, loglvl=None)

@seedbox.common.timeutil.timed(logger, loglvl)
A decorator that times the execution of a method/function and logs
using the supplied logger at the specified loglevel.

Parameters
• method – the method being timed

20 Chapter 3. Contents
seedbox, Release

• logger (logging.Logger) – reference to logger (defaults to logger of module holding the


decorated method or function)
• loglvl (int) – a logging level from logging.LEVELS defaults to DEBUG

seedbox.common.timeutil.total_seconds(delta)
Return the total seconds of datetime.timedelta object.
Compute total seconds of datetime.timedelta, datetime.timedelta doesn’t have method total_seconds in
Python2.6, calculate it manually.
Parameters delta (timedelta) – a delta to convert
Returns seconds
Return type int
seedbox.common.timeutil.utcnow()
Returns current time from utc
Return type datetime

The seedbox.db.exception Module

Database Exception definition classes


exception seedbox.db.exception.DBError(inner_exception=None)
Bases: exceptions.Exception
Wraps an implementation specific exception.
exception seedbox.db.exception.DbMigrationError(message=None)
Bases: seedbox.db.exception.DBError
Wraps migration specific exception.
exception seedbox.db.exception.MultipleResultsFound(inner_exception=None)
Bases: seedbox.db.exception.DBError
Represents when Multiple results found when searching by unique id
exception seedbox.db.exception.NoResultFound(inner_exception=None)
Bases: seedbox.db.exception.DBError
Represents when No results found when fetching by unique id

The seedbox.db.models Module

Model classes to represent the structures of data.


class seedbox.db.models.AppState(name, value)
Bases: seedbox.db.models.Model
Represents the state of the application and internal processing.
Parameters
• name (str) – an identifier for the capturing a state attribute
• value (str | int | bool | datetime.datetime) – the data associated with the identifier
Returns an instance of the AppState object

3.3. Source Code Index 21


seedbox, Release

Return type AppState


PK_NAME = ‘name’
class seedbox.db.models.MediaFile(media_id, torrent_id, filename, file_ext, file_path=None,
size=None, compressed=None, synced=None, missing=None,
skipped=None, error_msg=None, total_time=None)
Bases: seedbox.db.models.Model
Represents the attributes of media files associated to a torrent and the associated state based on parsing and
processing.
Parameters
• media_id (int) – primary key identifier of media file
• torrent_id (int) – foreign key identifier of torrent
• filename (str) – name of media file
• file_ext (str) – extension of media file
• file_path (str) – location of media file
• size (int) – file size of media file
• compressed (bool) – flag indicating compressed file
• synced (bool) – flag indicating sync to remote location
• missing (bool) – flag indicating file not found on file system
• skipped (bool) – flag indicating if skipped during processing
• error_msg (str) – error message that happened during processing
• total_time (int) – total amount of time to process file
Returns an instance of the MediaFile object
Return type MediaFile
PK_NAME = ‘media_id’
class seedbox.db.models.Model(**kwds)
Bases: object
Provides base methods for interacting with database model
Parameters kwds (dict) – keyword parameters and values of the model
PK_NAME = ‘id’
as_dict()
Generates a dictionary representation of the model.
Returns model as dict
Return type dict
items()
Provides a generator of key-value pair attributes for a model.
classmethod make_empty()
Create an instance of the model using the defined attributes from the parameters of the model.
Returns model class
Return type Model

22 Chapter 3. Contents
seedbox, Release

classmethod pk_filter(value=None)
Creates a primary key filter for the given model.
Parameters value – primary key value
Returns primary key filter
Return type dict
class seedbox.db.models.Torrent(torrent_id, name, created_at=None, updated_at=None,
state=None, retry_count=None, failed=None, error_msg=None,
invalid=None, purged=None, media_files=None)
Bases: seedbox.db.models.Model
Represents the attributes of a torrent and the associated state based on parsing and processing of the torrent.
Parameters
• torrent_id (int) – primary key identifier of torrent
• name (str) – name of the torrent file
• created_at (datetime.datetime) – date when db entry created
• updated_at (datetime.datetime) – date when db entry updated
• state (str) – current state of processing of torrent (init, ready, active, done, cancelled)
• retry_count (int) – number of times entry was reprocessed
• failed (bool) – flag indicating processing failed
• error_msg (str) – error message generated during processing
• invalid (bool) – flag indicating entry was invalid
• purged (bool) – flag indicating entry details were purged
• media_files (list) – list of associated media file(s)
Returns an instance of the Torrent object
Return type Torrent
PK_NAME = ‘torrent_id’

The seedbox.db Module

Provides access to the database API for interacting with the torrent data.
seedbox.db.dbapi(conf=<oslo.config.cfg.ConfigOpts object at 0x7fc9ddb11a90>)
Retrieves an instance of the configured database API.
Parameters conf (oslo.config.cfg.ConfigOpts) – an instance of the configuration file
Returns database API instance
Return type DBApi
seedbox.db.list_opts()
Returns a list of oslo.config options available in the library.
The returned list includes all oslo.config options which may be registered at runtime by the library.
Each element of the list is a tuple. The first element is the name of the group under which the list of elements in
the second element will be registered. A group name of None corresponds to the [DEFAULT] group in config
files.

3.3. Source Code Index 23


seedbox, Release

The purpose of this is to allow tools like the Oslo sample config file generator to discover the options exposed
to users by this library.
Returns a list of (group_name, opts) tuples

The seedbox.db.maintenance Module

Provides the ability to perform maintenance on a database.


seedbox.db.maintenance.backup(conf )
create a backup copy of the database file.
Parameters conf (oslo.config.cfg.ConfigOpts) – an instance of configuration

The seedbox.db.api Module

Public facing database api that leverages a provided implementation to execute database operations.
class seedbox.db.api.DBApi(impl)
Bases: object
Provides API for supported database operations that leverage specified implementation.
Parameters impl (Connection) – database plugin instance
backup()
Backup database.
bulk_create_medias(medias)
Perform save (insert) operation on a list of instances of media.
Parameters medias – a list of instances of media file
Returns media file instance(s)
Return type MediaFile
bulk_save_torrents(value_map, qfilter)
Performs save (update) operation on multiple instances of torrents using the provided data map.
Parameters
• value_map – data attributes and values to update
• qfilter – query filter to determine instances to update
clean_up()
Periodically check for data no longer needed (fully processed, invalid, or deleted) and then remove from
the database.
clear()
Clears all data from database.
delete_appstate(appstate)
Perform delete operation on an instance of appstate.
Parameters appstate – an instance of appstate
delete_media(media)
Perform delete operation on an instance of media.
Parameters media – an instance of media file

24 Chapter 3. Contents
seedbox, Release

delete_medias(qfilter)
Perform delete operation on a list of instances of media.
Parameters qfilter – query filter to determine instances to delete
delete_torrent(torrent)
Performs delete operation on specific instance of torrent.
Parameters torrent – an instance of torrent
delete_torrents(qfilter)
Performs delete operation on selection of torrent instances.
Parameters qfilter – query filter to determine instances to delete.
fetch_or_create_torrent(name)
Performs select operation using the name of torrent to fetch torrent, and if the torrent is not found, then the
save (insert) operation is performed using the name of the torrent.
Parameters name – the name of a torrent
Returns torrent instance(s)
Return type Torrent
get_appstate(name)
Perform select operation using name of appstate to fetch an instance of appstate.
Parameters name – name of an appstate instance
Returns appstate instance
Return type AppState
get_media(media_id)
Perform select operation using media primary key to fetch media.
Parameters media_id – media primary key
Returns media file instance
Return type MediaFile
get_medias(qfilter)
Perform select operation on selection of media instances.
Parameters qfilter – query filter to determine instances to fetch.
Returns media file instance(s)
Return type MediaFile
get_medias_by(torrent_id, file_path=None, compressed=None, synced=None, missing=None,
skipped=None)
Perform select operation using a combination of the provided parameters to find specified media.
Parameters
• torrent_id – primary key of torrent
• file_path – location where media exist on file system (default: None; ignore attribute)
• compressed – flag to indicate to include or exclude compressed media (default: None;
ignore attribute)
• synced – flag to indicate to include or exclude synced media ( default: None; ignored
attribute)

3.3. Source Code Index 25


seedbox, Release

• missing – flag to indicate to include or exclude missing media (default: None; ignored
attribute)
• skipped – flag to indicate to include or exclude skipped media (default: None; ignored
attribute)
Returns media file instance(s)
Return type MediaFile
get_medias_by_torrent(torrent_id)
Perform select operation using the primary key of torrent to fetch associated media.
Parameters torrent_id – primary key of torrent
Returns media file instance(s)
Return type MediaFile
get_processed_medias(torrent_id)
Performed select operation using torrent primary key and pre-defined criteria for what constitutes pro-
cessed media.
Parameters torrent_id – torrent primary key
Returns media file instance(s)
Return type MediaFile
get_torrent(torrent_id)
Perform select operation using the primary key of torrent to fetch torrent.
Parameters torrent_id – primary key of torrent
Returns torrent instance(s)
Return type Torrent
get_torrent_by_name(name)
Perform select operation using the name of torrent to fetch torrent.
Parameters name – the name of torrent
Returns torrent instance(s)
Return type Torrent
get_torrents(qfilter)
Perform select operation on selection of torrent instances.
Parameters qfilter – query filter to determine instances to fetch.
Returns torrent instance(s)
Return type Torrent
get_torrents_active()
Perform select operation using a pre-defined criteria for what constitutes an active torrent.
Returns torrent instance(s)
Return type Torrent
get_torrents_by_state(state, failed=False)
Perform select operation using a pre-defined criteria to get torrents by a specific state.
Parameters

26 Chapter 3. Contents
seedbox, Release

• state – name of the state of the torrent


• failed – flag indicating to include failed entries or not
Returns torrent instance(s)
Return type Torrent
get_torrents_eligible_for_purging()
Perform select operation using a pre-defined criteria for what constitutes eligible for purging.
Returns torrent instance(s)
Return type Torrent
get_torrents_eligible_for_removal()
Perform select operation using a pre-defined criteria for what constitutes eligible for removal.
Returns torrent instance(s)
Return type Torrent
save_appstate(appstate)
Perform save (insert/update) operation on an instance of appstate.
Parameters appstate – an instance of appstate
Returns appstate instance
Return type AppState
save_media(media)
Perform save (insert/update) operation on an instance of media.
Parameters media – an instance of media file
Returns media file instance
Return type MediaFile
save_torrent(torrent)
Performs save (insert/update) operation on an instance of torrent.
Parameters torrent – an instance of a torrent
Returns saved torrent instance
Return type Torrent
shrink_db()
Shrink database.

The seedbox.db.base Module

An abstract class representing the required capabilities of a database API.


class seedbox.db.base.Connection(conf )
Bases: object
Base class for database connections.
Parameters conf (oslo.config.cfg.ConfigOpts) – an instance of configuration file
backup()
Backup database.

3.3. Source Code Index 27


seedbox, Release

bulk_create(instances)
Save the instances in bulk to the database. :param instances: a list of instance of modeled data object
bulk_update(value_map, entity_type, qfilter)
Save the updated instances in bulk to the database. :param value_map: a dict of key-value pairs represent-
ing the data of an instance. :param entity_type: the model type :param qfilter: query filter to determine
which rows to update
clear()
Clear database.
delete(instance)
Delete the instance(s) based on filter from the database. :param instance: an instance of modeled data
object
delete_by(entity_type, qfilter)
Delete instances of a specific type based on filter criteria :param entity_type: the model type :param qfilter:
query filter to determine which rows to update
fetch(entity_type, pk)
Fetch the instance(s) based on filter from the database. :param entity_type: the model type :param pk:
primary key value
fetch_by(entity_type, qfilter)
Fetch the instance(s) based on filter from the database. :param entity_type: the model type :param qfilter:
query filter to determine which rows to update
save(instance)
Save the instance to the database. :param instance: an instance of modeled data object
shrink_db()
Shrink database.
upgrade()
Migrate the database to version or most recent version.

The seedbox.db.admin Module

Wrapper for sandman; replacement for sandmanctl that provides additional options not available via sandmanctl
seedbox.db.admin.print_version(ctx, value)
Print the current version of sandman and exit. :param ctx: application context :param value:

The seedbox.db.sqlalchemy.model_util Module

Provides utilities for managing models


seedbox.db.sqlalchemy.model_util.from_db(db_item)
Handles the conversion from the database model object to the corresponding public facing api model object. If
an item has a reference to another model object then the call is recursive.
Parameters db_item – an instance of a database model object
Returns an instance of an api model object
seedbox.db.sqlalchemy.model_util.to_db(api_item, db_item=None)
Handles the conversion from the api model object to the corresponding database model object. If an item has a
reference to another model object then the call is recursive.
Parameters

28 Chapter 3. Contents
seedbox, Release

• api_item – an instance of a api model object


• db_item – an instance of a database model object that is to be updated. (optional)
Returns an instance of an database model object

The seedbox.db.sqlalchemy.session Module

Manages connection to the database via sqlalchemy


class seedbox.db.sqlalchemy.session.EngineFacade(sql_connection, **kwargs)
Bases: object
A helper class that creates engine and sessionmaker on its instantiation and provides get_engine()/get_session()
methods.
engine/sessionmaker instances will be global.
Note: Two important things to remember:
1.An Engine instance is effectively a pool of DB connections, so it’s meant to be shared (and it’s thread-safe).
2.A Session instance is not meant to be shared and represents a DB transactional context (i.e. it’s not thread-
safe). sessionmaker is a factory of sessions.
Initialize engine and sessionmaker instances.
Keyword arguments:
Parameters
• idle_timeout – timeout before idle sql connections are reaped (defaults to 3600)
• connection_debug – verbosity of SQL debugging information. 0=None, 100=Everything
(defaults to 0)
engine
Get the engine instance (note, that it’s shared).
classmethod from_config(connection_string, conf )
Initialize EngineFacade using oslo.config config instance options.
Parameters
• connection_string (string) – SQLAlchemy connection string
• conf (oslo.config.cfg.ConfigOpts) – oslo.config config instance
session
Get a Session instance.
session_maker
Get the session maker instance
seedbox.db.sqlalchemy.session.create_engine(sql_connection, idle_timeout=3600, connec-
tion_debug=0)
Return a new SQLAlchemy engine.
Parameters
• sql_connection – sql connection string
• idle_timeout – timeout period the connection can be idle
• connection_debug – enable debugging for the connection

3.3. Source Code Index 29


seedbox, Release

seedbox.db.sqlalchemy.session.get_maker(engine)
Return a SQLAlchemy sessionmaker using the given engine.
Parameters engine – a database connection engine
seedbox.db.sqlalchemy.session.receive_connect(dbapi_con, con_record)
Ensures that the foreign key constraints are enforced in SQLite.
The foreign key constraints are disabled by default in SQLite, so the foreign key constraints will be enabled here
for every database connection
Parameters
• dbapi_con – database connection
• con_record – connection record

The seedbox.db.sqlalchemy.models Module

Provides database model compliant with sqlalchemy.


class seedbox.db.sqlalchemy.models.AppState(id, value=None)
Bases: sqlalchemy.ext.declarative.api.Base
Class representing an app state in the database
Parameters
• id (string) – primary key of app state (alias for name)
• value (string, int, bool, or datetime) – value of the app state
dtype
get(key, default=None)
Overrides base class get method to handle primary key of app state
Parameters
• key – id/name of app state
• default – default value if not found
Returns an appstate value attribute
Return type varies
get_value()
Retrieve the value of an instance of AppState
Returns value of app state
Return type varies
name
reset_value()
Resets the value associated with AppState to empty
set_value(value)
Sets the value of an instance of AppState
Parameters value – the value of an app state instance
Raises TypeError if value type does not map to supported type (string, int, boolean, datetime)
t_bool

30 Chapter 3. Contents
seedbox, Release

t_datetime
t_int
t_string
update(values)
Make the model object behave like a dict.
Parameters values – key-value pairs to update in database
class seedbox.db.sqlalchemy.models.HasId
Bases: object
Table mixin providing a class/table id attribute
id = Column(None, Integer(), table=None, primary_key=True, nullable=False)
class seedbox.db.sqlalchemy.models.HasTimestamp
Bases: object
Table mixin providing a class/table date attributes
created_at = Column(None, DateTime(), table=None, default=ColumnDefault(<function <lambda> at 0x7fc9dc68ecf8>
updated_at = Column(None, DateTime(), table=None, onupdate=ColumnDefault(<function <lambda> at 0x7fc9dc68ed
class seedbox.db.sqlalchemy.models.MediaFile(**kwargs)
Bases: sqlalchemy.ext.declarative.api.Base, seedbox.db.sqlalchemy.models.HasId
Class representing a media file in the database
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in kwargs.
Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any
mapped columns or relationships.
compressed
error_msg
file_ext
file_path
filename
id
missing
size
skipped
synced
torrent_id
total_time
class seedbox.db.sqlalchemy.models.QueryTransformer(table, query)
Bases: object
Provides the ability to transform a query filter into database query in the sqlalchemy compliant manner.
Parameters

3.3. Source Code Index 31


seedbox, Release

• table (Base) – an instance of database table


• query (Query) – an instance of database query
apply_filter(expression_tree)
Uses the filter to update the query
Parameters expression_tree – query filter to apply
Returns database query
Return type Query
complex_operators = {‘and’: <function and_ at 0x7fc9dd630758>, ‘not’: <function not_ at 0x7fc9dd61f938>, ‘or’: <fu
operators = {‘=>’: <built-in function ge>, ‘>=’: <built-in function ge>, ‘!=’: <built-in function ne>, ‘in’: <function <lam
class seedbox.db.sqlalchemy.models.Torrent(**kwargs)
Bases: sqlalchemy.ext.declarative.api.Base, seedbox.db.sqlalchemy.models.HasId,
seedbox.db.sqlalchemy.models.HasTimestamp
Class representing a torrent in the database
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in kwargs.
Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any
mapped columns or relationships.
created_at
error_msg
failed
id
invalid
media_files
name
purged
retry_count
state
updated_at
seedbox.db.sqlalchemy.models.purge_all_tables(engine)
Drops all the defined tables within the database
Parameters engine – database engine instance
seedbox.db.sqlalchemy.models.to_table_name(klass_name)
Convention is to take camel-case class name and rewrite it to an underscore form, e.g. ‘ClassName’ to
‘class_name’
Parameters klass_name – name of class
Returns properly formatted table name
Return type string
seedbox.db.sqlalchemy.models.verify_tables(engine)
Creates all the defined tables within the database

32 Chapter 3. Contents
seedbox, Release

Parameters engine – database engine instance

The seedbox.db.sqlalchemy Module

The seedbox.db.sqlalchemy.migration Module

Provides ability to manage versions of database models


seedbox.db.sqlalchemy.migration.db_sync(engine, version=None, init_version=0)
Upgrade or downgrade a database.
Function runs the upgrade() or downgrade() functions in change scripts.
Parameters
• engine – SQLAlchemy engine instance for a given database
• version – Database will upgrade/downgrade until this version. If None - database will
update to the latest available version.
• init_version – Initial database version
seedbox.db.sqlalchemy.migration.db_version(engine, init_version=0)
Show the current version of the repository.
Parameters
• engine – SQLAlchemy engine instance for a given database
• init_version – Initial database version
seedbox.db.sqlalchemy.migration.db_version_control(engine, version=None)
Mark a database as under this repository’s version control.
Once a database is under version control, schema changes should only be done via change scripts in this repos-
itory.
Parameters
• engine – SQLAlchemy engine instance for a given database
• version – Initial database version

The seedbox.db.sqlalchemy.api Module

Private database API implemented for sqlalchemy for database operations.


class seedbox.db.sqlalchemy.api.Connection(conf )
Bases: seedbox.db.base.Connection
SQLAlchemy connection.
Parameters conf (oslo.config.cfg.ConfigOpts) – an instance of configuration file
backup()
Backup database.
bulk_create(instances)
Save the instances in bulk to the database.
Parameters instances (list) – a list of instance of modeled data object

3.3. Source Code Index 33


seedbox, Release

bulk_update(value_map, entity_type, qfilter)


Perform bulk save based on filter criteria with values from value map to the database.
Parameters
• value_map (dict) – a dict of key-value pairs representing the data of an instance.
• entity_type (class) – the model type
• qfilter (dict) – query filter to determine which rows to update
clear()
Clear database.
delete(instance)
Delete the instance(s) based on filter from the database.
Parameters instance – an instance of modeled data object
delete_by(entity_type, qfilter)
Delete instances of a specific type based on filter criteria
Parameters
• entity_type – the model type
• qfilter – query filter to determine which rows to update
fetch(entity_type, pk)
Fetch the instance using primary key from the database.
Parameters
• entity_type – the model type
• pk – primary key value
fetch_by(entity_type, qfilter)
Fetch the instance(s) based on filter from the database.
Parameters
• entity_type – the model type
• qfilter – query filter to determine which rows to update
save(instance)
Save the instance to the database
Parameters instance – an instance of modeled data object
shrink_db()
Shrink database.
upgrade()
Migrate the database to version or most recent version.

The seedbox.db.sqlalchemy.migrate_repo.manage Module

Execution entry point for database migration

34 Chapter 3. Contents
seedbox, Release

The seedbox.db.sqlalchemy.migrate_repo Module

The seedbox.db.sqlalchemy.migrate_repo.versions.001_initial_tables Module

Initial version of the database model; maintained to support anyone upgrading from a previous version.
seedbox.db.sqlalchemy.migrate_repo.versions.001_initial_tables.downgrade(migrate_engine)
Not implemented because this is associated with initial version.
Parameters migrate_engine – an instance of database connection engine
Raises NotImplementedError if method executed
seedbox.db.sqlalchemy.migrate_repo.versions.001_initial_tables.upgrade(migrate_engine)
Creates the initial version of the database tables.
Parameters migrate_engine – an instance of database connection engine

The seedbox.db.sqlalchemy.migrate_repo.versions.003_drop_V1_tables Module

Provides removing the initial version of the database tables for clean up purposes.
seedbox.db.sqlalchemy.migrate_repo.versions.003_drop_V1_tables.downgrade(migrate_engine)
Does nothing since the ability to go version 1 from 3 is not supported directly.
Parameters migrate_engine – an instance of database connection engine
seedbox.db.sqlalchemy.migrate_repo.versions.003_drop_V1_tables.upgrade(migrate_engine)
Drops the initial version of the tables.
Parameters migrate_engine – an instance of database connection engine

The seedbox.db.sqlalchemy.migrate_repo.versions.002_migrate_to_V2_of_tables
Module

Migrates to version 2 of the database tables and supports downgrading to previous version.
seedbox.db.sqlalchemy.migrate_repo.versions.002_migrate_to_V2_of_tables.downgrade(migrate_eng
Converts back to previous version and removes the newly created tables.
Parameters migrate_engine – an instance of database connection engine
seedbox.db.sqlalchemy.migrate_repo.versions.002_migrate_to_V2_of_tables.upgrade(migrate_engine
Updates table model to new version and moves existing data from old structures to the new structure.
Parameters migrate_engine – an instance of database connection engine

The seedbox.db.sqlalchemy.migrate_repo.versions Module

The seedbox.tasks.filecopy Module

CopyFile task plugin for copying a file to specified location.


class seedbox.tasks.filecopy.CopyFile(media_file)
Bases: seedbox.tasks.base.BaseTask
Provides the capability of copying files from seeding location to another location.
execute()
Perform the action associated with task for the provided media_file.

3.3. Source Code Index 35


seedbox, Release

static is_actionable(media_file)
Perform check to determine if action should be taken.
Parameters media_file – an instance of a MediaFile to check
Returns a flag indicating to act or not to act
Return type boolean

The seedbox.tasks.filesync Module

SyncFile task plugin for performing rsync of a file to a specified location.


class seedbox.tasks.filesync.SyncFile(media_file)
Bases: seedbox.tasks.base.BaseTask
Provides the capability of rsync file to a specified location.
cmd
A property for accessing the rsync command used for specified media file.
Returns rsync command
Return type list
destination
A property for accessing the destination to sync file to.
Returns remote destination
Return type string
execute()
Perform the action associated with task for the provided media_file.
static is_actionable(media_file)
Perform check to determine if action should be taken.
Parameters media_file – an instance of a MediaFile to check
Returns a flag indicating to act or not to act
Return type boolean

The seedbox.tasks.subprocessext Module

Extends the subprocess.Popen class to provide logging threads for stdout and stderr of the child process such that it
will log stdout through logging module to provided logger.info() and log stderr to provided logger.warn().
Because using subprocess.PIPE for stdout and stderr with any of the convenience methods or with the wait() command
has potential for buffer issue, we leverage Popen directly with PIPE but then spawn a thread with access to the
corresponding logger function on the specific pipe. And then leverage the poll method to wait for child process to
complete. If the returncode is not 0, then just like check_call() we will raise the CalledProcessError to be as consistent
as possible. This is done via the staticmethod execute(). Because we subclass Popen, you can simply create the
instance and leverage Popen methods for interaction if you prefer.
The intent here is to have the output send directly to the logger vs. being buffered until the end with the communicate()
method, or dealing with buffer conflict issues like wait(). The logger can handle the different levels, StreamHandler or
even FileHandler and the corresponding subclasses to make it more manageable.
This was derived from several different responses on stackoverflow and blogs on this topic. After pulling them all
together and doing several rounds of testing, this was the result.

36 Chapter 3. Contents
seedbox, Release

class seedbox.tasks.subprocessext.ProcessLogging(cmd)
Bases: subprocess.Popen
Run a command as a subprocess sending output to a logger.
Parameters cmd (list) – command and options sent to subprocess to execute
complete()
Handle all the processing of the subprocess
static execute(cmd)
provide a convenience method for creating the object and waiting for the results; very similar to
check_call() but not at module level.
Parameters cmd (list) – command and options sent to subprocess to execute

The seedbox.tasks Module

Defines tasks to execute as part of the process


seedbox.tasks.list_opts()
Returns a list of oslo.config options available in the library.
The returned list includes all oslo.config options which may be registered at runtime by the library.
Each element of the list is a tuple. The first element is the name of the group under which the list of elements in
the second element will be registered. A group name of None corresponds to the [DEFAULT] group in config
files.
The purpose of this is to allow tools like the Oslo sample config file generator to discover the options exposed
to users by this library.
Returns a list of (group_name, opts) tuples

The seedbox.tasks.fileunrar Module

UnrarFile task plugin for decompressing archived files to specified location.


class seedbox.tasks.fileunrar.UnrarFile(media_file)
Bases: seedbox.tasks.base.BaseTask
Provides the capability of decompressing archived files to specified location.
execute()
Perform the action associated with task for the provided media_file.
static is_actionable(media_file)
Perform check to determine if action should be taken.
Parameters media_file – an instance of a MediaFile to check
Returns a flag indicating to act or not to act
Return type boolean

The seedbox.tasks.base Module

Provides the basic definition for a task that handles execution and basic error handling.

3.3. Source Code Index 37


seedbox, Release

class seedbox.tasks.base.BaseTask(media_file)
Bases: object
Provides the base definition of a task.
add_gen_files(files)
Adds media files included within an archived file to be processed separately to increase parallel processing.
Parameters files – a list of media files produced by a plugin to be included on the torrent.
execute()
Perform the action associated with task for the provided media_file.
static is_actionable(media_file)
Perform check to determine if action should be taken.
Parameters media_file – the file to inspect to determine if action is necessary
Returns a flag indicating to act or not to act
Return type boolean

The seedbox.tasks.filedelete Module

DeleteFile task plugin for deleting a file from specified location.


class seedbox.tasks.filedelete.DeleteFile(media_file)
Bases: seedbox.tasks.base.BaseTask
Provides capability of deleting file from a specified location.
execute()
Perform the action associated with task for the provided media_file.
static is_actionable(media_file)
Perform check to determine if action should be taken.
Parameters media_file – an instance of a MediaFile to check
Returns a flag indicating to act or not to act
Return type boolean

The seedbox.process.flow Module

Provides the definition of workflow (steps and transition), and implementation what happens during each step by
binding in the provided plugins for each and updating the db cache after each step.
class seedbox.process.flow.BaseFlow(torrent)
Bases: xworkflows.base.WorkflowEnabled
Provides the base workflow implementation on binding plugin tasks to each step and determining which plugin
is capable of operating on the media files of the specified torrent.
Parameters torrent (Torrent) – an instance of a parsed torrent metadata
activate
Executes all the tasks for a torrent at activate phase
cancel
NoOp function, ignores all arguments.

38 Chapter 3. Contents
seedbox, Release

complete
Executes all the tasks for a torrent at complete phase
is_done()
Checks if the current state of workflow is either done or cancelled.
Returns flag indicating workflow is done
Return type boolean
next_tasks()
Find the list of tasks and associated media eligible for processing.
Returns list of tasks
Return type generator
phase
The name of current step/phase of the workflow
Returns name of current phase
Return type string
prepare
Executes all the tasks for a torrent at prepare phase
state
Property-like attribute holding the state of a WorkflowEnabled object.
The state is stored in the internal __dict__ of the instance.
tasks
Property for accessing the tasks associated with current workflow step by looking up the configured plug-
ins.
Returns list of tasks (BaseTask)
Return type list
update_state(*args, **kwargs)
Handles the capturing the current state of processing
Parameters
• args – required parameter based on decorator (unused)
• kwargs – required parameter based on decorator (unused)
class seedbox.process.flow.Taskflow
Bases: xworkflows.base.Workflow
Define the workflow conditions for managing torrents;
initial_state = <State: ‘init’>
states = StateList({‘ready’: <State: ‘ready’>, ‘active’: <State: ‘active’>, ‘init’: <State: ‘init’>, ‘done’: <State: ‘done’>,
transitions = TransitionList([Transition(‘cancel’, [<State: ‘ready’>, <State: ‘active’>], <State: ‘cancelled’>), Transitio
seedbox.process.flow.get_tasks(phase)
Gets a list of tasks based the current phase of processing
Parameters phase – the name of the current phase/step of workflow

3.3. Source Code Index 39


seedbox, Release

The seedbox.process Module

The core process flow for managing the syncing of torrents to remote location.
seedbox.process.list_opts()
Returns a list of oslo.config options available in the library.
The returned list includes all oslo.config options which may be registered at runtime by the library.
Each element of the list is a tuple. The first element is the name of the group under which the list of elements in
the second element will be registered. A group name of None corresponds to the [DEFAULT] group in config
files.
The purpose of this is to allow tools like the Oslo sample config file generator to discover the options exposed
to users by this library.
Returns a list of (group_name, opts) tuples
seedbox.process.start()
The primary entry point for the process

The seedbox.process.workflow Module

Workflow implementation that handles automatic execution of each workflow step until the workflow reaches comple-
tion.
wf = Workflow(torrent)
tasks = wf.next_tasks()
# execute tasks in separate threads
<<logic>>
# now move the workflow to the next step
wf.run()

class seedbox.process.workflow.Workflow(torrent)
Bases: seedbox.process.flow.BaseFlow
Wrapper class around Flow that handles the orchestration of the process
activate
Executes all the tasks for a torrent at activate phase
cancel
NoOp function, ignores all arguments.
complete
Executes all the tasks for a torrent at complete phase
prepare
Executes all the tasks for a torrent at prepare phase
run()
Orchestrate each step of the process based on current state
state
Property-like attribute holding the state of a WorkflowEnabled object.
The state is stored in the internal __dict__ of the instance.

40 Chapter 3. Contents
seedbox, Release

The seedbox.process.manager Module

Manages the execution of tasks using parallel processes.


class seedbox.process.manager.TaskManager
Bases: object
Creates a pool of processes and executes the supplied tasks using the process pool.
add_tasks(tasks)
Adds tasks to list of tasks to be executed.
Parameters tasks – a task or list of tasks to add to the list of tasks to execute
run()
Executes the list of tasks.
Returns the result/output from each tasks
Return type list
shutdown()
Shuts down the process pool to free up resources.

The seedbox.torrent.loader Module

Torrent loader handles searches the directory where torrent files are stored and adds them to the database cache if not
already found.
seedbox.torrent.loader.load_torrents()
Find all the torrents in the specified directory, verify it is a valid torrent file (via parsing) and capture the relevant
details. Next create a record in the cache for each torrent.

The seedbox.torrent.bencode Module

Encodes and Decodes messages (http://en.wikipedia.org/wiki/Bencode) updated to support Python 3


exception seedbox.torrent.bencode.BTFailure
Bases: exceptions.Exception
Represents any torrent failure
class seedbox.torrent.bencode.Bencached(s)
Bases: object
Cached bencode class
bencoded
seedbox.torrent.bencode.bdecode(x)
Public method for decoding a message
Parameters x – message to decode
Returns decoded message
Return type dict
Raises BTFailure decoding failure
seedbox.torrent.bencode.bencode(x)
Public method for encoding message

3.3. Source Code Index 41


seedbox, Release

Parameters x – message
Returns encoded string
Return type string
seedbox.torrent.bencode.bytes(s, *args, **kwargs)
seedbox.torrent.bencode.bytes_index(s, pattern, start)
Returns the index of pattern within string starting at position start.
Parameters
• s (str) – byte string to search for pattern
• pattern (str) – pattern to search for within byte string
• start (int) – position/index within byte string to start scanning
Returns index of pattern within byte string
Return type int
seedbox.torrent.bencode.chr_(s)
Return the string representing a character whose Unicode code point is the integer i.
Parameters s (int) – integer representation of unicode character
Returns string representation of unicode character
Return type str
seedbox.torrent.bencode.ord_(s)
Given a string representing one Unicode character, return an integer representing the Unicode code point of that
character.
Parameters s (str) – string representing single unicode character
Returns integer representation of unicode character
Return type int

The seedbox.torrent.parser Module

Parses a torrent file and provides method to access the following attributes.
• Tracker URL
• Creation date
• Client name, if any
• For each file
– name
– length
– checksum
Created on 2012-03-07
@author: mohanr
exception seedbox.torrent.parser.ParsingError(error_msg)
Bases: exceptions.Exception
Error class representing errors that occur while parsing the torrent content.

42 Chapter 3. Contents
seedbox, Release

class seedbox.torrent.parser.TorrentParser(torrent_file_path)
Bases: object
Parses a torrent file and returns various properties based on the content of the torrent file.

Note: bencode is supremely more efficient parser of torrents but extremely strict in the format of the file.
The custom parser is an implementation that someone wrote that does a good job of parsing but it is not very
efficient. So we are going to pair the two solutions together. when the file is well formed we will leverage most
efficient standard but we will leverage the custom parser when bencode generates an exception. If the custom
parser fails then it is really invalid file and the consumer will handle it correspondingly.

Reads the torrent file and sets the content as an object attribute.

Todo
Investigate merging logic from custom parser into bencode code base such that a single unit of code supports
parsing.

Parameters torrent_file_path (str) – Path to the torrent file to be parsed


Raises IOError when a torrent_file_path does not exists

DICT_KEY_VALUE_SEP = ‘: ‘
DICT_LIST_END = ‘e’
DICT_LIST_ITEM_SEP = ‘, ‘
DICT_START = ‘d’
INT_START = ‘i’
LIST_START = ‘l’
get_client_name()
Returns the name of the client that created the torrent if present, from the parsed torrent file.
Returns name of the client that created the torrent
Return type str
get_creation_date(time_format=’iso’)
Retrieves creation date of the torrent, if present, in ISO time_format from the parsed torrent file.
Parameters time_format (str) – determines the time_format of the time value returned. [’iso’,
‘datetime’] defaults: ‘iso’
Returns creation date
Return type datetime
get_files_details()
Parses torrent file and returns details of the files contained in the torrent.
File details tuple:
• name
• length (size)
• checksum (of file in the torrent)

Returns file details embedded within torrent

3.3. Source Code Index 43


seedbox, Release

Return type list of tuples (name, length, checksum)

get_tracker_url()
Retrieves tracker URL from the parsed torrent file
Returns tracker URL from parsed torrent file
Return type str

The seedbox.torrent Module

seedbox package
seedbox.torrent.list_opts()
Returns a list of oslo.config options available in the library.
The returned list includes all oslo.config options which may be registered at runtime by the library.
Each element of the list is a tuple. The first element is the name of the group under which the list of elements in
the second element will be registered. A group name of None corresponds to the [DEFAULT] group in config
files.
The purpose of this is to allow tools like the Oslo sample config file generator to discover the options exposed
to users by this library.
Returns a list of (group_name, opts) tuples
seedbox.torrent.load()
Simple wrapper to provide parsing and loading torrents into a database cache.

3.4 Sample Configuration


[DEFAULT]

#
# Options defined in seedbox.options
#

# **REQUIRED** Base path (string value)


#base_path=<None>

# Location torrent client stores data files (string value)


#base_client_path=<None>

[database]

#
# Options defined in seedbox.db
#

# The connection string used to connect to the database (string value)


#connection=sqlite:///$config_dir/torrent.db

# Timeout before idle sql connections are reaped (integer value)


#idle_timeout=3600

44 Chapter 3. Contents
seedbox, Release

# Verbosity of SQL debugging information. 0=None, 100=All (integer value)


#connection_debug=0

[process]

#
# Options defined in seedbox.process.flow
#

# name of tasks associated with prepare phase (list value)


#prepare=

# name of tasks associated with activate phase (list value)


#activate=

# name of tasks associated with complete phase (list value)


#complete=

#
# Options defined in seedbox.process.manager
#

# max processes to use for performing sync of torrents (integer value)


#max_processes=4

[tasks]

#
# Options defined in seedbox.tasks.base
#

# Location to temp media copies for syncing to library (string value)


#sync_path=/tmp/sync

[tasks_filesync]

#
# Options defined in seedbox.tasks.filesync
#

# rsync dryrun option (boolean value)


#dryrun=false

# rsync verbose option (boolean value)


#verbose=false

# rsync progress option (boolean value)


#progress=false

# rsync perms option (boolean value)


#perms=true

# rsync delayupdates option (boolean value)


#delayupdates=true

3.4. Sample Configuration 45


seedbox, Release

# rsync recursive option (boolean value)


#recursive=true

# rsync chmod option (string value)


#chmod=ugo+rwx

# rsync-ssh identity option (ssh key) (string value)


#identity=<None>

# rsync-ssh port (string value)


#port=22

# User name on remote system (ssh) (string value)


#remote_user=<None>

# Host name/IP Address of remote system (string value)


#remote_host=<None>

# rsync destination path (string value)


#remote_path=<None>

[tasks_synclog]

#
# Options defined in seedbox.tasks.subprocessext
#

# Output directory for stdout files (string value)


#stdout_dir=$config_dir/sync_out

# Output directory for stderr files (string value)


#stderr_dir=$config_dir/sync_err

# Write output to stdout (boolean value)


#stdout_verbose=false

# Output verbose details about exceptions (boolean value)


#stderr_verbose=true

[torrent]

#
# Options defined in seedbox.torrent
#

# **REQUIRED** Location of the .torrent files (string value)


#torrent_path=<None>

# **REQUIRED** Location(s) of the media files (list value)


#media_paths=<None>

# **REQUIRED** Location of the downloading torrents (string value)


#incomplete_path=<None>

# List of video filetypes to support. (ignore others) (list value)


#video_filetypes=.avi,.mp4,.mkv,.mpg

46 Chapter 3. Contents
seedbox, Release

# List of compressed filetypes to support. (ignore others) (list value)


#compressed_filetypes=.rar

# Minimum file size of a media file (integer value)


#minimum_file_size=75000000

3.4. Sample Configuration 47


seedbox, Release

48 Chapter 3. Contents
CHAPTER 4

Indices and tables

• genindex
• modindex
• search

49
seedbox, Release

50 Chapter 4. Indices and tables


Python Module Index

s seedbox.tasks.filedelete, 38
seedbox, 17 seedbox.tasks.filesync, 36
seedbox.cli, 17 seedbox.tasks.fileunrar, 37
seedbox.common, 18 seedbox.tasks.subprocessext, 36
seedbox.common.timeutil, 18 seedbox.torrent, 44
seedbox.common.tools, 18 seedbox.torrent.bencode, 41
seedbox.constants, 18 seedbox.torrent.loader, 41
seedbox.db, 23 seedbox.torrent.parser, 42
seedbox.db.admin, 28 seedbox.version, 17
seedbox.db.api, 24
seedbox.db.base, 27
seedbox.db.exception, 21
seedbox.db.maintenance, 24
seedbox.db.models, 21
seedbox.db.sqlalchemy, 33
seedbox.db.sqlalchemy.api, 33
seedbox.db.sqlalchemy.migrate_repo, 35
seedbox.db.sqlalchemy.migrate_repo.manage,
34
seedbox.db.sqlalchemy.migrate_repo.versions,
35
seedbox.db.sqlalchemy.migrate_repo.versions.001_initial_tables,
35
seedbox.db.sqlalchemy.migrate_repo.versions.002_migrate_to_V2_of_tables,
35
seedbox.db.sqlalchemy.migrate_repo.versions.003_drop_V1_tables,
35
seedbox.db.sqlalchemy.migration, 33
seedbox.db.sqlalchemy.model_util, 28
seedbox.db.sqlalchemy.models, 30
seedbox.db.sqlalchemy.session, 29
seedbox.logext, 17
seedbox.options, 17
seedbox.process, 40
seedbox.process.flow, 38
seedbox.process.manager, 41
seedbox.process.workflow, 40
seedbox.tasks, 37
seedbox.tasks.base, 37
seedbox.tasks.filecopy, 35

51

You might also like