Professional Documents
Culture Documents
Release
1 Getting started 3
1.1 General Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Resources 5
3 Contents 7
3.1 CHANGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2 Project To Do List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.3 Source Code Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.4 Sample Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
i
ii
seedbox, Release
SeedboxManager is an automated task manager for for synchronizing files from a seedbox to your home library.
Contents 1
seedbox, Release
2 Contents
CHAPTER 1
Getting started
Create virutalenv
virtualenv ~/seedbox/
Start virutalenv
cd ~/seedbox
source bin/activate
Running SeedboxManager:
seedmgr
Note: As long as virtualenv is running (source ~/seedbox/bin/activate); you will be able to execute seedmgr from
anywhere.
Note: As part of installing in virtualenv the sample configuration files will be installed into the ~/seedbox/etc/seedbox
folder.
3
seedbox, Release
optional arguments:
-h, --help show this help message and exit
--config-dir DIR Path to a config directory to pull *.conf files from.
This file set is sorted, so as to provide a
predictable parse order if individual options are
over-ridden. The set is parsed after the file(s)
specified via previous --config-file, arguments hence
over-ridden options in the directory take precedence.
--config-file PATH Path to a config file to use. Multiple config files
can be specified, with values in later files taking
precedence. The default files used are: None
--logconfig LOG_CONFIG
specific path and filename of logging configuration
(override defaults)
--logfile LOG_FILE specify name of log file (location is resource path)
--loglevel LOG_LEVEL specify logging level to log messages at
--version show program’s version number and exit
Sample Configuration A generated configuration file that contains each option a designation for required, a help
message, default value, and associated type.
Resources
5
seedbox, Release
6 Chapter 2. Resources
CHAPTER 3
Contents
3.1 CHANGES
3.1.1 2.3.7
3.1.2 2.3.6
3.1.3 2.3.5
7
seedbox, Release
3.1.4 2.3.4
3.1.5 2.3.3
3.1.6 2.3.2
3.1.7 2.3.1
8 Chapter 3. Contents
seedbox, Release
3.1.8 2.3.0
3.1. CHANGES 9
seedbox, Release
• removed unncessary call to getLevelName as setLevel already handles accepting named level
• include default values when accessing values from dict to support python2/python3
• updated access to dict to work for both python2 and python3
• leverage six iteritems for dict to support python2 and python3
• replaced warn with warning for python3 support
• replaced the use of map method with simple list extend method to suppot python3
• removed extra parentheses
• method expected a string, replaced None with empty string
• fixed class name
• converted print builtin calls to print function calls to be python3 compatiable
• fixed class name
• fixed naming of class
• fixed naming of class
• removed extra parentheses
• simplified variable comparison against 2 values
• updated method parameter name to be different than local variable used within the outter method
• converted print builtin calls to print function calls to be python3 compatiable
• converted print builtin calls to print function calls to be python3 compatiable
• converted print builtin calls to print function calls to be python3 compatiable
• changed name of parameter to avoid collision with outter method parameter
• removed extra parentheses
• moved global variable from inside function to module level
• changed local variable that shadows builtin to more descriptive variable name
3.1.9 2.2.4
3.1.10 2.2.3
10 Chapter 3. Contents
seedbox, Release
3.1.11 2.2.2
3.1.12 2.2.1
• updated changelog
• bumped to dot release
• test on travis-ci seemed to fail when writing to /tmp/<dir> so added in logic to create the <dir> under /tmp before
trying to create files within <dir>
3.1.13 2.2
3.1. CHANGES 11
seedbox, Release
• simplified plugin model by migrating to stevedore and providing a simple abstract base task such that a task
must only implement execute() and/or the optional is_actionable() method
• added testing to support new implementation based on sqlalchemy
• replaced SQLObjects with sqlalchemy to lay the foundation for supporting multi-threaded/multi-process with
databases that include actual multiple concurrent requests. Added public object model for interacting with data
without resulting in direct database interactions (fetch, create, update). Included sqlalchemy-migration to handle
version the database schema
• moved gen_config to the tools directory to keep main directory clean
• added other details to setup.cfg
• PBR added ‘–use-mailcap’ in the call to git log to load AUTHORS which is causing it to no longer be found as
git log does not recognize the option. So removing the use of AUTHORS for a while
• added sample config generation anoptoin within tox
• some cleanup activities to reduce clutter and noise. Also small patch to version since my install library PBR
seems to struggle with app name being different from package name
• updated README to remove a badge
• another fix to publish coverage results
• updated README
• tweaks for coverage
• trying something
• update settings for travis-ci
• minor tweaks
• documentation updates
• updating changelog and increasing version
3.1.14 2.1
12 Chapter 3. Contents
seedbox, Release
• Seems a variable named errno was used which took over the namespace of the imported errno module. Needed
to remove the local variable to avoid clash
• Moved to leverage six instead of doing manual checks for PY2/PY3 and fixed some basic pep8 issues
• Removed old code left over as part of pssh
• MANIFEST.in had a missing ‘c’ so it was excluding all .py files instead of .pyc
3.1.15 2.0
• updated README
• added reference to travis-cli
• travis still
• fixing travis
• small change to get travis to work
• updated travis config and coverage config
• pep8 compliance integration with travis-ci
• fixed bug that cause version to stop working from cli
• added cli option –gen-sample so that generation of sample configuration can be accomplished via seedmgr as
well as shell script stored with project
• documentation config update
• updatd documentation configuration
• Updated release info and started work on making sample config generation a cli option instead of through a shell
script
• fixed setup.cfg to support upload into PyPi; ChangeLog automatic
• see previous commit with details. Moving to version 2.0
• Changed approach for configuration to simplify code and setup. Included is a generator to create a sample
configuration file with help, all available options, their type, default value, and what is required
3.1.16 0.1.20
• added ability for user to specifiy filetypes in configuration file to reduce hardcoding of filetypes. The initial
values are still supported by default
3.1.17 0.1.19
• fixed typo
3.1.18 0.1.18
3.1. CHANGES 13
seedbox, Release
3.1.19 0.1.17
3.1.20 0.1.16
• bug fix: forgot to escape sql input when doing select statement
3.1.21 0.1.15
• Reduced excess info logging to avoid growing logs while in cron mode. Added validate_phase plugin. The
new plugin will make sure all torrents are in the proper state before allowing them to continue to the next
phase. Optimized torrentparser; added dependency on Bittorrent-bencode as after performance testing it was
substantially more efficient but also stricter. Therefore it will work 98% of the time and the remaiing 2% of the
time we’ll leverage the custom parser to extract files associated with torrent
3.1.22 0.1.14
3.1.23 0.1.13
• lockfile on pypi is out of date by nearly 2 years. I pulled from GitHub to get latest version. No longered required
extension to lockfile
• updated README
• updated README
• updated README
3.1.24 0.1.12
3.1.25 0.1.11
14 Chapter 3. Contents
seedbox, Release
3.1.26 0.1.10
• added lockfile support to make sure that when running as a cron that multiple instances do not run at the same
time
3.1.27 0.1.9
3.1.28 0.1.8
• general cleanup
• removed commented code
3.1.29 0.1.7
• bug fix: variable name changed but didn’t change all locations
3.1.30 0.1.6
3.1.31 0.1.5
• moved purge from plugin to internals of datamanager. Deleted actual filepurge.py as it is no longer needed. Also
added in dependency on shutilwhich since running as a cron made it difficult to find unrar
3.1.32 0.1.4
3.1.33 0.1.3
• found bug where if torrent was still downloading it would be marked as missing and then purged without ever
doing sync. Added check to make sure it waits and checks again later. (automation will really help you find
issues
3.1. CHANGES 15
seedbox, Release
3.1.34 0.1.1
3.1.35 0.1
Todo
Investigate merging logic from custom parser into bencode code base such that a single unit of code supports parsing.
16 Chapter 3. Contents
seedbox, Release
3.3.1 Seedbox
Access to the version as configured as part of installation package. Leverages pbr (Python Build Reasonableness)
seedbox.version.release_string()
Provide a string representing the release of the project.
Returns project release
Return type string
seedbox.version.version_string()
Provide a string representing the version of the project.
Returns project version
Return type string
Configures logging module for the entire application, setting default log levels for library code.
seedbox.logext.configure()
configure all logging for this execution
The main program that is the entry point for the SeedboxManager application. Provides the ability to configure and
start up processing.
seedbox.cli.main()
Entry point for seedmgr
Defines all the common configurations for the application, and then manages loading all Options for system.
seedbox.options.initialize(args)
Handles finding and loading configuration options for the entire system. Searches for configuration files in the
following locations:
VIRTUAL_ENV
defined when virtualenv is started
source bin/activation
•/etc/
•/etc/seedbox/
•~/VIRTUAL_ENV/etc/
•~/VIRTUAL_ENV/etc/seedbox/
•~/
•~/.seedbox/
•./ (current working directory)
seedbox.options.list_opts()
Returns a list of oslo.config options available in the library.
The returned list includes all oslo.config options which may be registered at runtime by the library.
Each element of the list is a tuple. The first element is the name of the group under which the list of elements in
the second element will be registered. A group name of None corresponds to the [DEFAULT] group in config
files.
The purpose of this is to allow tools like the Oslo sample config file generator to discover the options exposed
to users by this library.
Returns a list of (group_name, opts) tuples
18 Chapter 3. Contents
seedbox, Release
class seedbox.common.timeutil.AfterDelta(method)
Bases: object
@AfterDelta
A decorator that has state and handles checking if period of time
(delta) has passed before executing the function. That way if you
need to control whether a piece of code executes in a loop, the delta
is applied before execution otherwise NOOP
DEFAULT_DELTA = 60
get_delta()
Retrieves the delta (period of time to wait between executions)
Note: To change the delta simply subclass and override this one method, and use the subclass name as
the decorator
seedbox.common.timeutil.advance_time_delta(dt, timedelta)
Advances a datetime by a datetime.timedelta
Parameters
• dt (datetime) – a specified date time
• timedelta (datetime.timedelta) – time offset (delta)
Returns a datetime incremented by delta
Return type datetime
seedbox.common.timeutil.advance_time_seconds(dt, seconds)
Advances a datetime by a seconds
Parameters
• dt (datetime) – a specified date time
• seconds (int) – seconds (delta)
Returns a datetime incremented by seconds
Return type datetime
seedbox.common.timeutil.delta_seconds(before, after)
Return the difference between two timing objects.
Compute the difference in seconds between two date, time, or datetime objects (as a float, to microsecond
resolution).
Parameters
• before – date, datetime, time object
• after – date, datetime, time object
Returns difference in seconds
Return type int
seedbox.common.timeutil.is_newer_than(after, seconds)
Checks if a datetime is newer than seconds
Parameters
• after (datetime) – a datetime to check
• seconds (int) – seconds (delta)
Returns True if before is newer than seconds else False
Return type bool
seedbox.common.timeutil.is_older_than(before, seconds)
Checks if a datetime is older than seconds
Parameters
• before (datetime) – a datetime to check
• seconds (int) – seconds (delta)
Returns True if before is older than seconds else False
Return type bool
seedbox.common.timeutil.is_soon(dt, window)
Determines if time is going to happen in the next window seconds.
Parameters
• dt – the time
• window – minimum seconds to remain to consider the time not soon
Returns True if expiration is within the given duration
seedbox.common.timeutil.nvl_date(dt, default=None)
Determines if the provided date, time, or datetime has a value, and returns the provided value back or the value
of default (current time)
Parameters
• dt – an instance of a date, time, or datetime
• default – value to return if provided dt has no value
Returns date, time, or datetime provided or default
Return type datetime
seedbox.common.timeutil.time_delta_seconds(seconds)
Retrieves a timedelta
Parameters seconds (int) – delta of seconds
Returns timedelta by seconds
Return type timedelta
seedbox.common.timeutil.timed(method=None, logger=None, loglvl=None)
@seedbox.common.timeutil.timed(logger, loglvl)
A decorator that times the execution of a method/function and logs
using the supplied logger at the specified loglevel.
Parameters
• method – the method being timed
20 Chapter 3. Contents
seedbox, Release
seedbox.common.timeutil.total_seconds(delta)
Return the total seconds of datetime.timedelta object.
Compute total seconds of datetime.timedelta, datetime.timedelta doesn’t have method total_seconds in
Python2.6, calculate it manually.
Parameters delta (timedelta) – a delta to convert
Returns seconds
Return type int
seedbox.common.timeutil.utcnow()
Returns current time from utc
Return type datetime
22 Chapter 3. Contents
seedbox, Release
classmethod pk_filter(value=None)
Creates a primary key filter for the given model.
Parameters value – primary key value
Returns primary key filter
Return type dict
class seedbox.db.models.Torrent(torrent_id, name, created_at=None, updated_at=None,
state=None, retry_count=None, failed=None, error_msg=None,
invalid=None, purged=None, media_files=None)
Bases: seedbox.db.models.Model
Represents the attributes of a torrent and the associated state based on parsing and processing of the torrent.
Parameters
• torrent_id (int) – primary key identifier of torrent
• name (str) – name of the torrent file
• created_at (datetime.datetime) – date when db entry created
• updated_at (datetime.datetime) – date when db entry updated
• state (str) – current state of processing of torrent (init, ready, active, done, cancelled)
• retry_count (int) – number of times entry was reprocessed
• failed (bool) – flag indicating processing failed
• error_msg (str) – error message generated during processing
• invalid (bool) – flag indicating entry was invalid
• purged (bool) – flag indicating entry details were purged
• media_files (list) – list of associated media file(s)
Returns an instance of the Torrent object
Return type Torrent
PK_NAME = ‘torrent_id’
Provides access to the database API for interacting with the torrent data.
seedbox.db.dbapi(conf=<oslo.config.cfg.ConfigOpts object at 0x7fc9ddb11a90>)
Retrieves an instance of the configured database API.
Parameters conf (oslo.config.cfg.ConfigOpts) – an instance of the configuration file
Returns database API instance
Return type DBApi
seedbox.db.list_opts()
Returns a list of oslo.config options available in the library.
The returned list includes all oslo.config options which may be registered at runtime by the library.
Each element of the list is a tuple. The first element is the name of the group under which the list of elements in
the second element will be registered. A group name of None corresponds to the [DEFAULT] group in config
files.
The purpose of this is to allow tools like the Oslo sample config file generator to discover the options exposed
to users by this library.
Returns a list of (group_name, opts) tuples
Public facing database api that leverages a provided implementation to execute database operations.
class seedbox.db.api.DBApi(impl)
Bases: object
Provides API for supported database operations that leverage specified implementation.
Parameters impl (Connection) – database plugin instance
backup()
Backup database.
bulk_create_medias(medias)
Perform save (insert) operation on a list of instances of media.
Parameters medias – a list of instances of media file
Returns media file instance(s)
Return type MediaFile
bulk_save_torrents(value_map, qfilter)
Performs save (update) operation on multiple instances of torrents using the provided data map.
Parameters
• value_map – data attributes and values to update
• qfilter – query filter to determine instances to update
clean_up()
Periodically check for data no longer needed (fully processed, invalid, or deleted) and then remove from
the database.
clear()
Clears all data from database.
delete_appstate(appstate)
Perform delete operation on an instance of appstate.
Parameters appstate – an instance of appstate
delete_media(media)
Perform delete operation on an instance of media.
Parameters media – an instance of media file
24 Chapter 3. Contents
seedbox, Release
delete_medias(qfilter)
Perform delete operation on a list of instances of media.
Parameters qfilter – query filter to determine instances to delete
delete_torrent(torrent)
Performs delete operation on specific instance of torrent.
Parameters torrent – an instance of torrent
delete_torrents(qfilter)
Performs delete operation on selection of torrent instances.
Parameters qfilter – query filter to determine instances to delete.
fetch_or_create_torrent(name)
Performs select operation using the name of torrent to fetch torrent, and if the torrent is not found, then the
save (insert) operation is performed using the name of the torrent.
Parameters name – the name of a torrent
Returns torrent instance(s)
Return type Torrent
get_appstate(name)
Perform select operation using name of appstate to fetch an instance of appstate.
Parameters name – name of an appstate instance
Returns appstate instance
Return type AppState
get_media(media_id)
Perform select operation using media primary key to fetch media.
Parameters media_id – media primary key
Returns media file instance
Return type MediaFile
get_medias(qfilter)
Perform select operation on selection of media instances.
Parameters qfilter – query filter to determine instances to fetch.
Returns media file instance(s)
Return type MediaFile
get_medias_by(torrent_id, file_path=None, compressed=None, synced=None, missing=None,
skipped=None)
Perform select operation using a combination of the provided parameters to find specified media.
Parameters
• torrent_id – primary key of torrent
• file_path – location where media exist on file system (default: None; ignore attribute)
• compressed – flag to indicate to include or exclude compressed media (default: None;
ignore attribute)
• synced – flag to indicate to include or exclude synced media ( default: None; ignored
attribute)
• missing – flag to indicate to include or exclude missing media (default: None; ignored
attribute)
• skipped – flag to indicate to include or exclude skipped media (default: None; ignored
attribute)
Returns media file instance(s)
Return type MediaFile
get_medias_by_torrent(torrent_id)
Perform select operation using the primary key of torrent to fetch associated media.
Parameters torrent_id – primary key of torrent
Returns media file instance(s)
Return type MediaFile
get_processed_medias(torrent_id)
Performed select operation using torrent primary key and pre-defined criteria for what constitutes pro-
cessed media.
Parameters torrent_id – torrent primary key
Returns media file instance(s)
Return type MediaFile
get_torrent(torrent_id)
Perform select operation using the primary key of torrent to fetch torrent.
Parameters torrent_id – primary key of torrent
Returns torrent instance(s)
Return type Torrent
get_torrent_by_name(name)
Perform select operation using the name of torrent to fetch torrent.
Parameters name – the name of torrent
Returns torrent instance(s)
Return type Torrent
get_torrents(qfilter)
Perform select operation on selection of torrent instances.
Parameters qfilter – query filter to determine instances to fetch.
Returns torrent instance(s)
Return type Torrent
get_torrents_active()
Perform select operation using a pre-defined criteria for what constitutes an active torrent.
Returns torrent instance(s)
Return type Torrent
get_torrents_by_state(state, failed=False)
Perform select operation using a pre-defined criteria to get torrents by a specific state.
Parameters
26 Chapter 3. Contents
seedbox, Release
bulk_create(instances)
Save the instances in bulk to the database. :param instances: a list of instance of modeled data object
bulk_update(value_map, entity_type, qfilter)
Save the updated instances in bulk to the database. :param value_map: a dict of key-value pairs represent-
ing the data of an instance. :param entity_type: the model type :param qfilter: query filter to determine
which rows to update
clear()
Clear database.
delete(instance)
Delete the instance(s) based on filter from the database. :param instance: an instance of modeled data
object
delete_by(entity_type, qfilter)
Delete instances of a specific type based on filter criteria :param entity_type: the model type :param qfilter:
query filter to determine which rows to update
fetch(entity_type, pk)
Fetch the instance(s) based on filter from the database. :param entity_type: the model type :param pk:
primary key value
fetch_by(entity_type, qfilter)
Fetch the instance(s) based on filter from the database. :param entity_type: the model type :param qfilter:
query filter to determine which rows to update
save(instance)
Save the instance to the database. :param instance: an instance of modeled data object
shrink_db()
Shrink database.
upgrade()
Migrate the database to version or most recent version.
Wrapper for sandman; replacement for sandmanctl that provides additional options not available via sandmanctl
seedbox.db.admin.print_version(ctx, value)
Print the current version of sandman and exit. :param ctx: application context :param value:
28 Chapter 3. Contents
seedbox, Release
seedbox.db.sqlalchemy.session.get_maker(engine)
Return a SQLAlchemy sessionmaker using the given engine.
Parameters engine – a database connection engine
seedbox.db.sqlalchemy.session.receive_connect(dbapi_con, con_record)
Ensures that the foreign key constraints are enforced in SQLite.
The foreign key constraints are disabled by default in SQLite, so the foreign key constraints will be enabled here
for every database connection
Parameters
• dbapi_con – database connection
• con_record – connection record
30 Chapter 3. Contents
seedbox, Release
t_datetime
t_int
t_string
update(values)
Make the model object behave like a dict.
Parameters values – key-value pairs to update in database
class seedbox.db.sqlalchemy.models.HasId
Bases: object
Table mixin providing a class/table id attribute
id = Column(None, Integer(), table=None, primary_key=True, nullable=False)
class seedbox.db.sqlalchemy.models.HasTimestamp
Bases: object
Table mixin providing a class/table date attributes
created_at = Column(None, DateTime(), table=None, default=ColumnDefault(<function <lambda> at 0x7fc9dc68ecf8>
updated_at = Column(None, DateTime(), table=None, onupdate=ColumnDefault(<function <lambda> at 0x7fc9dc68ed
class seedbox.db.sqlalchemy.models.MediaFile(**kwargs)
Bases: sqlalchemy.ext.declarative.api.Base, seedbox.db.sqlalchemy.models.HasId
Class representing a media file in the database
A simple constructor that allows initialization from kwargs.
Sets attributes on the constructed instance using the names and values in kwargs.
Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any
mapped columns or relationships.
compressed
error_msg
file_ext
file_path
filename
id
missing
size
skipped
synced
torrent_id
total_time
class seedbox.db.sqlalchemy.models.QueryTransformer(table, query)
Bases: object
Provides the ability to transform a query filter into database query in the sqlalchemy compliant manner.
Parameters
32 Chapter 3. Contents
seedbox, Release
34 Chapter 3. Contents
seedbox, Release
Initial version of the database model; maintained to support anyone upgrading from a previous version.
seedbox.db.sqlalchemy.migrate_repo.versions.001_initial_tables.downgrade(migrate_engine)
Not implemented because this is associated with initial version.
Parameters migrate_engine – an instance of database connection engine
Raises NotImplementedError if method executed
seedbox.db.sqlalchemy.migrate_repo.versions.001_initial_tables.upgrade(migrate_engine)
Creates the initial version of the database tables.
Parameters migrate_engine – an instance of database connection engine
Provides removing the initial version of the database tables for clean up purposes.
seedbox.db.sqlalchemy.migrate_repo.versions.003_drop_V1_tables.downgrade(migrate_engine)
Does nothing since the ability to go version 1 from 3 is not supported directly.
Parameters migrate_engine – an instance of database connection engine
seedbox.db.sqlalchemy.migrate_repo.versions.003_drop_V1_tables.upgrade(migrate_engine)
Drops the initial version of the tables.
Parameters migrate_engine – an instance of database connection engine
The seedbox.db.sqlalchemy.migrate_repo.versions.002_migrate_to_V2_of_tables
Module
Migrates to version 2 of the database tables and supports downgrading to previous version.
seedbox.db.sqlalchemy.migrate_repo.versions.002_migrate_to_V2_of_tables.downgrade(migrate_eng
Converts back to previous version and removes the newly created tables.
Parameters migrate_engine – an instance of database connection engine
seedbox.db.sqlalchemy.migrate_repo.versions.002_migrate_to_V2_of_tables.upgrade(migrate_engine
Updates table model to new version and moves existing data from old structures to the new structure.
Parameters migrate_engine – an instance of database connection engine
static is_actionable(media_file)
Perform check to determine if action should be taken.
Parameters media_file – an instance of a MediaFile to check
Returns a flag indicating to act or not to act
Return type boolean
Extends the subprocess.Popen class to provide logging threads for stdout and stderr of the child process such that it
will log stdout through logging module to provided logger.info() and log stderr to provided logger.warn().
Because using subprocess.PIPE for stdout and stderr with any of the convenience methods or with the wait() command
has potential for buffer issue, we leverage Popen directly with PIPE but then spawn a thread with access to the
corresponding logger function on the specific pipe. And then leverage the poll method to wait for child process to
complete. If the returncode is not 0, then just like check_call() we will raise the CalledProcessError to be as consistent
as possible. This is done via the staticmethod execute(). Because we subclass Popen, you can simply create the
instance and leverage Popen methods for interaction if you prefer.
The intent here is to have the output send directly to the logger vs. being buffered until the end with the communicate()
method, or dealing with buffer conflict issues like wait(). The logger can handle the different levels, StreamHandler or
even FileHandler and the corresponding subclasses to make it more manageable.
This was derived from several different responses on stackoverflow and blogs on this topic. After pulling them all
together and doing several rounds of testing, this was the result.
36 Chapter 3. Contents
seedbox, Release
class seedbox.tasks.subprocessext.ProcessLogging(cmd)
Bases: subprocess.Popen
Run a command as a subprocess sending output to a logger.
Parameters cmd (list) – command and options sent to subprocess to execute
complete()
Handle all the processing of the subprocess
static execute(cmd)
provide a convenience method for creating the object and waiting for the results; very similar to
check_call() but not at module level.
Parameters cmd (list) – command and options sent to subprocess to execute
Provides the basic definition for a task that handles execution and basic error handling.
class seedbox.tasks.base.BaseTask(media_file)
Bases: object
Provides the base definition of a task.
add_gen_files(files)
Adds media files included within an archived file to be processed separately to increase parallel processing.
Parameters files – a list of media files produced by a plugin to be included on the torrent.
execute()
Perform the action associated with task for the provided media_file.
static is_actionable(media_file)
Perform check to determine if action should be taken.
Parameters media_file – the file to inspect to determine if action is necessary
Returns a flag indicating to act or not to act
Return type boolean
Provides the definition of workflow (steps and transition), and implementation what happens during each step by
binding in the provided plugins for each and updating the db cache after each step.
class seedbox.process.flow.BaseFlow(torrent)
Bases: xworkflows.base.WorkflowEnabled
Provides the base workflow implementation on binding plugin tasks to each step and determining which plugin
is capable of operating on the media files of the specified torrent.
Parameters torrent (Torrent) – an instance of a parsed torrent metadata
activate
Executes all the tasks for a torrent at activate phase
cancel
NoOp function, ignores all arguments.
38 Chapter 3. Contents
seedbox, Release
complete
Executes all the tasks for a torrent at complete phase
is_done()
Checks if the current state of workflow is either done or cancelled.
Returns flag indicating workflow is done
Return type boolean
next_tasks()
Find the list of tasks and associated media eligible for processing.
Returns list of tasks
Return type generator
phase
The name of current step/phase of the workflow
Returns name of current phase
Return type string
prepare
Executes all the tasks for a torrent at prepare phase
state
Property-like attribute holding the state of a WorkflowEnabled object.
The state is stored in the internal __dict__ of the instance.
tasks
Property for accessing the tasks associated with current workflow step by looking up the configured plug-
ins.
Returns list of tasks (BaseTask)
Return type list
update_state(*args, **kwargs)
Handles the capturing the current state of processing
Parameters
• args – required parameter based on decorator (unused)
• kwargs – required parameter based on decorator (unused)
class seedbox.process.flow.Taskflow
Bases: xworkflows.base.Workflow
Define the workflow conditions for managing torrents;
initial_state = <State: ‘init’>
states = StateList({‘ready’: <State: ‘ready’>, ‘active’: <State: ‘active’>, ‘init’: <State: ‘init’>, ‘done’: <State: ‘done’>,
transitions = TransitionList([Transition(‘cancel’, [<State: ‘ready’>, <State: ‘active’>], <State: ‘cancelled’>), Transitio
seedbox.process.flow.get_tasks(phase)
Gets a list of tasks based the current phase of processing
Parameters phase – the name of the current phase/step of workflow
The core process flow for managing the syncing of torrents to remote location.
seedbox.process.list_opts()
Returns a list of oslo.config options available in the library.
The returned list includes all oslo.config options which may be registered at runtime by the library.
Each element of the list is a tuple. The first element is the name of the group under which the list of elements in
the second element will be registered. A group name of None corresponds to the [DEFAULT] group in config
files.
The purpose of this is to allow tools like the Oslo sample config file generator to discover the options exposed
to users by this library.
Returns a list of (group_name, opts) tuples
seedbox.process.start()
The primary entry point for the process
Workflow implementation that handles automatic execution of each workflow step until the workflow reaches comple-
tion.
wf = Workflow(torrent)
tasks = wf.next_tasks()
# execute tasks in separate threads
<<logic>>
# now move the workflow to the next step
wf.run()
class seedbox.process.workflow.Workflow(torrent)
Bases: seedbox.process.flow.BaseFlow
Wrapper class around Flow that handles the orchestration of the process
activate
Executes all the tasks for a torrent at activate phase
cancel
NoOp function, ignores all arguments.
complete
Executes all the tasks for a torrent at complete phase
prepare
Executes all the tasks for a torrent at prepare phase
run()
Orchestrate each step of the process based on current state
state
Property-like attribute holding the state of a WorkflowEnabled object.
The state is stored in the internal __dict__ of the instance.
40 Chapter 3. Contents
seedbox, Release
Torrent loader handles searches the directory where torrent files are stored and adds them to the database cache if not
already found.
seedbox.torrent.loader.load_torrents()
Find all the torrents in the specified directory, verify it is a valid torrent file (via parsing) and capture the relevant
details. Next create a record in the cache for each torrent.
Parameters x – message
Returns encoded string
Return type string
seedbox.torrent.bencode.bytes(s, *args, **kwargs)
seedbox.torrent.bencode.bytes_index(s, pattern, start)
Returns the index of pattern within string starting at position start.
Parameters
• s (str) – byte string to search for pattern
• pattern (str) – pattern to search for within byte string
• start (int) – position/index within byte string to start scanning
Returns index of pattern within byte string
Return type int
seedbox.torrent.bencode.chr_(s)
Return the string representing a character whose Unicode code point is the integer i.
Parameters s (int) – integer representation of unicode character
Returns string representation of unicode character
Return type str
seedbox.torrent.bencode.ord_(s)
Given a string representing one Unicode character, return an integer representing the Unicode code point of that
character.
Parameters s (str) – string representing single unicode character
Returns integer representation of unicode character
Return type int
Parses a torrent file and provides method to access the following attributes.
• Tracker URL
• Creation date
• Client name, if any
• For each file
– name
– length
– checksum
Created on 2012-03-07
@author: mohanr
exception seedbox.torrent.parser.ParsingError(error_msg)
Bases: exceptions.Exception
Error class representing errors that occur while parsing the torrent content.
42 Chapter 3. Contents
seedbox, Release
class seedbox.torrent.parser.TorrentParser(torrent_file_path)
Bases: object
Parses a torrent file and returns various properties based on the content of the torrent file.
Note: bencode is supremely more efficient parser of torrents but extremely strict in the format of the file.
The custom parser is an implementation that someone wrote that does a good job of parsing but it is not very
efficient. So we are going to pair the two solutions together. when the file is well formed we will leverage most
efficient standard but we will leverage the custom parser when bencode generates an exception. If the custom
parser fails then it is really invalid file and the consumer will handle it correspondingly.
Reads the torrent file and sets the content as an object attribute.
Todo
Investigate merging logic from custom parser into bencode code base such that a single unit of code supports
parsing.
DICT_KEY_VALUE_SEP = ‘: ‘
DICT_LIST_END = ‘e’
DICT_LIST_ITEM_SEP = ‘, ‘
DICT_START = ‘d’
INT_START = ‘i’
LIST_START = ‘l’
get_client_name()
Returns the name of the client that created the torrent if present, from the parsed torrent file.
Returns name of the client that created the torrent
Return type str
get_creation_date(time_format=’iso’)
Retrieves creation date of the torrent, if present, in ISO time_format from the parsed torrent file.
Parameters time_format (str) – determines the time_format of the time value returned. [’iso’,
‘datetime’] defaults: ‘iso’
Returns creation date
Return type datetime
get_files_details()
Parses torrent file and returns details of the files contained in the torrent.
File details tuple:
• name
• length (size)
• checksum (of file in the torrent)
get_tracker_url()
Retrieves tracker URL from the parsed torrent file
Returns tracker URL from parsed torrent file
Return type str
seedbox package
seedbox.torrent.list_opts()
Returns a list of oslo.config options available in the library.
The returned list includes all oslo.config options which may be registered at runtime by the library.
Each element of the list is a tuple. The first element is the name of the group under which the list of elements in
the second element will be registered. A group name of None corresponds to the [DEFAULT] group in config
files.
The purpose of this is to allow tools like the Oslo sample config file generator to discover the options exposed
to users by this library.
Returns a list of (group_name, opts) tuples
seedbox.torrent.load()
Simple wrapper to provide parsing and loading torrents into a database cache.
#
# Options defined in seedbox.options
#
[database]
#
# Options defined in seedbox.db
#
44 Chapter 3. Contents
seedbox, Release
[process]
#
# Options defined in seedbox.process.flow
#
#
# Options defined in seedbox.process.manager
#
[tasks]
#
# Options defined in seedbox.tasks.base
#
[tasks_filesync]
#
# Options defined in seedbox.tasks.filesync
#
[tasks_synclog]
#
# Options defined in seedbox.tasks.subprocessext
#
[torrent]
#
# Options defined in seedbox.torrent
#
46 Chapter 3. Contents
seedbox, Release
48 Chapter 3. Contents
CHAPTER 4
• genindex
• modindex
• search
49
seedbox, Release
s seedbox.tasks.filedelete, 38
seedbox, 17 seedbox.tasks.filesync, 36
seedbox.cli, 17 seedbox.tasks.fileunrar, 37
seedbox.common, 18 seedbox.tasks.subprocessext, 36
seedbox.common.timeutil, 18 seedbox.torrent, 44
seedbox.common.tools, 18 seedbox.torrent.bencode, 41
seedbox.constants, 18 seedbox.torrent.loader, 41
seedbox.db, 23 seedbox.torrent.parser, 42
seedbox.db.admin, 28 seedbox.version, 17
seedbox.db.api, 24
seedbox.db.base, 27
seedbox.db.exception, 21
seedbox.db.maintenance, 24
seedbox.db.models, 21
seedbox.db.sqlalchemy, 33
seedbox.db.sqlalchemy.api, 33
seedbox.db.sqlalchemy.migrate_repo, 35
seedbox.db.sqlalchemy.migrate_repo.manage,
34
seedbox.db.sqlalchemy.migrate_repo.versions,
35
seedbox.db.sqlalchemy.migrate_repo.versions.001_initial_tables,
35
seedbox.db.sqlalchemy.migrate_repo.versions.002_migrate_to_V2_of_tables,
35
seedbox.db.sqlalchemy.migrate_repo.versions.003_drop_V1_tables,
35
seedbox.db.sqlalchemy.migration, 33
seedbox.db.sqlalchemy.model_util, 28
seedbox.db.sqlalchemy.models, 30
seedbox.db.sqlalchemy.session, 29
seedbox.logext, 17
seedbox.options, 17
seedbox.process, 40
seedbox.process.flow, 38
seedbox.process.manager, 41
seedbox.process.workflow, 40
seedbox.tasks, 37
seedbox.tasks.base, 37
seedbox.tasks.filecopy, 35
51