issho package

Submodules

issho.cli module

class issho.cli.IsshoCLI[source]

Bases: object

CLI for Issho; right now only used for configuration

config(profile, env=None, ssh_profile='', ssh_config='~/.ssh/config', rsa_id='~/.ssh/id_rsa')[source]

Configures a single issho profile. Saves non-private variables to ~/.issho/conf.toml and passwords to the local keyring.

Parameters
  • profile – name of the profile to configure

  • env – Optional environment variable profile to draw from.

  • ssh_profile – The name of the associated ssh config profile; defaults to the profile name if not supplied.

  • ssh_config – the path to the ssh_config to be used for this profile

  • rsa_id – the path to the id_rsa file to be used for this profile

static env(env_name)[source]

Saves a set of variables to ~/.issho/envs.toml :param env_name: name of the environment to set up or update

static test_connection(profile, kinit=True)[source]

Tests the connection to the specified :param profile: The name of the profile :param kinit: if True, will try to kinit using the stored password :return:

static update_variable(profile, variable, value)[source]

Updates or add a single profile variable.

issho.cli.main()[source]

Inititates the CLI using python-fire

issho.config module

issho.config.read_issho_conf(profile, filename=PosixPath('/home/docs/.issho/conf.toml'))[source]

Writes issho variables out to a .toml file.

Parameters
  • profile – The name of the profile to read

  • filename – The output filename

Returns

a dict of data stored with that profile in the configuration file

issho.config.read_issho_env(profile)[source]

Reads issho environment variables to a dict :param profile: the name of the issho environment to draw from :return: a dict of data with that profile stored in the environment file

issho.config.read_ssh_profile(ssh_config_path, profile)[source]

Helper method for getting data from .ssh/config

issho.config.write_issho_conf(new_conf_dict, filename=PosixPath('/home/docs/.issho/conf.toml'))[source]

Updates the issho config file :param new_conf_dict: the new configuration to add :param filename: the location of the old configuration file

issho.config.write_issho_env(new_env_dict)[source]

Save a new issho environment :param new_env_dict: the new set of environment paramters to add

issho.helpers module

issho.helpers.able_to_connect(host, port, timeout=1.5)[source]

Returns true if it is possible to connect to the specified host and port, within the given timeout in seconds.

issho.helpers.absolute_path(raw_path)[source]

Gets the string absolute path from a path object or string.

Parameters

raw_path – a string or pathlib.Path object

issho.helpers.add_arguments_to_cmd(cmd, *args)[source]
issho.helpers.clean_spark_options(spark_options)[source]
issho.helpers.default_sftp_path(this_path, default_path)[source]

If this_path exists, return it as a path, else, return the pathlib.Path.name of the default path

issho.helpers.get_pkey(key_path)[source]

Helper for getting an RSA key

issho.helpers.get_user()[source]
issho.helpers.issho_pw_name(pw_type, profile)[source]

Helper for standardizing password names

issho.helpers.issho_ssh_pw_name(rsa_id)[source]

Helper for standardizing ssh password names

issho.issho module

Implementation for the Issho class, which implements a connection and some simple commands over ssh, using keyring to manage secrets locally.

class issho.issho.Issho(profile='dev', kinit=True)[source]

Bases: object

exec(cmd, *args, bg=False, debug=False, capture_output=False)[source]

Execute a command in bash over the SSH connection.

Note, this command does not use an interactive terminal; it instead uses a non-interactive login shell. This means (specifically) that your aliased commands will not work and only variables exported in your remote .bashrc will be available.

Parameters
  • cmd – The bash command to be run remotely

  • *args

    Additional arguments to the command cmd

  • bg – True = run in the background

  • debug – True = print some debugging output

  • capture_output – True = return stdout as a string

Returns

exec_bg(cmd, *args, **kwargs)[source]

Syntactic sugar for exec(bg=True)

get(remotepath, localpath=None, hadoop=False)[source]

Gets the file at the remote path and puts it locally.

Parameters
  • remotepath – The path on the remote from which to get.

  • localpath – Defaults to the name of the remote path

  • hadoop – Download from HDFS

get_output(cmd, *args, **kwargs)[source]

Syntactic sugar for exec(capture_output=True)

hadoop(command, *args, **kwargs)[source]

Execute the hadoop command :param command: :param args: :param kwargs: :return:

hdfs(*args, **kwargs)[source]

Syntactic sugar for hadoop

hive(query, output_filename=None, remove_blank_top_line=True)[source]

Runs a hive query using the parameters set in .issho/config.toml

Parameters
  • query – a string query, or the name of a query file name to run.

  • output_filename – the (local) file to output the results of the hive query to. Adding this option will also keep a copy of the results in /tmp

  • remove_blank_top_line – Hive usually has a blank top line when data is output, this parameter removes it.

kinit()[source]

Runs kerberos init

local_forward(remote_host, remote_port, local_host='0.0.0.0', local_port=44556)[source]

Forwards a port from a remote through this Issho object. Useful for connecting to remote hosts that can only be accessed from inside a VPC of which your devbox is part.

put(localpath, remotepath=None, hadoop=False)[source]

Puts the file at the local path to the remote.

Parameters
  • localpath – The local path of the file to put to the remote

  • remotepath – Defaults to the name of the local path

  • hadoop – Upload to HDFS

spark(*args, **kwargs)[source]

Syntactic sugar for spark_submit

spark_submit(spark_options=None, master='', jars='', files='', driver_class_path='', application_class='', application='', application_args='')[source]

Submit a spark job.

Parameters
  • spark_options – A dict of spark options

  • master – syntactic sugar for the –master spark option

  • jars – syntactic sugar for the –jars spark option

  • files – syntactic sugar for the –files spark option

  • driver_class_path – syntactic sugar for the –driver-class-path spark option

  • application_class – syntactic sugar for the –class spark option

  • application – the application to submit

  • application_args – any arguments to be passed to the spark application

Returns

Module contents

issho - simple connections to remote machines

issho is a Python package providing a simple wrapper over paramiko, providing = operators interacting with remote machines

Main Features

Here are a few of the things that issho (should) do well:
  • execute commands on a remote box

  • transfer files to and from a remote easily

  • set up an SSH tunnel through a remote

  • run Hive & Spark jobs

TODOs

  • make it easy to interact with hadoop

  • make it easy to configure new services

  • make it easy to add plugins to issho