issho package¶
Submodules¶
issho.cli module¶
-
class
issho.cli.
IsshoCLI
[source]¶ Bases:
object
CLI for Issho; right now only used for configuration
-
config
(profile, env=None, ssh_profile='', ssh_config='~/.ssh/config', rsa_id='~/.ssh/id_rsa')[source]¶ Configures a single issho profile. Saves non-private variables to
~/.issho/conf.toml
and passwords to the local keyring.- Parameters
profile – name of the profile to configure
env – Optional environment variable profile to draw from.
ssh_profile – The name of the associated ssh config profile; defaults to the profile name if not supplied.
ssh_config – the path to the ssh_config to be used for this profile
rsa_id – the path to the id_rsa file to be used for this profile
-
static
env
(env_name)[source]¶ Saves a set of variables to ~/.issho/envs.toml :param env_name: name of the environment to set up or update
-
issho.config module¶
-
issho.config.
read_issho_conf
(profile, filename=PosixPath('/home/docs/.issho/conf.toml'))[source]¶ Writes issho variables out to a
.toml
file.- Parameters
profile – The name of the profile to read
filename – The output filename
- Returns
a dict of data stored with that profile in the configuration file
-
issho.config.
read_issho_env
(profile)[source]¶ Reads issho environment variables to a dict :param profile: the name of the issho environment to draw from :return: a dict of data with that profile stored in the environment file
-
issho.config.
read_ssh_profile
(ssh_config_path, profile)[source]¶ Helper method for getting data from .ssh/config
issho.helpers module¶
-
issho.helpers.
able_to_connect
(host, port, timeout=1.5)[source]¶ Returns true if it is possible to connect to the specified host and port, within the given timeout in seconds.
-
issho.helpers.
absolute_path
(raw_path)[source]¶ Gets the string absolute path from a path object or string.
- Parameters
raw_path – a string or
pathlib.Path
object
issho.issho module¶
Implementation for the Issho
class, which implements
a connection and some simple commands over ssh
, using
keyring
to manage secrets locally.
-
class
issho.issho.
Issho
(profile='dev', kinit=True)[source]¶ Bases:
object
-
exec
(cmd, *args, bg=False, debug=False, capture_output=False)[source]¶ Execute a command in bash over the SSH connection.
Note, this command does not use an interactive terminal; it instead uses a non-interactive login shell. This means (specifically) that your aliased commands will not work and only variables exported in your remote
.bashrc
will be available.- Parameters
cmd – The bash command to be run remotely
*args –
Additional arguments to the command cmd
bg – True = run in the background
debug – True = print some debugging output
capture_output – True = return stdout as a string
- Returns
-
get
(remotepath, localpath=None, hadoop=False)[source]¶ Gets the file at the remote path and puts it locally.
- Parameters
remotepath – The path on the remote from which to get.
localpath – Defaults to the name of the remote path
hadoop – Download from HDFS
-
hadoop
(command, *args, **kwargs)[source]¶ Execute the hadoop command :param command: :param args: :param kwargs: :return:
-
hive
(query, output_filename=None, remove_blank_top_line=True)[source]¶ Runs a hive query using the parameters set in .issho/config.toml
- Parameters
query – a string query, or the name of a query file name to run.
output_filename – the (local) file to output the results of the hive query to. Adding this option will also keep a copy of the results in /tmp
remove_blank_top_line – Hive usually has a blank top line when data is output, this parameter removes it.
-
local_forward
(remote_host, remote_port, local_host='0.0.0.0', local_port=44556)[source]¶ Forwards a port from a remote through this Issho object. Useful for connecting to remote hosts that can only be accessed from inside a VPC of which your devbox is part.
-
put
(localpath, remotepath=None, hadoop=False)[source]¶ Puts the file at the local path to the remote.
- Parameters
localpath – The local path of the file to put to the remote
remotepath – Defaults to the name of the local path
hadoop – Upload to HDFS
-
spark_submit
(spark_options=None, master='', jars='', files='', driver_class_path='', application_class='', application='', application_args='')[source]¶ Submit a spark job.
- Parameters
spark_options – A dict of spark options
master – syntactic sugar for the –master spark option
jars – syntactic sugar for the –jars spark option
files – syntactic sugar for the –files spark option
driver_class_path – syntactic sugar for the –driver-class-path spark option
application_class – syntactic sugar for the –class spark option
application – the application to submit
application_args – any arguments to be passed to the spark application
- Returns
-
Module contents¶
issho - simple connections to remote machines¶
issho is a Python package providing a simple wrapper over paramiko, providing = operators interacting with remote machines
Main Features¶
- Here are a few of the things that issho (should) do well:
execute commands on a remote box
transfer files to and from a remote easily
set up an SSH tunnel through a remote
run Hive & Spark jobs
TODOs¶
make it easy to interact with hadoop
make it easy to configure new services
make it easy to add plugins to issho