mc_remote / remote execution functions

The following functions are related to do remote executions over ssh transport. This for both raw commands and local salt executions. Those functions are just variations from salt.utils.cloud (which i (kiorky) also helped to wrote for some parts, like... the remote exec ones :))

This have nice features like:

  • handling interactive passwords
  • using ssh gateways
  • transfering files using different fallback methods
  • running salt-call over remote (even masterless) salt installations

If this module gains salt core, there are some small makina-states deps:

  • mc_states.renderers.lyaml
  • mc_states.modules.mc_utils.magicstring (which needs chardet)
mc_states.modules.mc_remote.delete_remote(host, filepath, mode='-f', level='info', **kw)

Delete a remote file (or directory)

mc_states.modules.mc_remote.highstate(host, outputter='json', transformer='highstate', strip_out=True, **kw)

Run an highstate on an host and fails on error kwargs are forwarded to ssh helper functions !

host
host to connect onto
ssh_username/user (first win)
user to connect as (default: root)
ssh_port/port (first win)
Port to connect onto (default: 22)
sls
sls to execute

CLI Examples:

salt-call --local mc_remote.highstate foo.net
mc_states.modules.mc_remote.interactive_ssh(cmd, **kw)

Establish a ssh connection layer, executes a command, interact.

This session can can be password interactive and we will by default a password challenge and forward the result to the user.

The session control behavior can be controller by subclassing the AbstractSshSession class, see ‘ssh_interaction_class’:

cmd

the full ssh command to wrap the execution from

eg:

ssh foo.com "ls /"
ssh_quote
do we force the script to be quoted
ssh_display_ssh_output
do we send output on the console
ssh_output_loglevel
log level
ssh_password_retries
how many retries do we try for the password
ssh_interaction_class
_AbstractSshSession subclass(not instance) (_SSHPasswordChallenger by default)
timeout
timeout for command execution
mc_states.modules.mc_remote.local_salt_call(*a, **kw)

Execute salt-call locally, in another shell see salt-call

mc_states.modules.mc_remote.run(host, script, **kw)

Wrapper to ssh but get only stdout

kwargs are forwarded to ssh helper functions !

returning only the code exist status

mc_states.modules.mc_remote.salt_call(host, fun=None, arg=None, kwarg=None, outputter='json', transformer=None, unparse=True, loglevel='info', salt_call_bin='salt-call', masterless=None, minion_id='local', salt_call_script=None, strip_out=None, hard_failure=False, remote=None, use_vt=None, new_shell=None, ttl=0, *args, **kwargs)

Executes a salt_call call remotely via ssh

host
host to execute on
ssh_username/user (first win)
user to connect as (default: root)
ssh_port/port (first win)
Port to connect onto (default: 22)
fun
saltcall function to call
arg
args for the saltcall function
kwarg
kwargs for the saltcall function
outputter
outputter for the saltcall return
transformer
outputter used to unparse the value returned from the call (the “2nd pass”)
unparse
unserialise the return and tries to split out the local result from regular salt call
loglevel
loglevel to use
vt_loglevel
loglevel to use for vt
salt_call_bin
binary to run
masterless
do we run masterless (–local)
salt_call_script
override default salt call wrapper shell script
remote
use salt locally (you must set host to None !
new_shell

if we execute locally, new_shell can be set to false to execute directly the salt function instead of calling a new shell to call the function, this can be used in conjunction with ttl to cache results easily, eg

salt-call --local -lall mc_remote.salt_call fun=test.ping \
   host=127.0.0.1 new_shell=False ttl=60
use_vt
When ran locally, use use_vt to stream output
args
are appended to arg as arguments to the called salt function
kwargs
They are forwarded to ssh helper functions !
ttl
Use filecache based execution (the result will be cached for X seconds). If the cache is not expired, the result will be used and the function wont be executed.

CLI Examples:

salt-call --local mc_remote.salt_call \
        foo.net test.ping

salt-call --local mc_remote.salt_call \
        foo.net cmd.run \
        'for i in $(seq 5);do echo $i;sleep 1;done' \
        kwarg='{use_vt: True, python_shell: True, user: ubuntu}' \
        ssh_gateway=127.0.0.1 port=40007

salt-call --local mc_remote.salt_call \
        foo.net cmd.run \
        'for i in $(seq 2);do echo $i;sleep 1;done;echo é' \
        kwarg='{use_vt: True, python_shell: True, user: ubuntu}' \
        outputter=yaml

salt-call --local mc_remote.salt_call \
        foo.net cmd.run \
        'for i in $(seq 2);do echo $i;sleep 1;done;echo é' \
        kwarg='{use_vt: True, python_shell: True, user: ubuntu}' \
        unparse=False outputter=yaml
mc_states.modules.mc_remote.sls_(host, sls, outputter='json', transformer='highstate', strip_out=True, **kw)

Run a state file on an host and fails on error kwargs are forwarded to ssh helper functions !

host
host to connect onto
ssh_username/user (first win)
user to connect as (default: root)
ssh_port/port (first win)
Port to connect onto (default: 22)
sls
sls to execute

CLI Examples:

salt-call --local mc_remote.sls foo.net mysls
mc_states.modules.mc_remote.ssh(host, script, **kwargs)

Executes a script command remotly via ssh Attention, if you use a gateway, only key auth is possible on the gw Please also look ssh_kwargs

host
host to execute the script on
ssh_username/user (first win)
user to connect as (default: root)
ssh_port/port (first win)
Port to connect onto (default: 22)
tmpdir
tempfile to upload script to (default to /tmp, this mountpoint must not have the -noexec mount flag)
script

script or command to execute:

  • if the script contains multiple lines We put the content in a temporary file, as-is before uploading it
  • If the script is a filepath we upload it as-is
  • In other cases we wrap it in a simple shell wrapper before uploading
vt_loglevel
loglevel to use for vt

Even in case of a command, it will be wrapped before execution to ease shell quoting

Any extra keywords parameters will by forwarded to:
_get_ssh_args
(see doc) to mangle connection details
interactive_ssh(& ssh_interaction_class)
(see doc) to interact during ssh session

CLI Examples:

salt-call mc_remote.ssh foo.net ssh_gateway=127.0.0.1 port=40007 \
        "cat /etc/hostname" user=mytest password=secret

salt-call mc_remote.ssh foo.net \
        "cat /etc/hostname"
mc_states.modules.mc_remote.ssh_kwargs(first_argument_kwargs=None, **kw)

Lookup & sanitize input in kwargs to have only one value for various & well known SSH connection parameters & other related to mc_remote.

The resulting structure is an well known interface dict usable and used in all this module functions.

All kwargs will be lookup in the form ssh_param and then param.

Eg: ssh_username, if no value, user, if not value, default.

This supports for now:

user
root
tty
use ssh -t (true)
port
22
key_filename
~/id_{rsa,dsa} if existing else ~/.ssh/id_rsa
gateway_key
~/id_{rsa,dsa} if existing else ~/.ssh/id_rsa
gateway
gateway host/ip if any
gateway_user
root
gateway_port
22
password
clear password if any (None)
password_retries
how many do we retry a failed password (3)
known_hosts_file
known_hosts_file if any (/dev/null)
host_key_checking
toggle host checking (no)
makedirs
When transfering files or the shell wrapper, are we allowed to create the conttainer(s) (false)
quote
quote commands (false)
progress
display ssh transfer stats (false)
show_running_cmd
flag for extra logs (true)
no_error_log
Do not report execution failure in logs
display_ssh_output
flag to stream output streams (true)
mc_states.modules.mc_remote.ssh_retcode(host, script, **kw)

Wrapper to ssh

kwargs are forwarded to ssh helper functions !

returning only the code exist status

mc_states.modules.mc_remote.ssh_transfer_dir(host, orig, dest=None, **kwargs)

Transfer directories to an host via ssh layer Please also look ssh_kwargs

This will try then fallback on next transfer method. In order we try:

  • rsync
  • scp
  • sftp
host
host to tranfer to
ssh_username/user (first win)
user to connect as
ssh_port/port (first win)
Port to connect onto
orig
filepath to transfer
dest
where to upload, defaults to orig
makedirs
create parents if any
vt_loglevel
vt loglevel
progress
activate transfer progress
Any extra keywords parameters will by forwarded to:
_get_ssh_args
(see doc) to mangle connection details
interactive_ssh(& ssh_interaction_class)
(see doc) to interact during ssh session
mc_states.modules.mc_remote.ssh_transfer_file(host, orig, dest=None, **kwargs)

Transfer files to an host via ssh layer Please also look ssh_kwargs

This will try then fallback on next transfer method. In order we try:

  • rsync
  • scp
  • sftp
  • gzip piped to dest host gunzip
  • cat piped to dest host uncat
host
host to tranfer to
ssh_username/user (first win)
user to connect as
ssh_port/port (first win)
Port to connect onto
orig
filepath to transfer
dest
where to upload, defaults to orig
makedirs
create parents if any
vt_loglevel
vt loglevel
progress
activate transfers statsv
display_content_on_error
show script on error
Any extra keywords parameters will by forwarded to:
_get_ssh_args
(see doc) to mangle connection details
interactive_ssh(& ssh_interaction_class)
(see doc) to interact during ssh session
mc_states.modules.mc_remote.yamldump_arg(arg, default_flow_style=True, line_break='\n', strip=True)

yaml.safe_dump the arg This is the counterpart of salt.utils.args.yamlify_arg