Technical Notes

Automating Secure FTP (SFTP) Transfers over SSH
Technical Note 1101
Last Reviewed 07-Jun-2007
Applies To
Reflection FTP Client version 11.0 or higher
Summary

The secure shell protocol allows for Secure FTP (SFTP) transfers. This technical note discusses three ways to automate SFTP transfers: using a Reflection FTP client script, programming with the Reflection FTP API, or by running command line ssh utilities.

Note: For information about automating SFTP file transfers using Reflection for Secure IT, see Technical Note 2126.

About Secure FTP

The OpenSSH protocol 2 draft standard includes the Secure FTP (SFTP) protocol for file transfer. SFTP supports fewer commands than the full FTP protocol, but allows all commands and data to be sent through a single, secure channel. This means that all of your communications are encrypted.

You will also need to enable the SFTP subsystem in the host’s sshd configuration file (usually named sshd_config). To verify that this is enabled, view or print the sshd daemon configuration file and look for a line similar to the following:

If the SFTP subsystem is not enabled, refer to the host's documentation for information about enabling the subsystem.

Reflection SFTP Options

Reflection supports automating SFTP transfers using three methods:

Note: Beginning in Reflection 13.0, the SSH connection is called Secure Shell. Earlier Reflection versions called the SSH connection OpenSSH.

Limitations with Multi-File (Wildcarded) Transfers

Historically, SFTP clients and servers do not support wildcards, or MGET or MPUT commands, which facilitate multiple-file transfers. However, you may be able to overcome these limitations, depending on which method you use.

• Method A (FTP Client): Wildcarded uploads are fully functional using the MPUT command. However, MGET will always download all files in a directory due to the lack of server support for wildcards.
• Method B (VBA or VB): Wildcarded uploads and downloads can be achieved. (The referenced example demonstrates downloading all *.txt files using the strFilter variable.)
• Method C (Sftp.exe): Beginning in Reflection 14.0 SP1, wildcarded uploads and downloads (MGET and MPUT) are supported; earlier versions do not support this feature.

Method A – Using a Reflection FTP Client Script

To automate SFTP transfers with the Reflection FTP Client, three steps must be performed:

Step One – Configure an FTP Site to Use SFTP

You can create a new SFTP site or configure an existing FTP site to use SFTP. Follow the steps in the appropriate section below.

Creating a New SFTP Site

To create a new SFTP site that the Reflection FTP Client can connect to using Reflection Secure Shell client, follow these steps:

1. Start Reflection FTP Client, and click New.
2. Enter the FTP server name or IP address. Click Next.
3. Select User as the kind of login you want. Click Advanced.
4. In the "Log on as" group box, select User. Enter a user name.
5. Click Security.
6. Click the Secure Shell tab, and then select the Use Reflection Secure Shell check box.
7. Select the SFTP option (the default). Click OK.
8. Click OK. Click Next.
9. Verify that this is the user name you wish to use. Click Next.
10. Enter the name you want to use for this FTP site. Select Yes to connect to this FTP site now. Click Finish.

A key and/or padlock icon appears on the status bar when you have made a successful secure connection. The key indicates secure authentication; the padlock indicates an encrypted data stream.

Configuring an Existing FTP Site to use SFTP

You can configure an existing FTP site to use SFTP by following the steps below:

1. Start Reflection FTP Client.
2. In the Connect to FTP Site dialog box, select an FTP server.
3. Click Properties, and then click Security.
4. Click the Secure Shell tab, and then select the Use Reflection Secure Shell check box.
5. Select the SFTP option (the default).
6. Click OK twice.
7. In the Connect to FTP Site dialog box, click Connect.

A key and/or padlock icon appears on the status bar when you have made a successful secure connection. The key indicates secure authentication; the padlock indicates an encrypted data stream.

Step Two – Automate the Login

Once the FTP site is configured to use SFTP, you must automate the authentication process before you can automate secure file transfers. SSH requires a password unless another secure authentication is used.

To automate authentication, use either User Key Authentication or Kerberos key exchange.

Using User Key Authentication

User key authentication relies on a public/private key pair that is used for authentication purposes. Before you can make SSH connections to hosts this way, both your PC and the host must be correctly configured.

To configure the PC:

1. Start the Reflection FTP Client. Select an FTP site and click Properties.
2. Click Security.
3. For Reflection 13.0 or higher: Click the Secure Shell tab and select the Use Reflection Secure Shell check box. Click Configure to open the Reflection Secure Shell Settings dialog box.

4. For Reflection 13.0 or higher: In the User Authentication group box, select Public key. (This is the default setting.)

5. For Reflection 13.0 or higher: Click the User Keys tab, and then click Generate Key.

6. In the Key Type field, select the type of key you want to generate. For ssh2 hosts, use either RSA or DSA.
7. Specify a value for Key Length. The default is 1024. Increasing this value increases the time it takes to generate the key and also improves the security of the key you generate.
8. For Reflection 13.0 or higher: Click Create.

9. In the next dialog box, you can enter a passphrase to authenticate with this key. Leave this blank so that your authentication can be automated.

To configure the host:

(Host configuration may be done by the administrator of the host.)

Add the public key created for the PC to the authorized key file in the user's home directory: $HOME/.ssh/authorized_keys.

Note: You may see an SSH banner from your server displayed in a message box that requires you to click OK to continue, thereby disrupting the automated processing. To prevent this message box from displaying, go to Site Properties > Security Tab > SSH Tab > Configure Button > Logging and set logging level to Quiet.

For further details, see Technical Note 1881.

Using Kerberos Key Exchange

When Kerberos key exchange is selected, the Reflection Secure Shell client uses Kerberos for authentication. You must have the Reflection Kerberos Client installed and configured to use this feature; the key exchange setting is dimmed if the Kerberos Client is not installed.

Notes

• The Secure Shell client always uses your default principal and realm for Kerberos authentication. If you have more than one principal profile, you can use the Reflection Kerberos Manager to change the current default profile.
• By default, the Reflection Secure Shell client forwards your Kerberos Ticket Granting Ticket (TGT) to the host after authentication.
• The Kerberos key exchange setting is saved on a host-by-host basis. If you specify new connections to the same host, your current value will be applied. If you configure a connection to a new host, Reflection uses the default value.
• The value of this setting is saved to a configuration file called config. You can also configure SSH settings by editing this file manually in any text editor. The keyword used to configure Kerberos authentication is GssapiAuthentication (for ssh2.)

To configure Reflection FTP Client to use Kerberos authentication:

1. Start the Reflection FTP Client. Select an FTP site and click Properties. (You must log on as a user; Kerberos is not available for Anonymous logins.)
2. For Reflection 13.0 or higher: Click Advanced, and on the General tab, click Security.

3. In the Security Properties dialog box, click the Kerberos tab, and select the Reflection Kerberos check box.

4. Enter the principal name and realm you want to use for Kerberos authentication to this server. If you do not specify a principal and realm, Reflection Kerberos will use your default principal profile for authentication.
5. Click the Secure Shell tab and select the Use Reflection Secure Shell check box, select the SFTP option (the default), and click Configure to open the Reflection Secure Shell Settings dialog box.
6. For Reflection 13.0 or higher: In the User Authentication group box, select GSSAPI/Kerberos.

7. Click OK to close the Reflection Secure Shell Settings dialog box.
8. Click OK to close the Security Properties dialog box.
9. Click OK to close the FTP Server Properties dialog box.

10. Click Connect to authenticate to this host and establish a connection. A key and/or padlock icon appears on the status bar when you have made a successful secure connection. The key indicates secure authentication, the padlock indicates an encrypted data stream.

Step Three – Create an FTP Script File

Once you have successfully configured the Reflection FTP Client to automatically connect to an SFTP server, you can automate file transfers using the Script Recorder.

1. Connect to your SFTP site.
2. Click Script > Start Recording.
3. Navigate as needed, and transfer or delete files.
4. When you are finished, click Script > Stop Recording.
5. Edit the script to add overwrite on get commands and to add quit to the end of the script. See Technical Note 1308 for information about "Editing the FTP Script File."
6. Test the script.

Initiating the Script from the Command Line

You can run your script from the command line in a BAT file, Windows Shortcut, scheduler program, or shell command.

Example

The following command would start Reflection FTP Client from its default location, connect to the FTP Site named "MySFTPSite," run the script SFTPScript.rfs, and create a log file C:\Temp\SFTPLog.log:

Method B – Programming with Reflection FTP 1.0 Type Library (RFTPCOM.DLL)

Another method of automating SFTP transfers is to create VBA macros in Reflection for HP, Reflection for IBM, Reflection for UNIX and OpenVMS, Microsoft Office, or in any development environment that supports the OLE or COM interface to programming.

An example that uses VBA to create a filter for file extensions is available for you to review. The macro will transfer all files in a given directory that match the strFilter variable.

To explore this approach, download the sample file and import it to a VB or VBA editor:

1. Download the example SecureFTP.bas.
2. In your VB or VBA editor, add a reference to Reflection FTP 1.0 Type Library. This step is critical in version 10.0.x because early binding syntax is required for SFTP. In version 11.0 or higher, late binding is supported.
3. In the editor, click File > Import and import SecureFTP.bas.

Using VBA, you can add features to your macro, such as checking to see if files exist before attempting a transfer, deleting files when you are done, writing error logs, or emailing system administrators if a critical file transfer fails.

Method C – Running Sftp.exe on a Command Line

For information about running sftp.exe on a command line to automate a secure file transfer, see Technical Note 2126.

Related Technical Notes

1308	Automating Reflection FTP Client File Transfers
1881	Configuring Reflection for Public Key Authentication
2126	Automating SFTP File Transfers with Reflection for Secure IT