Poor transfer rate (SSH/SFTP)

xiphias_ · Post by **xiphias_** » Tue Sep 23, 2003 4:52 pm

I'm using Core Ftp between to local computers. And I'm only able to get 300KB/s over a ssh session.
In another program I tried(from www.ssh.com), I was able to get 1,2MB/s
Is this a bug/feature in the lite version?
I use ver. 1.2 build 1144

Also tried to fiddel with the Max send/receive in the properties field, without any luck.

Regards,
Finn Andersen
Norway

CardinalFang · Post by **CardinalFang** » Wed Sep 24, 2003 1:58 pm

CP,

Thank you for at least attempting to address this problem. Just wanted to let you know that I tested the latest build (1.2b) and tried messing with the buffer settings for an SFTP upload (to a Linux server running OpenSSH 3.7.1p, Blowfish cipher selected).

Unfortunately I didn't find any speed improvement. The best performance was obtained using the default send buffer of 512. Anything higher just seemed to reduce the upload speed a little. Uploading to that server over a T1 gave me a maximum speed of about 50kb/s as measured by CoreFTP. I no longer have the SSH client installed on this PC to compare, but the transfer rate was a whole lot faster.

If a friend located in another country tries to upload to my server she gets about 8k using CoreFTP vs 50k with the SSH client.

While CoreFTP is running a bit faster than the Putty-based clients out there (such as filezilla), it is still nowhere near the speed of the SSH client (version 3.2.5 for Windows available at ftp.ssh.com/pub/ssh/) as xiphias_ mentioned in the post before mine. The major problem I have with SSH's client is that it does not support resume, which is why I was looking at CoreFTP as an alternative.

If there's something you could do to bring the SFTP performance of CoreFTP up to the level of the SSH client I would make it my SFTP client of choice in a second.

CardinalFang · Post by **CardinalFang** » Wed Sep 24, 2003 2:47 pm

I just tried 1.2c and did not notice a difference with the buffer settings. Like the previous version, if anything setting it higher seemed to slightly reduce the rate, but maybe that's just a natural variation in the upload speed.

I tried send buffers of 512 (default), 1024, and 2048. To make sure the setting was taking effect, I quit the program after changing the setting and restarted it.

If there's anything I can do to help you test this let me know.

CardinalFang · Post by **CardinalFang** » Wed Sep 24, 2003 3:01 pm

If I set them to 65535 the program resets them both to 1024. I tried 65000 for both... no difference. How strange... I'd love to know what the secret of the SSH client is.

I do appreciate the quick replies to my post!

CardinalFang · Post by **CardinalFang** » Wed Sep 24, 2003 3:11 pm

65000 seems to be the max it will accept. Even going up to 65001 will cause it to reset to 1024 as soon as you check the settings again.

CardinalFang · Post by **CardinalFang** » Wed Sep 24, 2003 4:03 pm

Looks like that build has been accidentally tagged 1144, oops! At least I know I defintely got the version you changed.

Sorry to say that it did not make an improvement. Here's what I did: I installed over my existing 1.2c build, then started CoreFTP and found the buffer settings both at 1024. I changed those both to 65000 and connected. After no improvement in transfer speed I tried setting them to 2048 up / 4096 down. Didn't make any difference.

UPDATE: Just tried 8096 per your suggestion. Also 4096. I'm not seeing any difference.

CardinalFang · Post by **CardinalFang** » Wed Sep 24, 2003 4:22 pm

My fault, CP. I clicked the wrong download link.

With this new version there is a definite improvement of abut 25k... transfer speed has risen from 50 to 75. Nice job!

Best results seemed to be with 8096 as the send buffer. Just for fun I tried 32768 and speeds dropped a little.

I'm going to get the SSH client and update this post to tell you exactly what rates I get with that.

UPDATE: With the SSH client, I get 113-114k transferring the same file.

CardinalFang · Post by **CardinalFang** » Wed Sep 24, 2003 4:47 pm

That's correct, all my testing has been using CoreFTP to upload to the server. All speeds I reported were for uploading. The server is an athlon XP 1700 and the machine I'm running CoreFTP on is a Celeron 433 running Windows 2000 w/ 512MB ram.

Thanks for taking a look at this. If it ever gets to the point where you can overhaul the SSH code and make it as fast as the SSH client, that feature alone to me will be worth registering the Pro version to support your development. I think there are a lot of other people who are looking for a fast Windows SFTP client that supports resume too.

CardinalFang · Post by **CardinalFang** » Wed Sep 24, 2003 5:43 pm

I did a lot more testing letting the transfer go on longer, and it contradicts my earlier (hastily tested) findings a bit.

I am finding that the more you increase the buffer size, the better it's getting. After testing the long-term results, I'm getting about 5k more out of using 16300 than using 8096, and about 5-6k above that if I use 32768. You just need to wait for the connection to "settle" a bit first.

Hope this helps!

P.S. this thread is getting huge, let me know if you want to continue via email instead

CardinalFang · Post by **CardinalFang** » Wed Sep 24, 2003 7:00 pm

You were right about corruption, I never thought to check for that. The zip file I uploaded to the server did in fact corrupt with the last build. Too bad, since that build was so darn fast! I suppose that had something to do with the corruption.

Build 1155 is even slower than before: 35-40k even with buffer of 32768. I checked with the SSH client to make sure it was not my connection. Something has changed but not for the better.

CardinalFang · Post by **CardinalFang** » Wed Sep 24, 2003 7:34 pm

With 8096, 16384, and 32768 I got an average of 40k. When I set it to the default of 512 the rate did not worsen, in fact it improved by 1 or 2 k (which could just be a variation in the connection. I did observe that twice though.

I'll try 1156 and let you know.

CardinalFang · Post by **CardinalFang** » Wed Sep 24, 2003 8:05 pm

ok, in 1156 I'm getting pretty much the same as I used to with 1.2b: average of around 50k, whether using 512, 8192 (should have been using this instead of 8096 which is not a correct multiple), 16384, or 32768.

CardinalFang · Post by **CardinalFang** » Wed Sep 24, 2003 9:02 pm

Hate being the bearer of bad news, but rates with all buffer sizes are back down to the 40 range with build 1157. As usual, a test right afterward with the SSH client gives 100k+ rates.

Wierd huh!

CardinalFang · Post by **CardinalFang** » Thu Sep 25, 2003 7:06 pm

Thanks CP! I just found the 1158 build and am testing it. I'm actually seeing rates of 80k with this build compared with 50k before, that's a 60% increase... not at all bad!

Now I've got to wait to transfer the whole file to see if corruption is still a problem.

UPDATE: After uploading and downloading a zip file with 1158, it came out corrupted. I'll try again with a different file as a second test.

CardinalFang · Post by **CardinalFang** » Thu Sep 25, 2003 8:52 pm

In my testing with 1158, I discovered that there's nothing wrong with uploads. It's downloads that are corrupting.

If I upload a file with CoreFTP, then download it with the SSH client it's fine. If I download it with CoreFTP, the downloaded file is corrupt. Buffers were at defaults (512/4096), and 4096 is a multiple of 1k so that's not it...

Have not tried 1159 yet.

CardinalFang · Post by **CardinalFang** » Thu Sep 25, 2003 9:31 pm

New info after testing 1159:

What I thought was a minor cosmetic problem is actually leading to corruption: some files keep "downloading" past the actual size of the file. The downloaded file continues to grow in size, the progress indicator keeps going past 100%, and the status time starts going into negative numbers. The resulting download is unusable.

Looks like it's forgetting to slam on the brakes at the end or something!

On a positive note, resuming uploads seems to work perfectly and does not corrupt the file (Not that it ever did. This is just the first time I tried it.)

I didn't change the buffer settings: 512/4096.

I still get 80k on uploads. My friend gets 28k when she used to get 8. Very nice work! Can't wait to see the "cleaned up" code when you get around to it.

CardinalFang · Post by **CardinalFang** » Fri Sep 26, 2003 1:52 am

The server is RedHat 9. If you run out of other testing options we can probably work something out to get you an account, but see below... it may not be related to the server.

Testing here from a different machine over the LAN and using a dial-up connection, I couldn't reproduce the corruption nor the strangeness with "downloading" past the file size.

There's either something wrong with the installation of CoreFTP on my work machine, or perhaps the corruption can only be reproduced after uploading lots of files as I was doing all day. I'm confident there is nothing wrong with the T1 Internet connection at work because I would always successfully transfer the file using the SSH client. It also doesn't appear to be a latency issue because the latency of a dial-up connection is much more than that on the T1 and that worked.

When I was getting the corruption, I was testing with two zip files: one 3mb in size and one 20mb. I verified corruption by trying to unzip the downloaded file and receiving either bad CRC or Invalid Header from winzip. Also throughout, I have been using the Blowfish cipher since this gives me the fastest results.

Another thing: as is my habit, I did not reboot the machine all day and I run a ton of apps at once. I will try uninstalling, reinstalling, and testing fresh tomorrow.

Maybe this info will help you pinpoint your testing...

CardinalFang · Post by **CardinalFang** » Fri Sep 26, 2003 2:17 pm

I uninstalled, deleted the CoreFTP folder, and installed 1160. Tried the same thing as yesterday and didn't run into any corruption, so it looks like we're good for now.

The upload speed increase vs 1.2b is less now; seems to be around 20% as you mentioned previously.

I look forward to testing again when you have a chance to clean or rewrite the SSH code and make it faster.

Thanks again for looking into this issue.

Post by **ForumAdmin** » Tue Mar 21, 2006 8:13 pm

SFTP speed improvements should be noticable in the 1450 build and greater.

Core FTP

Poor transfer rate (SSH/SFTP)

Poor transfer rate (SSH/SFTP)

poor transfer rate