Hello,
I have used CoreFTP LE off and on for awhile. I keep trying other software because of file transfer speeds. Here is my problem.
I use Comcast (yes, I know they throttle upload speeds compared to download). When I use CoreFTP LE I get between 75 and 100KBps transfer rates (roughly about 2.5 to 3 hours per gigabyte in video transfers to my web host). However, when I try programs like SmartFTP or CuteFTP, they routinely transfer at 200+KBps (roughly 1 hour, 15 minutes per gigabyte). Basically, CoreFTP is about 1/2 as fast or less. These transfer speeds have been measured using the same files to the same server, alternating and are consistent. Even when done simultaneously (i.e. both programs transferring the same file to different directory locations).
What keeps me coming back to CoreFTP LE is that while it is slower, it is 100% reliable on large file transfers (~1GB plus). For my smaller stuff, I'll use CuteFTP. I would probably buy and use CoreFTP Pro exclusively if I knew it would get comparable transfer speeds.
Does anyone know what is going on? Is there some setting I haven't made that would fix this?
Any help would be greatly appreciated!
Best regards,
Scott
Slow upload speeds with CoreFTP LE
-
GlueFactoryBJJ
- Posts: 4
- Joined: Mon Nov 01, 2010 2:37 pm
I've been communicating with Chris Phillips in sales and thanks to his help and a bit of digging on my part, I think I've figured out why I'm getting such slow upload speeds.
If you open CoreFTP, open the Site Manager (click Sites, Site Manager), you will see your list of sites. Click on a site (left side of pop-up window) you are getting slow uploads (and maybe even downloads) to and then click the "Advanced" button, then click "Connections" (left side of new pop-up window).
In the middle of this window just over 1/2 way down, you will see, a "Buffer Sizes (Bytes)" box. This will list "Send" and "Receive" fields. In my copy the Send buffer was set to only 512 bytes. I don't know for sure if this was a default setting or something I had (in a moment of insanity) set. Setting these buffers to either 4096 (4KB) or 8192 (8KB) Bytes about doubled the transfer rate. I believe that the maximum size allowed in ftp is 65536 (64KB) bytes which would provide the maximum rate available. However, after some further testing, I found that CoreFTP supports a max buffer size of 32768 (32KB).
However, there is a downside to very large buffers. If the FTP server is a bit "laggy" or there is a lot of traffic/overloaded routers between your PC and the FTP server you are trying to upload to, larger numbers will result in larger packet retransmits when there are errors.
For example, if there is an error with a 4KB packet, then, on average this will result in a loss of 2048 (2KB) bytes (4KB / 2 = 2KB) in that packet that (net) need to be retransmitted. If there is an error with a 64KB packet, the loss will average to 32KB (64KB / 2 = 32KB).
If there are very few errors, then the 64KB buffer size will work best because there will be higher data to non-data (verification, etc.) traffic ratio. If there are many errors, then this will result in huge retransmits and even file upload errors as the connection and/or server are overwhelmed. Not to mention that I understand that some older FTP site software to not support the 64KB buffer sizes very well.
I believe (gut feeling, no data testing to support it) that a buffer size of 4KB or 8KB will provide a close to max transmission rate, while relatively minimizing problems if there are data errors in transmission. This is something you will need to play with to find what works best for you.
In my brief testing, going from 512 bytes buffer to 4096 bytes about doubled (or better) my transmission rate (and, therefore, cut the time to transmit about in 1/2). By going to 8192 bytes, I got about another 10-15%. Definitely getting into diminishing returns after this point.
With these new settings I found CoreFTP to be about 10-20% faster than CuteFTP Pro at any given buffer size setting indicating, at least to me, that CoreFTP is more efficient.
I'd like to thank Chris again for his help and I hope this helps anyone who has been having similar problems.
Best regards,
Scott
If you open CoreFTP, open the Site Manager (click Sites, Site Manager), you will see your list of sites. Click on a site (left side of pop-up window) you are getting slow uploads (and maybe even downloads) to and then click the "Advanced" button, then click "Connections" (left side of new pop-up window).
In the middle of this window just over 1/2 way down, you will see, a "Buffer Sizes (Bytes)" box. This will list "Send" and "Receive" fields. In my copy the Send buffer was set to only 512 bytes. I don't know for sure if this was a default setting or something I had (in a moment of insanity) set. Setting these buffers to either 4096 (4KB) or 8192 (8KB) Bytes about doubled the transfer rate. I believe that the maximum size allowed in ftp is 65536 (64KB) bytes which would provide the maximum rate available. However, after some further testing, I found that CoreFTP supports a max buffer size of 32768 (32KB).
However, there is a downside to very large buffers. If the FTP server is a bit "laggy" or there is a lot of traffic/overloaded routers between your PC and the FTP server you are trying to upload to, larger numbers will result in larger packet retransmits when there are errors.
For example, if there is an error with a 4KB packet, then, on average this will result in a loss of 2048 (2KB) bytes (4KB / 2 = 2KB) in that packet that (net) need to be retransmitted. If there is an error with a 64KB packet, the loss will average to 32KB (64KB / 2 = 32KB).
If there are very few errors, then the 64KB buffer size will work best because there will be higher data to non-data (verification, etc.) traffic ratio. If there are many errors, then this will result in huge retransmits and even file upload errors as the connection and/or server are overwhelmed. Not to mention that I understand that some older FTP site software to not support the 64KB buffer sizes very well.
I believe (gut feeling, no data testing to support it) that a buffer size of 4KB or 8KB will provide a close to max transmission rate, while relatively minimizing problems if there are data errors in transmission. This is something you will need to play with to find what works best for you.
In my brief testing, going from 512 bytes buffer to 4096 bytes about doubled (or better) my transmission rate (and, therefore, cut the time to transmit about in 1/2). By going to 8192 bytes, I got about another 10-15%. Definitely getting into diminishing returns after this point.
With these new settings I found CoreFTP to be about 10-20% faster than CuteFTP Pro at any given buffer size setting indicating, at least to me, that CoreFTP is more efficient.
I'd like to thank Chris again for his help and I hope this helps anyone who has been having similar problems.
Best regards,
Scott
CoreFTP Speed
Interesting . . . sounds like your comparisons were based on FTP? I know that for sftp, there is a little known feature called "Pipelining" that dramtically speeds up transfers, and most sftp clients do not have it. It is not turned on in CoreFTP LE by default (I think it is on by default in the server, and/or is not configurable), but can be found by clicking "Advanced" near the Site Name, and then on SSH in the left column. There should be a tick box for "Pipelining" on the right. I've posted the speed differences somewhere on here previously, and they are dramatic.
-
GlueFactoryBJJ
- Posts: 4
- Joined: Mon Nov 01, 2010 2:37 pm