Constant disconnects

[fkellyalbany]

My host has some kind of limits set on FTP sessions, either bandwidth or file sizes. I try to use compare contents between directories on my PC and what's on the host. I can compare maybe up to 200 files, then I get disconnected (my IP gets locked out) so I can't get back onto that host at any url for about 10 minutes. Very frustrating because I need compare contents to avoid overriding any customizations I may have done on the host end.

I will attach a trace of the log. Nahh, the log is too big. Here's the start of it:

9/15/2008 8:21:19 PM Username: xxxx\uuuuy
9/15/2008 8:21:20 PM Stat> Connected.
9/15/2008 8:21:20 PM Recv> 220---------- Welcome to Pure-FTPd [TLS] ----------
9/15/2008 8:21:20 PM 220-You are user number 3 of 50 allowed.
9/15/2008 8:21:20 PM 220-Local time is now 20:21. Server port: 21.
9/15/2008 8:21:20 PM 220-This is a private system - No anonymous login
9/15/2008 8:21:20 PM 220 You will be disconnected after 30 minutes of inactivity.

so you can see it is pure-ftpd ...

I have tried slowing the transfer speeds with not much success. I can run something like Smartftp and transfer 20+ meg of files without a hiccup but somehow you compare contents thing sets off big alarms at the server and gets me disconnected regularly. The host is a very competent one and has investigated but says I am the only one with the problem ... cause I am probably the only one trying to use BC3 (it did the same with BC2) for this purpose.

[Zoë]

You're welcome to email the complete log to [email protected].

Have you increased the number of connections that BC uses for FTP? It defaults to 1, but allows up to 10 simultaneous ones if you change it in the profile.

After you've been banned, if you connect again does it give you any message, or does it just refuse the connection?

I hate bouncing you around even more, but you might have some luck with the Pure-FTPd mailing lists too.

[fkellyalbany]

I can try increasing the number of connections. I doubt that will work. I think pureftp limits the amount of uploads and downloads you can do in a specified time. I'm not sure but I think that doing a large number of file comparisons must really stress the system. I have some success if I limit the number of files I try to compare to under 100 at a time ... problem is we have like 10,000 files in our distribution and I'd love to just let it run at a computer and come back when it's done. That doesn't work.

I have googled pureftpd and read thru their documentation. It appears they use a mysql table to impose limits on their FTP customers. I have also tried limiting the upload and download speeds in BC and, while it runs slower and is a little more reliable it is far from making the problem go away. I'm guessing that the problem when you are doing file comparisons is that each host file has to be downloaded into memory to be compared to the PC file. My guess is that this gets more intense that the host allows.

When you get disconnected all connections from your IP get knocked out for about 10 minutes. You can't even visit other sites on the same server or get your email from their web mail during that time.

Thanks for the mailing list link. I will look and I will send a log next time I replicate the problem.

[fkellyalbany]

I have looked at the mailing list. It appears pureftpd uses a mysql table to set limits for ftp clients. I am pretty sure that I am just overrunning the limits with BC3 (and was with BC2) when doing file compares. My product (a web content management system) has 4800 files and I need to run comparisons between my distribution on the pc and what's on various host sites ... to avoid overwriting customizations as I update.

I tried the suggestion of increasing connections this morning. I am writing this as I wait for my 10 minute disconnect to "clear". Increasing the connections just resulted in an almost immediate disconnect as I thought it would. But I would try anything.

I mailed in the log. By the way, the log will show I was using binary compare but most of the time I use rules based.

Thanks for you attention.

Maybe you could have a big "throttle" button that could be adjusted until the product runs slow enough to keep the host happy. I think I will try setting the upload and download speeds to something ridiculously low and see if the product will run that way ... I will just devote a pc to the task if it does and let it run all day. I just need a complete picture of differences.

[Zoë]

Sorry for the confusion, I was asking whether you had adjusted the number of connections, not recommending that you do so.

If you aren't able to connect to web or email when you're locked out it sounds more like it's a firewall or OS issue than something specific to Pure-FTPd. An FTP server, regardless of what limits it imposes, shouldn't be able to affect other servers.

I also loaded a copy of Pure-FTPd in a Linux virtual machine (Ubuntu 7.10), and at least with the default configuration, it doesn't have any trouble. I've been doing repeated content comparisons of 2800 files over 10 connections and it's working just fine.

[fkellyalbany]

If I dumb my transfer speeds down to 32kbs I seem to be able to complete the file comparisons of 4800 files without the disconnects. I tried boosting it to 64k and got disconnected. It's slow but I can have lunch and come back and it's done.

There is just one server I am connecting to (if I have that terminology right). It is one IP address for all my account. Then I have different public_htmls and different document roots and different ftp accounts within there. However when I get disconnected I cannot reconnect from my IP address on any account for 10 minutes.

I will show the Host manager this thread and maybe he can see something that can be done. However this is a site that is usually very generous in allocating resources and I'm not sure that at higher speeds this might not overwhelm any server.

Thanks for your efforts. I'm impressed. If you feel like it you might look at the bandwidth throttling features of pureftpd. But without knowing exactly what settings my host is imposing that could be chasing your tail. I'll be satisfied if 32kbs works. Heck, maybe I should just connect through a 9600 baud modem just like the good old days.

[fkellyalbany]

Nahhh!!! Even cranking down to 32kbs on both uploads and downloads I get disconnected after comparing 878 out of 4800 items.

Back to the drawing board.

[Zoë]

Like I said before, if it's blocking you out of your webmail it's a firewall issue. It's possible it has some features to try to prevent denial of service attacks by locking out IP addresses temporarily. You might try toggling the "Passive transfers (PASV)" checkbox in the FTP Profiles' "Connection" tab, which would make the server connect to you to transfer files instead of you connecting to it.

[fkellyalbany]

Tried non-Pasv. Thought it was working but ultimately the same thing. Disconnected. I got greedy and boosted the speed to 256kbs. I might try it again at 56kbs when I can get connected again.