Topic: SSG 550 and FTP Problems

[enyman]

I'm having some FTP problems that are bugging me a little. FTP seems to be getting blocked when using passive FTP  which makes sense because the outbound port is blocked. I know linux uses a connection module ""ip_conntrack_ftp ip_conntrack_tftp ip_nat_ftp ip_conntrack" to recognize the connection. Support just said to open the ports the server is responding on but the problem here is that I have a group of servers accessing the universe via FTP and I would have to open a wide range of outbound ports to allow that to happen. Any ideas? I'm using the SSG 550 and with the NSM server.

Active FTP :
command : client >1023 -> server 21
data    : client >1023 <- server 20

Passive FTP :
command : client >1023 -> server 21
data    : client >1023 -> server >1023

[alan]

Maybe a clue here?

http://kb.juniper.net/CUSTOMERSERVICE/index?page=kbdetail&article_id=KB8424

[hairybiddy]

passive ftp doesnt work for me either through a ssg550 running 5.4.0r3a.0. you can open up a connection to the server but then it tells me it cant open a data connection when you try to do a directory listing or anything..  My firewall is currently running in allow any any in each direction where this traffic is flowing.. I cant work it out, I'm waiting for my support contract to come through so i can raise a JTAC case for this as its really annoying, i have 1 old box that needs to use passive ftp! active works fine..

[alan]

Aha! Think I have the answer on this one - you need to use the FTP ALG.
Create the rule: from <any> / to <any> / service <ftp> / application <ftp>
That's the secret - the "application <ftp>"

CLI
set policy id 4 name "ftp alg" from "Trust" to "Untrust"  "Any" "Any" "FTP" permit log
set policy id 4 application "FTP"

"The FTP ALG monitors the automatically solves this problem by monitoring the FTP channel, looking for FTP port commands that specify which source and destination ports are being requested, and dynamically opening up that specific combination of source IP/port and destination IP/port firewall policy (called a gate) that permits the session to flow"
-from "Juniper Networks Netscreen & SSG Firewalls" book
http://www.amazon.com/Configuring-Juniper-Networks-NetScreen-Firewalls/dp/1597491187/ref=sr_1_1/104-6137524-9091957?ie=UTF8&s=books&qid=1175293268&sr=8-1

Please let us know if this solves the passive ftp problem for you

[hairybiddy]

I have that book too!  not got to that chapter yet.. Ill give it a go when im back off my holiday and let you know. Cheers!

[alan]

FYI, tested this out with passive mode (MIP to internal host) & works perfectly.

[junipoint]

You shouldn't need to specify an Application in a policy for a known service. If you have to do this to make FTP work, then there must be a bug. All ScreenOS documentation references the use of the application command in a policy when using non-standard ports for a particular application. For instance, if you had a custom FTP service defined that used tcp 2121 for the destination port then you would specify application FTP in the policy.

[alan]

I disagree junipoint - ScreenOS uses polices to determine which traffic is allowed to flow between security zones & interfaces. However there are protocols that use dynamic ports and protocols that need to be modified in order for NAT to work. "Active FTP" does not need an ALG but "passive FTP" does (unless you want to open all ports > 1023) The JTAC person that recommended that should be sent back to boot camp.

If the protocol is well behaved then yes, the service is all you should need.

There are many known services that do this dynamic allocation or have some protocol deficiency- a list is at the end for the various ALGs in 5.4R3a. Unfortunately these are poorly documented - if at all. I'm trying to get info what some of these do now.

The DNS ALG for example closes the session after a reply is received rather than waiting for protocol timeout.
The H.323 protocol is well known for dynamic ports - the ALG opens these pinholes for it to work.
Some protocols change from TCP to UDP such as RTSP. Unless you open all these services the service will fail.
MS-RPC (as with Sun-RPC) can use TCP, UDP, named pipes and HTTP as the transport.
And here the FTP ALG opens the pinholes needed by listening to the "FTP PORT" commands it sees from the client - something the firewall could not possibly know in advance.

DNS                  Domain Name Service
FTP                  File Transfer Protocol
HTTP                HyperText Transfer Protocol
IMAP                IMAP
SMTP               Simple Mail Transfer Protocol
POP3                Post Office Protocol V3
PPTP                Point-to-Point Tunneling Protocol
RSH                 Remote Shell
H245                H.245
Q931                Q.931
RAS                 RAS
SCCP               SCCP
MGCP_UA          MGCP-UA
MGCP_CA          MGCP-CA
PORTMAPPER     SUN-RPC portmapper
SIP                  Session Initiation Protocol
SQLNETV2         SQL*Net Version 2
TALK                Talk program
TFTP                TFTP
REAL                RealMedia
RTSP                Real Time Streaming Protocol
VDO                 VDOLive
XING                Xing
IGNORE             Ignore application type
MSRPC_EPM       MSRPC Endpoint Mapper
GNUTELLA          Gnutella File Sharing Protocol
AIM                  AOL Instant Messenger
YMSG               Yahoo! Messenger
SMB                 Server Message Block Protocol
MSN                 MSN Messenger
NBNAME            NetBIOS Name Service
NBDS                NetBIOS Datagram Service
NAS                 Custom ALG

[alan]

http://kb.juniper.net/CUSTOMERSERVICE/index?page=kbdetail&record_id=0244022611e8310108031d1d8f00592d

[junipoint]

Alan, that article is titled "Configuring FTP service over non-standard ports", which is exactly what I stated earlier:

All ScreenOS documentation references the use of the application command in a policy when using non-standard ports for a particular application

I just did a passive ftp transfer using my existing policy that doesn't have the Application FTP command and it went through without a hitch. Attached is a sample of the debug flow:

netscreen-> get policy id 64
name:"FTP/NTP/SSH" (id 64), zone Trust -> Untrust,action Permit, status "enabled"
1 source: "10.0.0.0/8_trust"
1 destination: "Any"
3 services: "FTP", "NTP", "SSH"
Policies on this vpn tunnel: 0
nat src, Web filtering disabled

****** 4234301.0: <Trust/ethernet0/0> packet received [48]******
  ipid = 12035(2f03), @2e5a9110
  packet passed sanity check.
  ethernet0/0:10.1.100.246/2634->209.xxx.229.58/21,6<Root>
  no session found
  flow_first_sanity_check: in <ethernet0/0>, out <N/A>
  [ Dest] 14.route 10.1.100.246->10.0.254.254, to ethernet0/0
  chose interface ethernet0/0 as incoming nat if.
  flow_first_routing: in <ethernet0/0>, out <N/A>
  search route to (ethernet0/0, 10.1.100.246->209.xxx.229.58) in vr trust-vr for vsd-0/flag-0/ifp-null
  [ Dest] 13.route 209.xxx.229.58->63.xxx.28.1, to ethernet6/0
  routed (x_dst_ip 209.xxx.229.58) from ethernet0/0 (ethernet0/0 in 0) to ethernet6/0
  policy search from zone 2-> zone 1
 policy_flow_search  policy search nat_crt from zone 2-> zone 1
  RPC Mapping Table search returned 0 matched service(s) for (vsys Root, ip 209.xxx.229.58, port 21, proto 6)
  No SW RPC rule match, search HW rule
  Permitted by policy 64
  dip id = 2, 10.1.100.246/2634->63.xxx.28.5/5800
  choose interface ethernet6/0 as outgoing phy if
  no loop on ifp ethernet6/0.
  session application type 1, name FTP, nas_id 0, timeout 1800sec
ALG vector is attached  service lookup identified service 1.
  flow_first_final_check: in <ethernet0/0>, out <ethernet6/0>
  existing vector list 83-66d4630.
  Session (id:62327) created for first pak 83
  flow_first_install_session======>
  route to 63.xxx.28.1
 arp entry found for 63.xxx.28.1
  nsp2 wing prepared, ready
  cache mac in the session
  make_nsp_ready_no_resolve()
  search route to (ethernet6/0, 209.xxx.229.58->10.1.100.246) in vr trust-vr for vsd-0/flag-3000/ifp-ethernet0/0
  [ Dest] 14.route 10.1.100.246->10.0.254.254, to ethernet0/0
  route to 10.0.254.254
  flow got session.
  flow session id 62327
  Got syn, 10.1.100.246(2634)->209.xxx.229.58(21), nspflag 0x801801, 0x800800
 flow_send_vector_, vid = 0, is_layer2_if=0
.
.
.
.
.
.
.
.
.
.
.PASSIVE FTP starts here
****** 4234301.0: <Trust/ethernet0/0> packet received [48]******
  ipid = 12048(2f10), @2e6a0910
  packet passed sanity check.
  ethernet0/0:10.1.100.246/2635->209.xxx.229.58/64769,6<Root>
  no session found
  flow_first_sanity_check: in <ethernet0/0>, out <N/A>
  [ Dest] 14.route 10.1.100.246->10.0.254.254, to ethernet0/0
 vsd (0) is active, make hole active  active hole found
  make_nsp_ready_no_resolve()
  route to 63.xxx.28.1
  existing vector list 83-66d4630.
  flow_first_install_session======>
  make_nsp_ready_no_resolve()
  route to 10.0.254.254
  flow got session.
 flow session id 62507
  Got syn, 10.1.100.246(2635)->209.xxx.229.58(64769), nspflag 0x801801, 0x800800
 flow_send_vector_, vid = 0, is_layer2_if=0
****** 4234301.0: <Untrust/ethernet6/0> packet received [48]******
  ipid = 25394(6332), @2e50a910
  packet passed sanity check.
  ethernet6/0:209.xxx.229.58/64769->63.xxx.28.5/2635,6<Root>
  existing session found. sess token 6
  flow got session.
  flow session id 62507
  Got syn_ack, 209.xxx.229.58(64769)->63.xxx.28.5(2635), nspflag 0x801800, 0x801801
 flow_send_vector_, vid = 0, is_layer2_if=0
****** 4234301.0: <Trust/ethernet0/0> packet received [40]******
  ipid = 12050(2f12), @2e504910
  packet passed sanity check.
  ethernet0/0:10.1.100.246/2635->209.xxx.229.58/64769,6<Root>
  existing session found. sess token 4
  flow got session.
  flow session id 62507
  Got ack, 10.1.100.246(2635)->209.xxx.229.58(64769), natpflag 0x1000, nspflag 0x801801, 0x801800, timeout=900


Observations:
-ScreenOS activates the FTP ALG automatically if FTP is specified as a service in the policy
- ScreenOS creates a temporary session as part of the FTP ALG permitting ports >1023 between client and server
- This temporary session is closed immediately once the transfer is complete
- The client's source port is not port-translated during the PASSIVE FTP

[spingineer]

I'll re-iterate Junipoint ... The Application option is only used when you have a custom service defined, but you want layer 7 pinhole checking for a specific application.  FTP is a perfect example.

We have seen some passive FTP not working, due to some non-standard FTP implementation on the server end.  Usually, a debug flow basic will tell you when the passive data is passed across, and it complains about no active hole found.

[junipoint]

Exactly. I ran into this with FTPS. Certain FTPS implementations use TCP 21 and behave like passive FTP servers, but the Netscreen FTP ALG doesn't recognize them. In this situation, you have to open up >1023 to the server you are establishing the session with.

[alan]

Hmm. Interesting point, Junipoint. I need to dig deeper into what's going on here.
Why is there a selection for the ALG if it gets automagically invoked?
This was not a "Custom" service above, just "ftp", so why did the ALG kick in?

Opening > 1023 outbound doesn't bother me - it's the inbound to a passive server in the DMZ that's the kicker.

[spingineer]

Alan,
By default, if you have Application None enabled, it will look at the session created, and based on the destination port the session created, it will look up its Application id to determine if it has to do further layer 7 processing.  However, if you wanted to specify a different port, but choose a specific layer 7 processing, that's where you use the Application option.

Let's take http for example.  If you choose the default http service, and have Application None, it will notice a session for port 80 has been built.  Since Application None has been specified, it knows to go to the decision tree of looking at the default Application id's.  It notices this is http, and therefore uses that, and proceeds with http layer 7 processing.  If you choose a custom port (e.g. 8080), and you have Application None, it will notice that it does not match an Application id, and therefore will do no further layer 7 processing.  However, if you choose Application http, it knows to process http layer 7.

[hairybiddy]

annoyingly even after following the advice above i still cant get directory listings from this one old box that uses passive FTP, connection happens fine but when i try and do ls or similar i get

ftp> ls
425 Can't build data connection

 I have tried setting the service and application to FTP in the policy to no avail, yet I know it works fine when i dont try and go through the firewall.. Any ideas?

[alan]

debug flow basic and see what the firewall is seeing would be my next step.
tcpdump to see if the packets are getting through. Anything in the firewall logs?
Did you try other ftp clients like Filezilla - perhaps they're more verbose as to what's missing.

[oldo]

I've seen passive ftp getting all f**ked up when using the services ftp, ftp-get, ftp-put in the same policy, (no application match). I bet this isn't the case here, but I just want to mention it if you happen to stumble on this type of policy... The customer in this case wanted to allow FTP outbound... The fix, simply scrap the ftp-get/ftp-put in the policy. Then it worked as it should, matching the alg for ftp.

[jmcgrady]

I have the same issue on my 550 cluster.  Standard ftp using passive mode doesnt. I tried setting the FTP ALG and it still doesnt work. Active FTP works fine.

[hairybiddy]

I raised this with juniper TAC they told me they couldn't replicate the problem in their lab after months of moaning at them (yay for customer service!) there is a work around it is to disable tcp-syn checking, however that has security implications as you can only disable it globally. Ill remember the command when I'm back at work and let you know but do some research on what effects the command has to security.

[hairybiddy]

unset flow tcp-syn-check is the command to make it work.