Routing Problem 1

[M8tt]

I have a Firewall running Solaris 2.6 with a /30 link network connecting it to a Router on its qfe0 interface. I am attempting to set-up a second /30 link network as a virtual interface to connect to the same Router but am having problems getting the connection up.

The connections look as follows: -
qfe0 is x.x.208.138 with the Router interface set as 208.137
qfe0:1 is z.z.2.14 with the Router interface set as 2.13

I am able to ping x.x.208.137 and z.z.2.14 but not z.z.2.13.

The relevant entries have been made in ifconfig and netstat and I am sure that the Firewall rules are allowing traffic through.

I have created an /etc/hostname.qfe0:1 file with the reference qfe0-1 in it and have referenced this in /etc/hosts against z.z.2.14. I have also entered the relevant reference in /etc/netmasks.

I have not rebooted the machine yet but do not believe that I need to given the changes I've made.

Can anyone suggest why I can not get the virtual interface working?

 

[AHinMaine]

Did you perform the

ifconfig qfe0:1 up

?

--
Andy

http://www.nachoz.com

"Historically speaking, the presence of wheels in Unix has never precluded their reinvention."
Larry Wall

 

[M8tt]

Yes I did, and ifconfig -a shows that my qfe0:1 interface is up.

 

[daFranze]

can I see

ifconfig -a
netstat -rn
cat /etc/hostname.q*
cat /etc/hosts
grep hosts: /etc/nsswitch.conf

if you prefere not to show us the full IP just X-out the first two tupels like above...

Regards
-- Franz
Sorry I'm not a native spaeker, I'm from Munich, Germany - "Home of the Whopper", oh no, "Home of the Oktoberfest" ;-)
Solaris System Manager; I used to work for Sun Microsystems Support (EMEA) for 5 years

 

[M8tt]

Franz,

As you suggest I have substituted the first two numbers of the IP out for w,x,y or z to represent the four different ranges referenced. I have applied this consistently (hopefully) to the below.

Thanks, Matt.

# ifconfig -a
lo0: flags=849<UP,LOOPBACK,RUNNING,MULTICAST> mtu 8232
inet 127.0.0.1 netmask ff000000
hme0: flags=863<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST> mtu 1500
inet x.x.46.23 netmask fffff800 broadcast x.x.47.255
ether 8:0:20:d1:45:2a
hme0:1: flags=863<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST> mtu 1500
inet y.y.64.1 netmask fffff800 broadcast y.y.71.255
qfe1: flags=863<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST> mtu 1500
inet w.w.2.254 netmask ffffff00 broadcast w.w.2.255
ether 8:0:20:d0:53:7a
qfe2: flags=863<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST> mtu 1500
inet w.w.3.1 netmask ffffff00 broadcast w.w.3.255
ether 8:0:20:d0:53:7b
qfe3: flags=863<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST> mtu 1500
inet w.w.79.1 netmask ffffff80 broadcast w.w.79.127
ether 8:0:20:d0:53:7c
qfe0: flags=843<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet x.x.208.138 netmask fffffffc broadcast x.x.208.139
ether 8:0:20:d0:53:79
qfe0:1: flags=843<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet z.z.2.14 netmask fffffffc broadcast z.z.2.15


# netstat -rn
Routing Table:
Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ------ ---------
y.y.64.204 x.x.44.74 UGH 0 78
y.y.64.205 x.x.46.116 UGH 0 2
x.x.44.132 y.y.64.150 UGH 0 11909
y.y.64.200 x.x.46.95 UGH 0 635
y.y.64.201 x.x.44.107 UGH 0 98
y.y.64.202 x.x.46.70 UGH 0 828
y.y.64.203 x.x.46.231 UGH 0 1421
x.x.43.40 w.w.3.2 UGH 0 24
x.x.46.254 y.y.64.153 UGH 0 14035
z.z.2.12 z.z.2.14 U 3 0 qfe0:1
x.x.208.136 x.x.208.138 U 3 2291 qfe0
w.w.79.0 w.w.79.1 U 2 1355 qfe3
w.w.2.0 w.w.2.254 U 2 0 qfe1
w.w.3.0 w.w.3.1 U 2 55823 qfe2
x.x.40.0 x.x.46.23 U 39010880 hme0
y.y.64.0 y.y.64.1 U 34464039 hme0:1
default x.x.208.137 UG 06298303
127.0.0.1 127.0.0.1 UH 0 447082 lo0


# cat hosts
#
# Internet host table
#
127.0.0.1 localhost
x.x.46.23 ATL-FW-1 loghost
y.y.64.1 ALT_FW-1-RFC
x.x.208.138 qfe0
w.w.2.254 qfe1
w.w.3.1 qfe2
w.w.4.254 qfe3
x.x.44.132 mcl01_159
x.x.46.254 mcl04_159
z.z.2.14 qfe0-1


# cat hostname.qfe0
qfe0
# cat hostname.qfe0:1
qfe0-1
#


Contents of /etc/nsswitch.conf

passwd: files
group: files
hosts: files
networks: files
protocols: files
rpc: files
ethers: files
netmasks: files
bootparams: files
publickey: files
# At present there isn't a 'files' backend for netgroup; the system will
# figure it out pretty quickly, and won't use netgroups at all.
netgroup: files
automount: files
aliases: files
services: files
sendmailvars: files

 

[daFranze]

can I see some more?

route get z.z.2.13
snoop z.z.2.13 -> in another shell please ping z.z.2.13

since all I can see from the Sun is ok, I guess it's a routing back problem. z.z.2.13 send's it's answer to another port than z.z.2.14

Regards
-- Franz
Sorry I'm not a native spaeker, I'm from Munich, Germany - &quot;Home of the Whopper&quot;, oh no, &quot;Home of the Oktoberfest&quot; ;-)
Solaris System Manager; I used to work for Sun Microsystems Support (EMEA) for 5 years

 

[blainepruitt]

M8tt,

I had the same problem while adding an additional NIC to a 2.6 box last week. For some reason a reboot was the only way to fix it. We tried everything you did (plumb, unplumb, up, down, etc). Before rebooting if you haven't done so already make sure you create an /etc/notrouter file and add your new routes to /etc/gateways.

-HTH, good luck.
bp

 

[M8tt]

Franz,

Results below.

You will notice that the interface is now showing as qfe1:1. This is because I have moved the z.z.2.12/30 network from qfe0 to qfe1, which is spare and allows me to play with the interface a bit more - I've set-up a PC on this interface with the z.z.2.13 IP to replicate the Router and changed ifconfig and netstat accordingly.

# route get z.z.2.13
route to: z.z.2.13
destination: z.z.2.12
mask: 255.255.255.252
interface: qfe1:1
flags: <UP,DONE>
recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire
0 0 0 0 0 0 1500 0
#


# snoop z.z.2.13
Using device /dev/hme (promiscuous mode)
y.y.67.116 -> z.z.2.13 ICMP Echo request
y.y.67.116 -> z.z.2.13 ICMP Echo request
y.y.67.116 -> z.z.2.13 ICMP Echo request
y.y.67.116 -> z.z.2.13 ICMP Echo request

The related ping timed out.

 

[daFranze]

strange, 'route get' says, it will use qfe1:1 and snoop says the IP-package is leaving the host on hme0...

please run 'snoop -d qfe1:1' and ping z.z.2.13; next try to ping z.z.2.14 from the PC/Router

do you have traceroute on this host?

Maybe it is a good Idea to do, what blainepruitt did: reboot, so we are clean (caches etc.)

Regards
-- Franz
Sorry I'm not a native spaeker, I'm from Munich, Germany - &quot;Home of the Whopper&quot;, oh no, &quot;Home of the Oktoberfest&quot; ;-)
Solaris System Manager; I used to work for Sun Microsystems Support (EMEA) for 5 years

 

[M8tt]

Franz,

I have changed nothing since launching the route get and snoop commands yesterday however I can now ping the z.z.2.13 PC. The only thing I have done is turn it back on this morning.

The first 8 lines of the below appeared before I had launched the ping but you can now see the responses after that.

# snoop -d qfe1:1
Using device /dev/qfe (promiscuous mode)
z.z.2.13 -> z.z.2.15 UDP D=138 S=138 LEN=216
z.z.2.13 -> z.z.2.15 UDP D=137 S=137 LEN=58
z.z.2.13 -> z.z.2.15 UDP D=137 S=137 LEN=58
z.z.2.13 -> z.z.2.15 UDP D=137 S=137 LEN=58
z.z.2.13 -> z.z.2.15 UDP D=138 S=138 LEN=209
qfe1-1 -> (broadcast) ARP C Who is z.z.2.13, z.z.2.13 ?
z.z.2.13 -> qfe1-1 ARP R z.z.2.13, z.z.2.13 is z:zz:d0:51:da:be
y.y.67.116 -> z.z.2.13 ICMP Echo request
z.z.2.13 -> y.y.67.116 ICMP Echo reply
y.y.67.116 -> z.z.2.13 ICMP Echo request
z.z.2.13 -> y.y.67.116 ICMP Echo reply
y.y.67.116 -> z.z.2.13 ICMP Echo request
z.z.2.13 -> y.y.67.116 ICMP Echo reply
y.y.67.116 -> z.z.2.13 ICMP Echo request
z.z.2.13 -> y.y.67.116 ICMP Echo reply

I also launched it without specifying qfe1:1, and as you say the packet is leaving the host on hme0. Could this be because y.y.67.116 comes through hme0?

# snoop z.z.2.13
Using device /dev/hme (promiscuous mode)
y.y.67.116 -> z.z.2.13 ICMP Echo request
z.z.2.13 -> y.y.67.116 ICMP Echo reply
y.y.67.116 -> z.z.2.13 ICMP Echo request
z.z.2.13 -> y.y.67.116 ICMP Echo reply
y.y.67.116 -> z.z.2.13 ICMP Echo request
z.z.2.13 -> y.y.67.116 ICMP Echo reply
y.y.67.116 -> z.z.2.13 ICMP Echo request
z.z.2.13 -> y.y.67.116 ICMP Echo reply


I'm now sorry that I changed my interface from the live one to a spare one!

I will try reconfiguring z.z.2.14 on my live qfe0 interface to see if there is any difference now. If not I can try a reboot but am a bit reluctant since it is 18 months+ since it has been done.

Many Thanks to both blainepruitt and yourself for your help so far.

Matt.

 

[daFranze]

I also launched it without specifying qfe1:1, and as you say the packet is leaving the host on hme0. Could this be because y.y.67.116 comes through hme0?

# snoop z.z.2.13
Using device /dev/hme (promiscuous mode)
y.y.67.116 -> z.z.2.13 ICMP Echo request
z.z.2.13 -> y.y.67.116 ICMP Echo reply
y.y.67.116 -> z.z.2.13 ICMP Echo request
z.z.2.13 -> y.y.67.116 ICMP Echo reply
y.y.67.116 -> z.z.2.13 ICMP Echo request
z.z.2.13 -> y.y.67.116 ICMP Echo reply
y.y.67.116 -> z.z.2.13 ICMP Echo request
z.z.2.13 -> y.y.67.116 ICMP Echo reply

strange thing... are you doing NAT on these interfaces and send them somewhere else? BTW: this might be too much about a FW in an internet forum, but I would like to know if i am right...

I'm now sorry that I changed my interface from the live one to a spare one!

no problem, I think it's the problem of the arp cache, if you are switching to another interface better flush it... (man arp)


Regards
-- Franz
Sorry I'm not a native spaeker, I'm from Munich, Germany - &quot;Home of the Whopper&quot;, oh no, &quot;Home of the Oktoberfest&quot; ;-)
Solaris System Manager; I used to work for Sun Microsystems Support (EMEA) for 5 years

 

[M8tt]

Franz,

I do use the FW to NAT but only for specific IP addresses, and not for those IPs being used in the above test. No traffic on qfe1 should be affected, in fact yesterday was the first time this interface has ever been used.

Can you expand a little on what you mean by flushing the arp cache? And what commands you would recommend to do this?

Thanks again, Matt.

 

[daFranze]

i had to look in the manpage first...

arp -d hostname

deletes the arp entry on a Sun, you have to delete this on the Router;
your Router caches the MAC Address of your Sun it stores something like
inet z.z.2.14 ether 8:0:20:d0:53:79
when you move to another interface, such as your qfe1 (which is ether 8:0:20:d0:53:7a), the NIC has another MAC, finaly the transfere from peer-to-peer works with MAC Addresses but the router still finds the old MAC in it's Cache; I am not sure how long this entry lasts (aka "time to life"), on High Availability Systems the ARP Entry is deleted and set by a HA - Software...


Regards
-- Franz
Sorry I'm not a native spaeker, I'm from Munich, Germany - &quot;Home of the Whopper&quot;, oh no, &quot;Home of the Oktoberfest&quot; ;-)
Solaris System Manager; I used to work for Sun Microsystems Support (EMEA) for 5 years