Dead in 16 minutes?

[hrswift]

I have a Cisco 3002 VPN router that loses inside and outside connectivity precisely, to the second, every 16 minutes. It must be restarted to gain connections again. When it dies you can't ping it from the inside or the outside. Is there some kind of setting to turn off?
Weird huh!!

 

[KiscoKid]

Unfortunately I can't tell you what's causing the issue however I'm pretty certain it's some kind of configuration setting that's causing this due to predictable nature of the outage.

Out of interest.. when it dies... can devices on the same IP subnet resolve the 3000's IP address?

 

[JOAMON]

Kind of sounds like something is building up and eventually overloading the CPU. Prior to it dying do a show arp and see what your arp cache looks like. I have seen another router do almost the same thing when the default route was via and ethernet interface like:
ip route 0.0.0.0 0.0.0.0 ethernet0/0
This will ARP the world and grow a huge cache that can crash the router. Also do a show process CPU to see what process is loading up the CPU.

 

[hrswift]

When it dies you can't ping the internal IP from the inside either. This all started on 11/04/2005 at 8:44

 

[artesian2000]

Can we see the logs to see what might be happening please

 

[hrswift]

1 12/22/2005 08:15:44.970 SEV=1 EVENT/37 RPT=1
Reset Reason : 3 (Hardware-Reset)

2 12/22/2005 08:15:44.680 SEV=4 CONFIG/15 RPT=1
READ header End not found.

3 12/22/2005 08:15:44.910 SEV=4 HTTP/28 RPT=1
HTTP server starting.

4 12/22/2005 08:15:44.960 SEV=4 EVENT/15 RPT=1
Initializing event section in NVRAM

5 12/22/2005 08:15:45.070 SEV=3 IP/1 RPT=1
IP Interface 1 status changed to Link Up.

6 12/22/2005 08:15:45.370 SEV=3 IP/1 RPT=2
IP Interface 2 status changed to Link Up.

7 12/22/2005 08:15:45.880 SEV=3 IP/2 RPT=1
IP Interface 2 status changed to Link Down.

8 12/22/2005 08:15:48.880 SEV=3 IP/1 RPT=3
IP Interface 2 status changed to Link Up.

9 12/22/2005 08:15:50.070 SEV=5 SSL/5 RPT=1
SSL Initialized.

10 12/22/2005 08:15:50.550 SEV=4 TELNETDBG/1 RPT=1
Telnetd started

11 12/22/2005 08:15:50.550 SEV=4 TELNET/3 RPT=1
Telnetd server running

12 12/22/2005 08:15:50.790 SEV=4 SSH/2 RPT=1
SSH server starting.

13 12/22/2005 08:15:57.720 SEV=3 PPPOE/7 RPT=1
PPPoE Interface 1 status changed to Link Up.

14 12/22/2005 08:16:05.720 SEV=4 IKE/41 RPT=1 151.172.7.248
IKE Initiator: New Phase 1, Intf 12, IKE Peer 151.172.7.248
local Proxy Address 68.16.171.251, remote Proxy Address 151.172.7.248,
SA (ESP-3DES-MD5)

17 12/22/2005 08:16:06.050 SEV=5 IKEDBG/64 RPT=1 151.172.7.248
IKE Peer included IKE fragmentation capability flags:
Main Mode: True
Aggressive Mode: True

19 12/22/2005 08:16:06.680 SEV=5 IKE/115 RPT=1 151.172.7.248
Group [HW3002]
Client rejected NAT enabled IPSec request,
falling back to standard IPSec

21 12/22/2005 08:16:06.680 SEV=3 AUTH/24 RPT=1
Tunnel to headend device HW3002 connected

22 12/22/2005 08:16:06.680 SEV=4 IKE/119 RPT=1 151.172.7.248
Group [HW3002]
PHASE 1 COMPLETED

23 12/22/2005 08:16:06.750 SEV=5 IKE/73 RPT=1 151.172.7.248
Group [HW3002]
Responder forcing change of IKE rekeying duration from 2147483647 to 86400 secon
ds

26 12/22/2005 08:16:06.750 SEV=5 IKE/73 RPT=2 151.172.7.248
Group [HW3002]
Responder forcing change of IPSec rekeying duration from 2147483647 to 28800 sec
onds

29 12/22/2005 08:16:06.760 SEV=4 IKE/49 RPT=1 151.172.7.248
Group [HW3002]
Security negotiation complete for peer (HW3002)
Initiator, Inbound SPI = 0x22f9a2db, Outbound SPI = 0x0f3a5aa7

32 12/22/2005 08:16:06.770 SEV=4 IKE/120 RPT=1 151.172.7.248
Group [HW3002]
PHASE 2 COMPLETED (msgid=2d6c7cf7)

33 12/22/2005 08:16:10.770 SEV=4 IKE/41 RPT=2
IKE Initiator: New Phase 2, Intf 12, IKE Peer 151.172.7.248
local Proxy Address 10.105.3.0, remote Proxy Address 0.0.0.0,
SA (ESP-3DES-MD5)

35 12/22/2005 08:16:10.840 SEV=5 IKE/73 RPT=3 151.172.7.248
Group [HW3002]
Responder forcing change of IPSec rekeying duration from 2147483647 to 28800 sec
onds

38 12/22/2005 08:16:10.840 SEV=4 IKE/49 RPT=2 151.172.7.248
Group [HW3002]
Security negotiation complete for peer (HW3002)
Initiator, Inbound SPI = 0x215f5155, Outbound SPI = 0x765de01c

41 12/22/2005 08:16:10.850 SEV=4 IKE/120 RPT=2 151.172.7.248
Group [HW3002]
PHASE 2 COMPLETED (msgid=c2a7816f)

42 12/22/2005 08:21:27.250 SEV=4 TELNET/43 RPT=1
Telnet session established for peer 10.105.1.142, process TN01

43 12/22/2005 08:21:36.880 SEV=5 AUTH/36 RPT=1 10.105.1.142
User [ admin ] Protocol [ Telnet ] attempted ADMIN logon.
Status: <ACCESS GRANTED>

45 12/22/2005 08:21:36.880 SEV=4 AUTH/83 RPT=1
User [admin] connected

46 12/22/2005 08:26:41.010 SEV=3 PPPOE/8 RPT=1
PPPoE Interface 1 status changed to Link Down.

47 12/22/2005 08:26:41.010 SEV=4 IP/5 RPT=1
Client Static ARP delete of 0.0.0.0 failed for Interface 2.

48 12/22/2005 08:28:41.950 SEV=4 IKE/123 RPT=1 151.172.7.248
Group [HW3002]
IKE lost contact with remote peer, deleting connection (keepalive type: DPD)

50 12/22/2005 08:28:41.970 SEV=3 AUTH/25 RPT=1 151.172.7.248
Tunnel to headend device HW3002 disconnected: duration: 0:12:35

51 12/22/2005 08:29:53.400 SEV=4 AUTH/29 RPT=1
Management user admin disconnected: duration 0:08:16

52 12/22/2005 08:29:53.400 SEV=4 PSH/17 RPT=1
PSH - terminating PShell session tname SH01, ip addr 10.105.1.142

53 12/22/2005 08:30:08.380 SEV=5 TELNET/35 RPT=1
Telnets process TN01, peer 10.105.1.142: timeout waiting for Shell response

54 12/22/2005 08:30:08.380 SEV=4 TELNET/33 RPT=1
Telnet session terminated for peer 10.105.1.142, process TN01

 

[KiscoKid]

For quirky issues like this that follow no real rhyme or reason, I would usually advise to upgrade the software on the unit to the latest, most stable release.

Might be worth a try and seeing if the problem goes away or not.

 

[hrswift]

Is the ARP table as described by JOAMON?
ARP table

Intf Physical Address IP Address Mapping Type
---------------------------------------------------
Priv FF.FF.FF.FF.FF.FF 10.105.3.0 Static
Priv 00.05.31.A1.24.E6 10.105.3.1 Static
Priv 00.11.11.71.07.43 10.105.3.10 Dynamic
Priv 00.B0.D0.F1.8F.4F 10.105.3.102 Dynamic
Priv FF.FF.FF.FF.FF.FF 10.105.3.255 Static
Pub FF.FF.FF.FF.FF.FF 0.0.0.0 Static
Pub 00.05.31.A1.24.E7 0.0.0.1 Static
Pub FF.FF.FF.FF.FF.FF 0.255.255.255 Static
12 00 68.16.171.251 Static
12 00 68.216.193.1 Static


Routing table

IP Address Mask Next Hop Intf Protocol Age Metric
------------------------------------------------------------------------
0.0.0.0 0.0.0.0 68.216.193.1 12 Default 0 1
0.0.0.0 255.0.0.0 0.0.0.0 Pub Local 0 1
10.105.3.0 255.255.255.0 0.0.0.0 Priv Local 0 1
68.16.171.251 255.255.255.255 0.0.0.0 12 Local 0 1
192.168.1.249 255.255.255.255 0.0.0.0 Pub Static 0 1

 

[JOAMON]

KiscoKid mentioned obtaining the latest IOS for your platform and feature set. This would be the best place to begin. You may have a release with a bug in it. If that does not help take a look at your environment. Is this on a UPS? I have seen it were a similiar problem was caused by a room air conditioner cycling and causing a brown out that was not noticeable to the naked eye but did cause systems to restart. If nothing like that and you have tried the latest IOS and no problem with your ISP(like they would admit to it even if there was) then that leaves only the hardware itself. If you have smartnet then open a TAC case with Cisco. Might need to get that unit replaced.