NGX R62 on Nokia IP530 issue

[guez]

Hello,
We have a cluster of firewall checkpoint FW1 NGX R62 running on Nokia IP530 platforms. Since, I upgraded to R62 (from R55), every 3/4 days, the master firewall stop running correctly, it doesn't forward the traffic anymore but the hardware is still up. So the failover is not working. I have to switch to the backup one manually. Here, what can I see on the logs of the firewall when it fail:

Jan 7 01:42:51 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:42:55 egb [LOG_CRIT] kernel: cpas_pcb_connect: received NULL pcb as argument, failing
Jan 7 01:42:55 egb [LOG_CRIT] kernel: cpas_pcb_connect: received NULL pcb as argument, failing
Jan 7 01:42:58 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:42:58 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:42:58 egb [LOG_CRIT] kernel: cpas_pcb_connect: received NULL pcb as argument, failing
Jan 7 01:44:01 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:44:06 egb last message repeated 3 times
Jan 7 01:44:10 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (90, 0, 0)!
Jan 7 01:44:10 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:44:13 egb [LOG_CRIT] kernel: cpas_tcp_input: failed to allocate cpas_pkt_buf for in-sequence data
Jan 7 01:44:14 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:44:18 egb [LOG_CRIT] kernel: FW-1: fwtcpstr_init_stream: error allocating kbuf
Jan 7 01:44:19 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:44:21 egb [LOG_CRIT] kernel: cpas_tcp_input: failed to allocate cpas_pkt_buf for in-sequence data
Jan 7 01:44:26 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:46:25 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:46:25 egb last message repeated 5 times
Jan 7 01:46:28 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (91, 0, 0)!
Jan 7 01:46:28 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:46:28 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:46:31 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (79, 0, 0)!
Jan 7 01:46:31 egb [LOG_CRIT] kernel: FW-1: loghandle_create: failed to allocate memory for the log handle
Jan 7 01:46:31 egb last message repeated 2 times
Jan 7 01:46:31 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (91, 0, 0)!
Jan 7 01:46:34 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested
Jan 7 01:46:34 egb [LOG_CRIT] kernel: cpas_tcp_input: failed to allocate cpas_pkt_buf for in-sequence data
Jan 7 01:46:35 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested
Jan 7 01:46:37 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (91, 0, 0)!
Jan 7 01:47:23 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (79, 0, 0)!
Jan 7 01:47:26 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (79, 0, 0)!
Jan 7 01:47:32 egb [LOG_CRIT] kernel: FW-1: loghandle_create: failed to allocate memory for the log handle
Jan 7 01:47:32 egb [LOG_CRIT] kernel: FW-1: fw_xlate_packet: fw_xlate_log failed
Jan 7 01:47:35 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (90, 0, 0)!
Jan 7 01:47:35 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:47:38 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (90, 0, 0)!
Jan 7 01:47:39 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:47:39 egb last message repeated 4 times
Jan 7 01:47:45 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (79, 0, 0)!
Jan 7 01:47:46 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:47:58 egb last message repeated 4 times
Jan 7 01:48:05 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (29, 0, 0)!
Jan 7 01:48:05 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:48:43 egb last message repeated 2 times
Jan 7 01:48:48 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (32, 11, 0)!
Jan 7 01:48:48 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested
Jan 7 01:48:48 egb last message repeated 2 times
Jan 7 01:48:58 egb [LOG_CRIT] kernel: FW-1: loghandle_create: failed to allocate memory for the log handle
Jan 7 01:48:58 egb [LOG_CRIT] kernel: Error: FW-1 failed to generate the log record.
Jan 7 01:48:58 egb [LOG_CRIT] kernel: FW-1: fwx_create_xlbuf: error allocating kbuf
Jan 7 01:48:58 egb [LOG_CRIT] kernel: FW-1: fwx_get_xlation: fwxl is NULL
Jan 7 01:48:58 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:48:58 egb [LOG_CRIT] kernel: FW-1: fw_spii_new_h: failed to put data in kbuf.
Jan 7 01:48:58 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:49:17 egb last message repeated 5 times
Jan 7 01:49:20 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (29, 0, 0)!
Jan 7 01:49:22 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:49:22 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:49:35 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested
Jan 7 01:49:35 egb [LOG_CRIT] kernel: cpas_tcp_input: failed to allocate cpas_pkt_buf for in-sequence data
Jan 7 01:49:36 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:49:38 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (79, 0, 0)!
Jan 7 01:49:38 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:49:38 egb [LOG_CRIT] kernel: FW-1: loghandle_create: failed to allocate memory for the log handle
Jan 7 01:49:38 egb [LOG_CRIT] kernel: FW-1: fw_xlate_packet: fw_xlate_log failed
Jan 7 01:49:38 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested
Jan 7 01:49:38 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested
Jan 7 01:49:38 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:49:39 egb last message repeated 2 times
Jan 7 01:49:41 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (79, 0, 0)!
Jan 7 01:49:41 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:49:41 egb [LOG_CRIT] kernel: FW-1: loghandle_create: failed to allocate memory for the log handle
Jan 7 01:49:41 egb [LOG_CRIT] kernel: Error: FW-1 failed to generate the log record.
Jan 7 01:49:41 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation
Jan 7 01:49:44 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested
Jan 7 01:49:44 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (4, 0, 0)!
Jan 7 01:49:45 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation

If anyone have already see those error messages? I found nothing on internet and checkpoint KB.
Thanks for your help...

 

[FWHATER]

I'm not entirely sure, but it looks like your IP530 lacks sufficient memory to support the R62 upgrade. What are the system requirements for R62 and what is your IP530 running ?