Sites and services server 2003 sp1 1

[asine]

Hi,
I need some help!
We have a forest with four sites. Each site has a single domain controller. All servers running server 2003 sp1. We use a VPN to connect the sites up, using watchguard firebox/soho6tcs.
No errors in any sites, apart from one.
Its is a KCC error, whereby replication is not occurring correctly - event id 1862.
However, when we force a replication the errors go away, so it points to a problem with the automatic rep.
The VPN is up and running, but it is confisuing the hell out of us.
Any help would be appreciated

Thanks

Paul

 

[adcfaq]

use dcdiag on all those DCs and repadmin/showreps to check out repl. post the log here for future anaylist.

---------------------------------------
Sr. Directory Services/Exchange Consultant

http://adcfaq.8m.com/

 

[asine]

Ok, the repadmin /showreps from the root domain controller

Bristol\FELIX

DC Options: IS_GC

Site Options: (none)

DC object GUID: db923430-xxxx-xxxx-xxxx-xxxx

DC invocationID: db923430-xxxx-xxxx-xxxx-xxxx



==== INBOUND NEIGHBORS ======================================



CN=Configuration,DC=europe,DC=com

Leeds\RISDALE via RPC

DC object GUID: b0143553-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 03:45:30 was successful.

London\LONSERV via RPC

DC object GUID: d54f8649-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 04:45:48 failed, result 1722 (0x6ba):

Can't retrieve message string 1722 (0x6ba), error 1815.

1 consecutive failure(s).

Last success @ 2005-07-11 15:15:32.

Madrid\ESP via RPC

DC object GUID: c7438bb8-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 05:45:31 was successful.



CN=Schema,CN=Configuration,DC=europe,DC=com

Leeds\RISDALE via RPC

DC object GUID: b0143553-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 03:45:30 was successful.

London\LONSERV via RPC

DC object GUID: d54f8649-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 04:46:09 failed, result 1722 (0x6ba):

Can't retrieve message string 1722 (0x6ba), error 1815.

1 consecutive failure(s).

Last success @ 2005-07-11 15:15:32.

Madrid\ESP via RPC

DC object GUID: c7438bb8-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 05:45:31 was successful.



DC=ForestDnsZones,DC=europe,DC=com

Leeds\RISDALE via RPC

DC object GUID: b0143553-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 03:45:30 was successful.

London\LONSERV via RPC

DC object GUID: d54f8649-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 04:45:48 failed, result 1256 (0x4e8):

Can't retrieve message string 1256 (0x4e8), error 1815.

1 consecutive failure(s).

Last success @ 2005-07-11 15:15:34.

Madrid\ESP via RPC

DC object GUID: c7438bb8-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 05:45:31 was successful.



DC=leeds,DC=europe,DC=com

Leeds\RISDALE via RPC

DC object GUID: b0143553-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 03:45:31 was successful.

London\LONSERV via RPC

DC object GUID: d54f8649-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 04:45:48 failed, result 1256 (0x4e8):

Can't retrieve message string 1256 (0x4e8), error 1815.

1 consecutive failure(s).

Last success @ 2005-07-11 15:15:32.

Madrid\ESP via RPC

DC object GUID: c7438bb8-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 05:45:31 was successful.



DC=london,DC=europe,DC=com

Leeds\RISDALE via RPC

DC object GUID: b0143553-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 03:45:31 was successful.

London\LONSERV via RPC

DC object GUID: d54f8649-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 04:45:48 failed, result 1256 (0x4e8):

Can't retrieve message string 1256 (0x4e8), error 1815.

1 consecutive failure(s).

Last success @ 2005-07-11 15:15:34.

Madrid\ESP via RPC

DC object GUID: c7438bb8-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 05:45:32 was successful.



DC=madrid,DC=europe,DC=com

Leeds\RISDALE via RPC

DC object GUID: b0143553-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 03:45:31 was successful.

London\LONSERV via RPC

DC object GUID: d54f8649-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 04:45:48 failed, result 1256 (0x4e8):

Can't retrieve message string 1256 (0x4e8), error 1815.

1 consecutive failure(s).

Last success @ 2005-07-11 15:15:34.

Madrid\ESP via RPC

DC object GUID: c7438bb8-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 05:45:32 was successful.



Source: London\LONSERV

******* 1 CONSECUTIVE FAILURES since 2005-07-11 15:15:34

Last error: 1256 (0x4e8):

Can't retrieve message string 1256 (0x4e8), error 1815.

*******

And this is from the failing child domain controller:

******


London\LONSERV

DC Options: IS_GC

Site Options: (none)

DC object GUID: d54f8649-xxxx-xxxx-xxxx-xxxx

DC invocationID: 88f0185f-xxxx-xxxx-xxxx-xxxx



==== INBOUND NEIGHBORS ======================================



CN=Configuration,DC=europe,DC=com

Bristol\FELIX via RPC

DC object GUID: db923430-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 04:50:55 failed, result 1722 (0x6ba):

Can't retrieve message string 1722 (0x6ba), error 1815.

1 consecutive failure(s).

Last success @ 2005-07-11 10:16:05.



CN=Schema,CN=Configuration,DC=europe,DC=com

Bristol\FELIX via RPC

DC object GUID: db923430-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 04:51:16 failed, result 1722 (0x6ba):

Can't retrieve message string 1722 (0x6ba), error 1815.

1 consecutive failure(s).

Last success @ 2005-07-11 10:16:06.



DC=ForestDnsZones,DC=europe,DC=com

Bristol\FELIX via RPC

DC object GUID: db923430-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 04:50:55 failed, result 1256 (0x4e8):

Can't retrieve message string 1256 (0x4e8), error 1815.

1 consecutive failure(s).

Last success @ 2005-07-11 10:16:06.



DC=europe,DC=com

Bristol\FELIX via RPC

DC object GUID: db923430-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 04:50:55 failed, result 1256 (0x4e8):

Can't retrieve message string 1256 (0x4e8), error 1815.

1 consecutive failure(s).

Last success @ 2005-07-11 10:16:06.



DC=leeds,DC=europe,DC=com

Bristol\FELIX via RPC

DC object GUID: db923430-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 04:50:55 failed, result 1256 (0x4e8):

Can't retrieve message string 1256 (0x4e8), error 1815.

1 consecutive failure(s).

Last success @ 2005-07-11 10:16:05.



DC=madrid,DC=europe,DC=com

Bristol\FELIX via RPC

DC object GUID: db923430-xxxx-xxxx-xxxx-xxxx

Last attempt @ 2005-07-12 04:50:55 failed, result 1256 (0x4e8):

Can't retrieve message string 1256 (0x4e8), error 1815.

1 consecutive failure(s).

Last success @ 2005-07-11 10:16:06.



Source: Bristol\FELIX

******* 1 CONSECUTIVE FAILURES since 2005-07-11 10:16:06

Last error: 1256 (0x4e8):

Can't retrieve message string 1256 (0x4e8), error 1815.


*****

All other sites were totally successful

Thanks for your time

Paul

 

[adcfaq]

yes, felix and LONSERV lost sync, that's kcc error kicks in.

u need to dcdiag felix and LONSERV to take a look at.

also, delete the connect in dssite.msc for those 2 DCs, and use "repadmin/kcc" to trigger the connection creation, see what happen, until u succesfully remove the error in repadmin/showreps, u have AD repl issue for sure.



---------------------------------------
Sr. Directory Services/Exchange Consultant

http://adcfaq.8m.com/

 

[asine]

Hi benlu,
Thanks for your response.
In terms of deleting a connection in dssite.msc, do you mean the <automatically generated> connection?
If so, because the link is down, we get an 'object does not exist' error.

Paul

 

[asine]

we deleted the connection, and used the repadmin /kcc. I think that this has solved the problem for the time being. However I imagine that tomorrow the errors will be back!

Do you think that because we have scheduled the replication to occur only once a day, this may be a problem?

Thanks

Paul

 

[asktheman]

Why not delete all of the automatically created connector between site and manually create a hub and bespoke setup.

KCC does not always give you the best replication path.

So created a connection on each of the sites looking back at your main office DC', this will ensure that all replication is push via your main office. At the main offie created (manually) link to all of your remote sites DC's. this will give you a few errors in the event log once created but you will see a improvment in Replication after the first hour.

Do you have you domain and forest running 2K3 native mode. ? as this also improve replication

 

[asine]

Hi asktheman,

We tried deleting the links between LONSERV and FELIX, at around lunchtime yesterday, as suggested by benlu, but by 11pm the following event ids appeared :
1311, 1865

We will try your idea of manually entering the site links. However, we are totally confused as to why this is happening at just one site, and not the others. Any other ideas?

Thanks

Paul

 

[asktheman]

is the routing the same in to this site. are the router options configured the same

 

[asine]

yes they are as far as i know.
we're using watchguard fb 700 and soho 6tc setup the same.
the routers are different although we're passing everything through to the firewall.

 

[asine]

we have tried all of the above ideas, but to no avail. We can manually force a replication and everything is ok, then it all goes t*ts up.

so thought i would post the dcdiag from the rdc:

Domain Controller Diagnosis

Performing initial setup:
Done gathering initial info.

Doing initial required tests

Testing server: Bristol\FELIX
Starting test: Connectivity
......................... FELIX passed test Connectivity

Doing primary tests

Testing server: Bristol\FELIX
Starting test: Replications
[Replications Check,FELIX] No replication recently attempted:
From RISDALE to FELIX
Naming Context: DC=ForestDnsZones,DC=europe,DC=com
The last attempt occurred at 2005-07-18 03:45:37 (about 6 hours ago)
.
[Replications Check,FELIX] A recent replication attempt failed:
From LONSERV to FELIX
Naming Context: DC=ForestDnsZones,DC=europe,DC=com
The replication generated an error (1256):
Win32 Error 1256
The failure occurred at 2005-07-18 04:45:54.
The last success occurred at 2005-07-11 15:15:34.
7 failures have occurred since the last success.
[Replications Check,FELIX] No replication recently attempted:
From ESP to FELIX
Naming Context: DC=ForestDnsZones,DC=europe,DC=com
The last attempt occurred at 2005-07-18 05:45:40 (about 4 hours ago)
.
[Replications Check,FELIX] No replication recently attempted:
From RISDALE to FELIX
Naming Context: CN=Schema,CN=Configuration,DC=europe,DC=com
The last attempt occurred at 2005-07-18 03:45:37 (about 6 hours ago)
.
[Replications Check,FELIX] No replication recently attempted:
From LONSERV to FELIX
Naming Context: CN=Schema,CN=Configuration,DC=europe,DC=com
The last attempt occurred at 2005-07-18 04:46:00 (about 5 hours ago)
.
[Replications Check,FELIX] No replication recently attempted:
From ESP to FELIX
Naming Context: CN=Schema,CN=Configuration,DC=europe,DC=com
The last attempt occurred at 2005-07-18 05:45:38 (about 4 hours ago)
.
[Replications Check,FELIX] No replication recently attempted:
From RISDALE to FELIX
Naming Context: CN=Configuration,DC=europe,DC=com
The last attempt occurred at 2005-07-18 03:45:37 (about 6 hours ago)
.
[Replications Check,FELIX] A recent replication attempt failed:
From LONSERV to FELIX
Naming Context: CN=Configuration,DC=europe,DC=com
The replication generated an error (1722):
Win32 Error 1722
The failure occurred at 2005-07-18 04:45:54.
The last success occurred at 2005-07-11 15:15:32.
7 failures have occurred since the last success.
The source LONSERV is responding now.
[Replications Check,FELIX] No replication recently attempted:
From ESP to FELIX
Naming Context: CN=Configuration,DC=europe,DC=com
The last attempt occurred at 2005-07-18 05:45:38 (about 4 hours ago)
.
[Replications Check,FELIX] No replication recently attempted:
From RISDALE to FELIX
Naming Context: DC=leeds,DC=europe,DC=com
The last attempt occurred at 2005-07-18 03:45:37 (about 6 hours ago)
.
[Replications Check,FELIX] A recent replication attempt failed:
From LONSERV to FELIX
Naming Context: DC=leeds,DC=europe,DC=com
The replication generated an error (1256):
Win32 Error 1256
The failure occurred at 2005-07-18 04:45:54.
The last success occurred at 2005-07-11 15:15:32.
7 failures have occurred since the last success.
[Replications Check,FELIX] No replication recently attempted:
From ESP to FELIX
Naming Context: DC=leeds,DC=europe,DC=com
The last attempt occurred at 2005-07-18 05:45:40 (about 4 hours ago)
.
[Replications Check,FELIX] No replication recently attempted:
From RISDALE to FELIX
Naming Context: DC=london,DC=europe,DC=com
The last attempt occurred at 2005-07-18 03:45:37 (about 6 hours ago)
.
[Replications Check,FELIX] A recent replication attempt failed:
From LONSERV to FELIX
Naming Context: DC=london,DC=europe,DC=com
The replication generated an error (1256):
Win32 Error 1256
The failure occurred at 2005-07-18 04:45:54.
The last success occurred at 2005-07-11 15:15:34.
7 failures have occurred since the last success.
[Replications Check,FELIX] No replication recently attempted:
From ESP to FELIX
Naming Context: DC=london,DC=europe,DC=com
The last attempt occurred at 2005-07-18 05:45:40 (about 4 hours ago)
.
[Replications Check,FELIX] No replication recently attempted:
From RISDALE to FELIX
Naming Context: DC=madrid,DC=europe,DC=com
The last attempt occurred at 2005-07-18 03:45:38 (about 6 hours ago)
.
[Replications Check,FELIX] A recent replication attempt failed:
From LONSERV to FELIX
Naming Context: DC=madrid,DC=europe,DC=com
The replication generated an error (1256):
Win32 Error 1256
The failure occurred at 2005-07-18 04:45:54.
The last success occurred at 2005-07-11 15:15:34.
7 failures have occurred since the last success.
[Replications Check,FELIX] No replication recently attempted:
From ESP to FELIX
Naming Context: DC=madrid,DC=europe,DC=com
The last attempt occurred at 2005-07-18 05:45:41 (about 4 hours ago)
.
REPLICATION-RECEIVED LATENCY WARNING
FELIX: Current time is 2005-07-18 09:51:40.
DC=ForestDnsZones,DC=europe,DC=com
Last replication recieved from LONSERV at 2005-07-11 15:15:33.
CN=Configuration,DC=europe,DC=com
Last replication recieved from LONSERV at 2005-07-11 15:15:32.
DC=london,DC=europe,DC=com
Last replication recieved from LONSERV at 2005-07-11 15:15:34.
......................... FELIX passed test Replications
Starting test: NCSecDesc
......................... FELIX passed test NCSecDesc
Starting test: NetLogons
......................... FELIX passed test NetLogons
Starting test: Advertising
......................... FELIX passed test Advertising
Starting test: KnowsOfRoleHolders
......................... FELIX passed test KnowsOfRoleHolders
Starting test: RidManager
......................... FELIX passed test RidManager
Starting test: MachineAccount
......................... FELIX passed test MachineAccount
Starting test: Services
......................... FELIX passed test Services
Starting test: ObjectsReplicated
......................... FELIX passed test ObjectsReplicated
Starting test: frssysvol
......................... FELIX passed test frssysvol
Starting test: frsevent
......................... FELIX passed test frsevent
Starting test: kccevent
An Warning Event occured. EventID: 0x8000061E
Time Generated: 07/18/2005 09:39:01
Event String: All domain controllers in the following site that
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC000051F
Time Generated: 07/18/2005 09:39:01
Event String: The Knowledge Consistency Checker (KCC) has
An Warning Event occured. EventID: 0x80000749
Time Generated: 07/18/2005 09:39:01
Event String: The Knowledge Consistency Checker (KCC) was
An Warning Event occured. EventID: 0x8000061E
Time Generated: 07/18/2005 09:39:01
Event String: All domain controllers in the following site that
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC000051F
Time Generated: 07/18/2005 09:39:01
Event String: The Knowledge Consistency Checker (KCC) has
An Warning Event occured. EventID: 0x80000749
Time Generated: 07/18/2005 09:39:01
Event String: The Knowledge Consistency Checker (KCC) was
An Warning Event occured. EventID: 0x8000061E
Time Generated: 07/18/2005 09:39:01
Event String: All domain controllers in the following site that
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC000051F
Time Generated: 07/18/2005 09:39:01
Event String: The Knowledge Consistency Checker (KCC) has
An Warning Event occured. EventID: 0x80000749
Time Generated: 07/18/2005 09:39:01
Event String: The Knowledge Consistency Checker (KCC) was
An Warning Event occured. EventID: 0x8000061E
Time Generated: 07/18/2005 09:39:01
Event String: All domain controllers in the following site that
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC000051F
Time Generated: 07/18/2005 09:39:01
Event String: The Knowledge Consistency Checker (KCC) has
An Warning Event occured. EventID: 0x80000749
Time Generated: 07/18/2005 09:39:01
Event String: The Knowledge Consistency Checker (KCC) was
An Warning Event occured. EventID: 0x8000061E
Time Generated: 07/18/2005 09:39:01
Event String: All domain controllers in the following site that
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC00006FC
Time Generated: 07/18/2005 09:39:01
Event String: When processing the topology for partition
An Error Event occured. EventID: 0xC000051F
Time Generated: 07/18/2005 09:39:01
Event String: The Knowledge Consistency Checker (KCC) has
An Warning Event occured. EventID: 0x80000749
Time Generated: 07/18/2005 09:39:01
Event String: The Knowledge Consistency Checker (KCC) was
......................... FELIX failed test kccevent
Starting test: systemlog
......................... FELIX passed test systemlog
Starting test: VerifyReferences
......................... FELIX passed test VerifyReferences

Running partition tests on : ForestDnsZones
Starting test: CrossRefValidation
......................... ForestDnsZones passed test CrossRefValidation

Starting test: CheckSDRefDom
......................... ForestDnsZones passed test CheckSDRefDom

Running partition tests on : DomainDnsZones
Starting test: CrossRefValidation
......................... DomainDnsZones passed test CrossRefValidation

Starting test: CheckSDRefDom
......................... DomainDnsZones passed test CheckSDRefDom

Running partition tests on : Schema
Starting test: CrossRefValidation
......................... Schema passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... Schema passed test CheckSDRefDom

Running partition tests on : Configuration
Starting test: CrossRefValidation
......................... Configuration passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... Configuration passed test CheckSDRefDom

Running partition tests on : europe
Starting test: CrossRefValidation
......................... europe passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... europe passed test CheckSDRefDom

Running enterprise tests on : europe.com
Starting test: Intersite
......................... europe.com passed test Intersite
Starting test: FsmoCheck
......................... europe.com passed test FsmoCheck

thanks kindly to anyone who can help

Paul

 

[asine]

noone any ideas?

 

[NetIntruder]

Ok, to help you fully troubleshoot this beast...

http://www.microsoft.com/downloads/...7c-7ca5-408f-88b7-f9c79b7306c0&displaylang=en

Download the MPSRPT DriSvc.exe and run it against the failing DC and the replication partner. This will wrap all of the logs into a nice .cab file - search the logs on both ends for failures. Most of the AD Problems i've had in the past years i've been able to find a solution based on these reports.

~Intruder~
CEH, MCSA/MCSE 2000/2003

"The Less You Do, The Less Can Go Wrong"