Customer doesn't believe abrupt power outage damaged equipment your thoughts 9

[Rhinorhino]

Hi everyone. I have a customer with an Avaya IP office in a small hotel with 100 rooms analog handsets version 9.1.

I got to call at 5:00 in the morning on Saturday that the entire phone system was offline.

I showed up in the first thing I noticed was that they had changed television vendors from Cox to DirecTV and that the DirecTV vendor put a whole bunch of new rack mounted equipment in and plugged directly into the Avayas APC power supply backup UPS.

I'm assuming that there was an overload in an already overloaded Network closet.

When I turned the AVAYA back on after pulling their equipment off of the backup system I ended up with a couple of blown power supplies for the expansion modules and trouble with the PC that houses the voicemail pro.

I've had problems like this before but in this case the customer absolutely refuses to believe that simply unplugging or turning off a computer or phone system will cause damage like this. she's telling me this is absolutely not possible even though it happened.

I'm curious how you all will handle this or what you would say to the customer.

Is there any documentation from Avaya about abruptly shutting down phone systems.

The vendor at DirecTV who of course is not taking any responsibility is telling her that it's not their fault even though they caused the overload.

Thanks.

 

[SE_GA_IPO]

Is the IPO still in warranty period, is that why customer is skeptical? Are they trying to get out of paying for the repair?

 

[Traveler74]

Hopefully you took pictures of the closet before you did anything. If not then it may get more complicated. Sounds like DTV knew they messed up and tried to pawn it off on you guys by convincing the customer they did nothing wrong. Hopefully its under warranty and it covers situations like this if no it will get ugly. At this point I would not engage the customer any further until you brief your leadership on what's going on and get their guidance as this issue might be above your paygrade and might require the companies legal team to get involved with what it sounds like what the customer is trying to pull.

 

[gbiddlecombe]

We had several IP offices that redlighted if, when plugging in the system, the cord connected power, then jiggled enough to lose power then regained power as the plug was inserted in the outlet.

We learned to use the switch on the ups connected power strip. To turn on the power

 

[john3voltas]

When I turned the AVAYA back on after pulling their equipment off of the backup system I ended up with a couple of blown power supplies for the expansion modules and trouble with the PC that houses the voicemail pro.

Well, it seems to me we're talking about a couple of IP500 power supplies and a problem with the computer that houses the VMPro.
A couple of blown PSU's will cost peanuts.
OTOH you don't specify what kind of problem you're having with the VMPro server.
From what you're saying, if the system is a R9.1, that means it is dated. It was GA'd in late 2013 and was maintained until late 2017. Even if they bought in 2017, it is a 8 years old system. That's just for starters. No one can expect a system to work flawlessly for 8 years and not experience some issue, especially when it comes to power supply units.

I'm guessing those PSU's were already faulty for some time but never had to power-cycle because the UPS provided power at all times. That's happened to me on a couple of occasions.

Now, I'm guessing that if you replace de PSU's the IP500 and the expansion modules will start up just fine. Right? Can you confirm if you have already done that already?

Coming around the server issue, are you telling me that this adult person (your customer) has never had issues from restarting a computer without properly shutting it down?
Why does she think you've put a UPS in the first place? Have you told her the UPS was there to cope with minor power outages and more specifically to allow for proper system shutdown in case of long power outages?
It is a computer. It has a hard drive in it. That hard drive has a filesystem. That filesystem will very become corrupt over time whether it is FAT/NTFS/EXT3/EXT4. It doesn't matter. It's a filesystem that's had open files probably for years. Very easy to go corrupt and having to undergo maintenance.
It can also be a hdd malfunction. A 9yo harddrive will easily go bananas and decide to show all it's problems exactly when you need it to...boot up.

If she's a normal adult human being, she will understand all this. But I'm betting that she's not a normal adult human being, otherwise you wouldn't be here commenting with us.
Again, the couple of PSU's are a cheap replacement and you might have some in your workshop from old retired systems.
As for the server, only you can tell us what's wrong with it. If it's a simple filesystem corruption, just go ahead and reinstall the server and restore your backups. You do have backups, don't you?
If it's an hdd problem, you might have to ask her to chip in 80€ or something to replace the drive. It's not the end of the world.

Keep us up to date.
Cheers

 

[FractalTelecom]

To add to the comments already, do this:

Add up the amperage draw on every piece of equipment connected to the UPS, and the amperage pull on all the phone equipment.

Now, what is the amperage capacity of the UPS?


I always tell providers to get their own ups, or we provide a second one for them.

 

[IPOriface]

When I turned the AVAYA back on after pulling their equipment off

So the IPO & expansion modules were powered off when you arrived, you disconnected the DTV gear from the UPS, then how did you turn the Avaya gear back on? Unplug and re-plug to UPS?

Was there any damage to the UPS? I've hit dead UPS sockets more often than I've lost PSU's. A damaged UPS would be additional proof that your gear was impacted by external factors.

customer absolutely refuses to believe that simply unplugging or turning off a computer or phone system will cause damage

It shouldn't, but depending on how long that gear's been turned on for, powering it off, especially for an extended period can damage a PSU due to thermal contraction. I've not hit this a lot recently, but it was a big issue with the IP500v1 PSU's; much less so with the v2's. I have also lost expansion module PSU's here & there. A lot cheaper to replace one of them than replace the module lol.

trouble with the PC that houses the voicemail pro.

What sort of trouble? PSU dead? Won't POST? Data corruption?
If it's a dead PSU, then all signs point to a power surge than made it past the UPS and damaged 3 different pieces of equipment.

 

[Rhinorhino]

Thanks for the replies. I feel that it's coming down to the customer just does not want to look bad in front of her boss that she decided to switch to DTV and then this happened and is causing her to pay for repairs and her boss has to see the extra bill.

I feel she's trying to pretend it's faulty Avaya equipment and she should not have to pay.

DTV also keeps telling her impossible that by them plugging all their new devices into an existing vendors installation would cause an overload.

I decided not to meet the DTV tech onsite because it would just end up in a finger pointing match in front of the customer.

Yes I took lots of pictures before and after and posted notices not to plug anything into the installation.

It's the principle of the matter that it comes down to.