Wait 20 Minutes and Try Again or How I Almost Got Fired...


Recommended Posts

This is a copy / paste from Reddit, i thought it was a good read!

Sometimes fellow techs that makes you want to smash your keyboard. (all names have been changed to protect the incompetent).

I used to work in a Service-Desk/Triage capacity (lovingly dubbed Tier 1.9) where we would monitor, ticket, troubleshoot, and escalate as needed (we were expected to have a 90% self-solution return).

One thing that we never were allowed to touch was network issues. Our area telcom team had lost their extremely competent team lead some 18 months earlier and their team had quickly degraded to people who didn't really seem to care about their job. They were also secure in the fact that the telcom system had degraded enough to be sufficiently obfuscated enough where it would take considerable resources to figure it out by a competent replacement. This made them act like they were untouchable. As such, I had learned fast to start recording my phone calls to them (thank you Google Voice!) so they couldn't pin anything on us.

Anyway, one day around 6PM the network dropped out. As such, I called the Telcom On-Call.

Me Leaving Voicemail: Hi, this is Craysh from Tier 1.9. It looks like the network is having issues again. Internet, communicator, and email are all unresponsive. All three webfarms are unavailable. Trace route to the gateway has failed. Ticket is unavailable as the ticketing system is inaccessable due to the issue. Please call back to confirm your taking ownership of the incident.

15 minutes go by and SOP dictates that I call back before escalating to their manager. The on-call answers.

Tech: On-Call Phone, this is Steve

Me: Hi Steve, have you gotten my voicemail? We're having issues with the network.

Tech: No, I haven't. Tell me what's wrong.

Me: Repeats what I said above.

Tech: Ok. Wait 20 minutes and try again. Click

I had assumed that by 'Wait 20 minutes' he meant that he was taking a look at it and that it should be back up in 20 minutes. 20 minutes go by and the network is still having issues (The network has now been down for the last 40 minutes or so)

I call the On-Call phone again. No answer.

Me Leaving Voicemail: Hi, this is Craysh again. The issue is still ongoing. Please call me back as soon as possible.

15 Minutes go by so it's time to call again. No answer. Time to call the acting supervisor.

Supervisor: This is Aaron

Me: Hi Aaron, this is Craysh from Tier 1.9. We are having issues with the network and the On-Call has stopped answering their phone.

Supervisor: Stopped? You mean they initially answered?

Me: Yes, he said to wait 20 minutes.

Supervisor: OK, he's probably working on it. Call me back in an hour if the issue persists. click

At this point I'm calling my supervisor and apprising him of the situation.

For the next hour, I continue to call the On-Call every 15 minutes (as is procedure). No answer.

I call the supervisor again

Supervisor: This is Aaron

Me: Hi Aaron, this is Craysh from Tier 1.9 again. We are still having issues with the network and the On-Call still has not answered his phone.

Supervisor: OK, I'll take a look at it.

Turns out that Telcom had pushed a policy to the Junipers that had scrubbed the white lists (completely blocking all access to all ports).

All in all it took 5 hours to get everything back up and running. I documented the ticket extremely thoroughly once the system was available again.

The next day came the fall-out. Telcom was claiming that I had never paged them, and they said that a customer ended calling the supervisor to find out what was going on. They even showed zero calls on their cellphone (they had removed the calls and messages).

As such, we had a meeting to 'discuss Tier 1.9's failure to report a severe incident'.

I walked into the room filled with extremely ****ed off big wigs, including Aaron and and Steve. They demanded why I hadn't contacted the on-call, and I stated that I had and that they had lead me to believe that they had been working on the issue.

Nobody believed me because "The On-Call phone didn't have any messages or call logs from Tier 1.9" Their excuse for having a completely empty call log was that they cleared it daily so they could keep track of the calls made on the phone.

I calmly walked to the overhead projector, logged into Google Voice, and played each consecutive voicemail/conversation.

They no longer work here, and I've had three promotions since.

Edited by John S.
properly tagged
  • Like 1

Good stuff.

I was once on the phone to Vodafone on behalf of my girlfriends mother whose contract had finished but she was still getting charged.

I recorded everything they said as i knew it was going to be a pain in the arse.

I was completely right and a dozen phone calls later no one at the company admitted to any call logs of previous correspondence between myself and Vodafone.

After playing back part of my conversation to the Indian chap on the phone, i was forwarded to another Indian chap and then another one and eventually ending up with a well spoken English chap who solved all my issues, closed the account, paid back all money owed and compensated us to the tune of ?150 for wasted time.

Sad fact is i had to go through all that for a simple account closure. There is no accountability any more.

Wow. Awesome story with a good ending. Incompetent people should realize that eventually their foolishness will catch up and bite them in the butt. Good to hear you've been promoted since then too! It sounds like management know when they've got someone who can do good work.

This topic is now closed to further replies.
  • Posts

    • Display Driver Uninstaller (DDU) 18.1.5.5 by Razvan Serea Display Driver Uninstaller (DDU) is a utility for completely removing AMD/NVIDIA/INTEL graphics drivers and related packages from your system, attempting to eliminate all leftovers (including registry entries, folders and files, driver store). Though AMD/NVIDIA/INTEL drivers can usually be removed via the Windows Control Panel, this uninstaller tool was created for situations where standard uninstall fails, or when you need to fully remove NVIDIA or ATI graphics card drivers. After using this driver cleaner, your system will behave as though it’s the first time you’re installing a new driver—similar to a fresh Windows installation. As with all such tools, we recommend creating a restore point beforehand, allowing you to undo changes if issues arise. If you're having trouble installing an older or newer driver, try it—there are reports that it resolves such problems. Recommended usage: The tool can be used in Normal mode but for absolute stability when using DDU, Safemode is always the best. Make a backup or a system restore (but it should normally be pretty safe). It is best to exclude the DDU folder completely from any security software to avoid issues. You do NOT need to uninstall the driver prior using DDU. Requirements: .NET Framework 4.8 Compatible with Windows 7, 8, 8.1, 10, and 11 (32-bit or 64-bit) Note: Using on Insider Preview builds is at your own risk. Display Driver Uninstaller (DDU) 18.1.5.5 changelog: Added 'Reset to recommended' button for the Options. General fixes and improvements. Download: Display Driver Uninstaller (DDU) 18.1.5.5 | 1.7 MB (Freeware) Download: DDU Portable | 1.2 MB Links: Display Driver Uninstaller Home Page | Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
    • WACUP 1.99.51.24568 Preview by Razvan Serea WACUP (WinAmp Community Update Project) is a modern, enhanced version of the classic Winamp music player, designed for better stability, performance, and compatibility. Built for Windows, WACUP retains the familiar Winamp interface while adding 64-bit support, bug fixes, and new features like improved audio format support, customizable skins, and optimized playlist management. Unlike bloated alternatives, WACUP focuses on lightweight performance and regular updates, making it the best choice for fans of the classic Winamp experience. Basically, if you miss the good old days of Winamp and want a modern upgrade that doesn’t mess things up, WACUP is for you! WACUP key features: Classic Winamp Feel – Keeps the familiar interface and functionality. Bug Fixes & Stability – Fixes old Winamp issues and improves performance. 64-Bit Support – Works better on modern systems. More Formats & Plugins – Supports additional audio formats and third-party plugins. Customizable UI – Skins and tweaks for a personalized look. Better Library Management – Improved playlists, media organization, and search. No Bloat – Focuses on performance without unnecessary extras. Regular Updates – Community-driven development with new features and fixes. WACUP 1.99.51.24568 Preview changelog: Fixed a deadlock seen from the recent crash reports when doing some of the drag + drop actions within the media library window Fixed a loading crash seen related to a problem with some of the artwork cache image files being restored which should now be better handled allowing for the bad image to be removed without it failing Fixed a deadlock seen from the recent crash reports when the internal metadata cache clearing is triggered which could block the main ui thread for too long with this now being moved to a background thread Fixed some performance issues with some of the methods related to determining artwork support which mainly affected the local library import / refresh (this is still slower for some compared to other players because there's more data & artwork aspects being checked for which means doing more processing on a single file despite the best of attempts to reduce duplicate / heavy processing where possible) Fixed a crash with the JTFE based missing files hotkey which no one seems to have used for an age for this to appear (maybe it's time to seriously consider stripping out features that aren't being used) Fixed how some of the file types which use extra information to reference their sub-songs is handled which was preventing some from being correctly resolved back to their base file (noticed fixing above) Fixed an issue with the handling of files with underscores in their filepath which wasn't being correctly handled causing some of the filename to be lost when shown as the title if title reading is delayed Fixed a few things that might be behind NotSoDirect not being stable for some setups though am still not certain that the changes done for this are going to fully resolve the problem from the crash reports Fixed the OS toast handling when there's no prior shortcut in the OS start menu to now create the shortcut (needed to allow the yes/no buttons for the new build / post-release toast) to be done as a hidden one so it's less likely to cause annoyance for those not wanting to see it whilst still allowing this less than ideal OS api implementation requirement to be met to avoid toasts without the needed buttons Fixed a regression when moving from taglib1 to taglib2 which broke some of the handling in place to allow for external programs to still access files when wacup has a held open cached instance of the file Everything else Updated cppwinrt (gen_win10shell.dll) to 3.0.260520.1 (26 May 2026) Updated libcurl (libcurl.dll) to 8.2.1 (24 Jun 2026) Updated Monkey's Audio (in_ape.dll) to 13.15 (28 Jun 2026) Updated mpg123 (mpg123.dll) to 1.33.6 (6 Jun 2026) Updated OpenSSL (libcurl.dll) to 3.5.7 (9 Jun 2026) Updated pugixml to 1.16 (16 Jun 2026) Updated taglib (tag2.dll) to 2.3.0 (11 May 2026) Updated vgmstream (in_vgmstream.dll) to the latest Git commit from 28 Jun 2026 Download: WACUP 64-bit | 9.6 MB (Freeware) Download: WACUP 32-bit View: WACUP Website | Screenshots Get alerted to all of our Software updates on Twitter at @NeowinSoftware
    • "over a thousand engineering hours" and started selling it but could not take a couple of minuets to send an AI email to ask permission. What an expensive lesson.
    • just tested it yesterday, a simple page with autoloading ADS takes 60mb....just 1 page for 60 megabytes.   poor people with a limited internet never will visit neolose
    • Tor Browser 15.0.17 by Razvan Serea Protect your privacy. Defend yourself against network surveillance and traffic analysis. Tor is a network of virtual tunnels that allows people and groups to improve their privacy and security on the Internet. The Tor software protects you by bouncing your communications around a distributed network of relays run by volunteers all around the world: it prevents somebody from watching your Internet connection and learning what sites you visit, it prevents the sites you visit from learning your physical location, and it lets you access sites which are blocked. The Tor Browser Bundle lets you use Tor on Windows, Mac OS X, or Linux without needing to install any software. It can run off a USB flash drive, comes with a pre-configured web browser to protect your anonymity, and is self-contained. Tor Browser 15.0.17 changelog: All Platforms Updated Tor to 0.4.9.11 Updated NoScript to 13.6.25.1984 Build System / All Platforms Bug tor-browser-build#41821: Update gpg subkeys for boklm Bug tor-browser-build#41827: Update morgan's keychain with renewed key Download: Tor Browser (64-bit) | Tor Browser (32-bit) | 109.0 MB (Open Source) View: Tor Browser Website | Other Operating Systems Get alerted to all of our Software updates on Twitter at @NeowinSoftware
  • Recent Achievements

    • Reacting Well
      Wakeen1966 earned a badge
      Reacting Well
    • Rookie
      Almohandis went up a rank
      Rookie
    • Apprentice
      jahara21 went up a rank
      Apprentice
    • Reacting Well
      NovaEdgeX earned a badge
      Reacting Well
    • Week One Done
      NovaEdgeX earned a badge
      Week One Done
  • Popular Contributors

    1. 1
      +primortal
      530
    2. 2
      +Edouard
      266
    3. 3
      PsYcHoKiLLa
      148
    4. 4
      Steven P.
      99
    5. 5
      macoman
      55
  • Tell a friend

    Love Neowin? Tell a friend!