Featured

Why Verizon is the worst. Literally.

The Intro


Before this story begins, the background… I’m not one to get a new phone every year, or even every couple of years. Right now I am still rocking my iPhone 5S from when they came out. When I get a new phone, it’s a big deal. This year with the iPhone X coming out I decided it was time to treat myself. Not only that, for the first time ever I wanted to be one of those select few able to have the phone on the first day it was available, before the phone got backordered. So I set my alarm for 2am CST on October 27, when the preorder process opened. Yay!

But 2am is where the pain began.

The Preorder

A minute or so before 2am, Verizon’s preorder page opened up. Already knowing exactly what options I wanted, I quickly clicked through the pages, until I got to the shipping confirmation page. Here’s where the problems began. Verizon’s pre order website had no way to specify a shipping address. The only option was to ship to the billing address — this was not where I needed the phone to end up. After going back a couple of times to be sure I hadn’t missed something, I opted to just place the order, then update the address through support. My biggest concern was getting my preorder in before the shipping dates started getting pushed out. As it turns out, by morning shipping dates for the iPhone X were pushed out 6 weeks.

So once my preorder was placed (scheduled to ship to the wrong address, I opened a chat support session with a Verizon representative). The representative quickly confirmed that an address change was not an issue and I could do so via phone support the next morning. So I happily went to bed.

At 7am the following day I called support and spoke with a friendly representative named Cawanna who happily took the corrected address for the preorder and assured me that everything was updated and I’d receive my phone. Everything was set! My iPhone X was preordered and would ship to my address on November 3, the first day the phones shipped out. I couldn’t have been happier!

But then came November 2. This is when FedEx tracking numbers started being sent out. When I checked the tracking information, it showed that my phone was being shipped to the billing address on the account. The wrong address. So I called Verizon support again with this information. A friendly representative named Gina reviewed my preorder and assured me that although the shipping information showed that the package was going to the wrong address, once the package was scanned to be delivered on the 3rd, this would be updated with the correct address. I skeptically accepted her word on this. I figured she should know more than I. Foolish assumption.

The Delivery (or Not)

Finally, Friday the 3rd rolls in. I took the day off of work in order to be able to receive the package since it required a signature.

After waking up, I opened up the FedEx tracking page to see where my shiny new iPhone was. It was on the truck to be delivered. In the wrong city.

Now came the 3rd call to support. This first call of the day took 1hr 28min. I went through 3 separate support representatives, up to a manager. During this call, the first revelation was that the fraud department had some kind of hold on the package. This was an excuse given multiple times throughout the process, as you will see later on.

Finally, when I got to the manager level, the answer I was given is that preorders cannot be rerouted and that the only option is to cancel the order and place it again. This was problematic given that 3 representatives to this point had told me the exact opposite, and at this point placing a new order would result in me waiting until somewhere around December 13 to receive my new phone.

At this point, I was given two options by the manager: 1) use Verizon’s “jump the line” process to get a phone next time there is stock being shipped out rather than waiting until everyone who placed an order up to that point received a phone — there was no cost to this option, but also no assurance when the phone would arrive or 2) since the phone was being shipped to an address where I knew the residents, they could receive the package and ship it to me. Given the fixed timeline of the second option, I reluctantly opted for that. I wasn’t going to get it the first day, but I knew I would receive it the first week. Better than nothing…

At this point I go into work to try and get something productive with what remained of the day since there would be no delivery today. During this time, my mother waited at the billing address, changing her day’s plans to intercept the package and ship it on to me.

Around noon, I got a text from my mother. The FedEx tracking page indicates that the order has been cancelled. I take to Twitter to find out why Verizon had cancelled my order. The FedEx Twitter account even stepped in, confirming that the shipper had cancelled the order.

By this point, I’m pretty ticked. What was supposed to be an exciting, painless experience was quickly turning into a nightmare.

The Nightmare

The Verizon Twitter account went inexplicably quiet, after having previously responded (albeit not usefully). So back to phone support I went.

I called in, and spoke to a representative. I immediately indicated that she would not be able to answer my query and that she needed to elevate the call. She assured me that she could help, so I explain the problem. Immediate response: “Please hold. I need to go speak to a manager.” Yes, that’s what I told you…

Anyway, after a long hold, she comes back on the line and indicates that the fraud department had intercepted the package and had it shipped back to Verizon. After I asked why, given that fraud had cleared everything up during my morning conversation, she forwarded me to the fraud department and I spent maybe 15min talking to a gentleman there. He confirmed that fraud had nothing to do with the package being turned around, but didn’t have access to information in my account necessary to determine who had or why. So then fraud bounces me back to general support.

At this point, the two options I had been provided this morning: 1) new order entered to “jump the line” or 2) have the package forwarded from the wrong address had been wittled down to one. Jump the line. So I start this process with the representative.

A little over 2 hours later, my new order was placed and a “jump the line” request had been submitted with an assurance from the support rep that the order should ship within a couple of days.

At the time of entry, the order showed the 6-week-out ship date of December 13, but the date would update once the “jump the line” form was processed.

The Nightmare – Day 2

The next day, Saturday the 4th, the ship date for my order still shows as December 13, so I call Verizon again to get a status update. I spoke first to a low level support representative, who seemed to have no clue what “jump the line” was and said that my order would ship on December 13. At this point, I lost it a bit… I told her that I needed to speak to a manager right then. Shortly after, a manager comes on the line.

This manager very shortly tells me that I should “just give up.” He said that my “jump the line” order request was rejected and that I would need to wait until the order shipped on December 13th. When I told him this was unacceptable (with a fair amount of yelling interlaced), he hung up on me.

Wow. So apparently if Verizon fails to ship to the correct location, then erroneously cancels your order, that is not justification to “jump the line.” This begs the question — what would justify it?

The Bowels of Hell

Having exhausted my options via phone/Twitter support, the only option remaining was to walk into the bowels of Hell — a Verizon store. So immediately following this phone call, I headed to the nearest Verizon store, where they called phone support, and I got to retell the same ever-growing story of what was going on with my order (let me tell you, I was really tired of explaining the situation by this point).

With the story told, the support representative begins digging through the history of my account. Apparently the information on status of a “jump the line” request is not available through an account history and is only known to the person who initiated the request. Seems efficient.

The only information available was a line item from a day earlier indicating that the “jump the line” request had been approved and then a manually-entered line item this morning from the manager I had spoken with indicating that the request had been rejected. No indication on reasoning, no nothing.

Conveniently, the team responsible for processing “jump the line” requests is not available to answer questions, so the reason for this rejection will likely remain a mystery.

The Last Stand

At this point, the service representative I was speaking with suggested I either 1) check local stores for a phone (we did this — nothing in my entire city) or 2) retry the “jump the line” option. The latter was suggested because it had initially been accepted and seemed likely some mistake was made at some point during the process, so it might be worth a try. I took the only option available — retry “jumping the line.”

The End

At this point, the end to this story is unknown. The “jump the line” entry was entered, but being the weekend, it’s unclear whether the request will be processed until Monday. The last support representative I spoke with promised to contact me as soon as he heard anything regarding the request, so at this point all I can do after almost an entire 8-hour work day on the phone with Verizon trying to fix this mess is to sit and wait and loathe their existence.

The one clear revelation from this experience, whatever the end result for my phone, is that Verizon’s various teams have no idea what the other teams are doing and their support staff and management do not all have the same information regarding policies, resulting in consistent misinformation being disseminated to customers trying to find out the status of their accounts. The amount of entirely conflicting information I received on things as basic as when an address can and cannot be changed for an order is truly disturbing and seems bound to limit the ability of a company to manage customer needs.

The End: Part II

Following the above, on November 8, 5 days after my phone was supposed to arrive, I got notification that the “Jump the Line” request had been approved and the shipping process was beginning for phone #2. Finally, afternoon of the 9th I had the phone in my hands.

I also neglected to mention above that Verizon did offer some compensation throughout this ordeal. I was initially given a $50 credit on my account, then after tweeting this article at their account, they followed that with offering to buy me a case for my phone, which I accepted. I maintain that for the more than 8 hours talking with their staff trying to get the situation resolved, the failure to deliver a phone on the day of release which on its own has significant value, and the extremely rude “customer service” throughout this process warrants a lot more than what they’ve given, but there isn’t much I can do beyond making sure everyone knows what happened.

The Remaining Questions

  • Why was my first “Jump the Line” request denied?
  • Why does Verizon allow its managers to be as rude as the one who told me I just needed to “give up”? Especially after a customer has been put through what I had?
  • Why did so many different Verizon employees have no clue where phone #1 was going? Even when I specifically indicated that it looked like it was going to the wrong place, I was assured that everything was fine.

Apache: Redirecting HTTP to HTTPS and WWW to Non-WWW

I was reading through the Apache documentation the other day as part of work on one of my pet projects (Thumber.co) and came across the following little tidbit (emphasis added).

mod_rewrite should be considered a last resort, when other alternatives are found wanting. Using it when there are simpler alternatives leads to configurations which are confusing, fragile, and hard to maintain.

Apache docs

Well, this got me thinking. I was in the process of updating Thumber to have everything redirect to HTTPS with no WWW prefix. I had implemented this functionality with mod_rewrite, but after reading the above I felt like making this better so I set out to convert redirects to use the basic Redirect directive, as provided by mod_alias.

My mod_rewrite starting point before any changes looked something like the following:

<VirtualHost *:80>
 ServerName thumber.co
 ServerAlias www.thumber.co

 RewriteEngine On
 RewriteRule ^ https://thumber.co%{REQUEST_URI} [L,QSA,R=301]
</VirtualHost>

<VirtualHost *:443>
 ServerName thumber.co
 ServerAlias www.thumber.co

 RewriteEngine On
 RewriteCond %{HTTP_HOST} ^www\.
 RewriteRule ^ https://thumber.co%{REQUEST_URI} [L,QSA,R=301]
</VirtualHost>

With the above, if the HTTP virtual host was hit, it would redirect to the non-WWW HTTPS equivalent  path. If the HTTPS virtual host was hit and www.thumber.co was the host then it would redirect to thumber.co. This seemed pretty clean as far as mod_rewrite go, but its still mod_rewrite. The following is what I came up with as a replacement.

<VirtualHost *:80>
 ServerName thumber.co
 ServerAlias www.thumber.co

 Redirect permanent / https://thumber.co/
</VirtualHost>

<VirtualHost *:443>
 ServerName thumber.co
 ServerAlias www.thumber.co

 <If "%{HTTP_HOST} == 'www.thumber.co'">
  Redirect permanent / https://thumber.co/
 </If>
</VirtualHost>

Much more straightforward! The only gotcha in the above is that the If directive is new to Apache 2.4. If you’re running an older version, you may still be stuck with mod_rewrite for the HTTPS virtual host, though the HTTP virtual host could still use the Redirect directive in an older Apache version.

Tunneling Through the Great Firewall of China Again

Nearly two years ago now, I wrote about tunneling through the Great Firewall of China (GFW). I recently revisited this topic and wanted to give a more detailed walkthrough of the process I took to make everything work.

Before I dive into the “how,” I first wanted to mention why I recommend SSH tunnels over standard VPN connections. There is really only one reason — China likes to mess with VPN connections (e.g.: here and here). By using SSH tunnels, we avoid this known issue with VPN solutions for circumventing the GFW. Additionally, even if China chose to try and prevent SSH tunneling, this would be nearly impossible due to one simple fact: VPN providers are publically known and a limited number of IP ranges, your personal SSH server is not publically known and could be any IP anywhere.

If you are convinced that this is the right approach for your tunneling needs, I highly recommend that you have multiple servers available. You never know when a server will go offline or when the Middle Kingdom will opt to block an IP and you do not want to find yourself stranded with no tunnel available. Redundancy is a must.

PuTTY Configuration

The first step is configuring the various PuTTY connections you plan to use. When setting up accounts on your server(s), I recommend making these accounts highly restricted as they only need to be able to port forward. This can be easily achieved using rbash, as described in Creating a Restricted SSH User for SSH Tunneling Only. Once your servers are setup with appropriate tunnel user accounts, you’ll want to configure your PuTTY settings as follows:

  1. Session: Set the host IP address (not hostname) and port number.
  2. Window -> Behavior: Uncheck “Warn before closing window.”
  3. Connection: Set “Seconds between keepalives” to 30 (or some other number as appropriate)
  4. Connection -> Data: Set “Auto-login username” to the user you’ve setup on the target server.
  5. Connection -> SSH: Check “Don’t start a shell command at all” and “Enable compression.”
  6. Connection -> SSH -> Tunnels: Set “Source port” to 8080, select “Dynamic” radio button, then click “Add”; you should now have “D8080” listed in the “Forwarded ports” box.
  7. Go back to Session, under “Saved Sessions” type the name you want this configuration saved as and click “Save.”
  8. For each of your remaining servers, change the hostname/port on Session tab and set the username on the Connection -> Data tab, then re-save with a new name (this avoids repeating all of the above steps for each server).
  9. * If you are doing this setup for a non-technical friend and cannot be at their computer for some reason, you can configure PuTTY with all of the servers to be used on your machine, then send the configurations to the friend. Unfortunately, PuTTY stores configurations in the registry, making it a bit tricky to access, but I used the export PowerShell script show here to create a reg file after doing all of the necessary configurations. I then sent the reg file to my friend, who can simply double click the file to merge the PuTTY settings into their registry.

Once PuTTY is configured, you’ll probably want an easy way to launch PuTTY each time you need to browse. There are many ways to do this, by my solution was a one-line batch file:
start "Proxy" "%~dp0putty.exe" -load hello-china -pw ******

In the above line, “hello-china” should be replaced with the name you saved your server configuration as in step #7 above. If you configured multiple servers, you can just create multiple batch files, one for each configuration. Note that this script expects putty.exe to be located in the same directory as the batch file. If it’s somewhere else, you should modify %~dp0putty.exe accordingly.

Firefox Configuration

As of writing this, Firefox is still the only mainstream browser I know of where SOCKS5 proxying is supported, which is necessary for remote DNS resolution — a requisite for this approach. Since I wrote the first post on this topic, Firefox has made the necessary settings much more accessible, no longer requiring searching through about:config settings.

To enable proxying through your SSH tunnel, perform the following steps:

  1. Launch Firefox and go click the top right button with three horizontal bars and in the resulting menu click “Options.”
  2. Go to “Advanced” in the left navigation bar, then click “Network” in the top navigation.
  3. Under the Connection header, click “Settings…”
  4. Select the “Manual proxy configuration” radio button and fill in SOCKS host as “127.0.0.1,” port as “8080,” then check the “Remote DNS” box.
  5. Click OK.

With all of this setup, launch one of your configured SSH tunnels and in Firefox visit ipecho.net and verify that your IP is reported as the IP address of the SSH server your are using to proxy connections. Further verify that if you kill your PuTTY connection and try to browse anywhere in Firefox you get a proxy error.

And that’s it. Happy browsing!

FFmpeg Supported MIME Types

I’m shocked… Maybe I’m missing some well-hidden FFmpeg flag, but for the life of me I can’t find a way to pull out the supported MIME types.

There are some commands that come sort of close, but not MIME types. For example, ffmpeg -formats will list supported formats, which give a general idea of what types of files can be encoded/decoded, but not MIME types.

After over an hour googling (I’m really persistent…), I’ve come to the conclusion that a clean solution doesn’t exist. For that reason, I’ve developed my own unclean solution. It’s dirty, but it meets my needs. Maybe someone out there will also find it useful.

The Downside of the WordPress Plugin Directory

One of the most powerful and useful parts of WordPress and other popular CMS software offerings is the seemingly endless number of available plugins to extend functionality in nearly any way you like. WordPress provides the Plugin Directory, where developers can publish their open source plugins free of charge for other users to download and use at no cost. In fact, I’ve contributed to the Plugin Directory with a number of offerings over the years, including Document Gallery, Hello Simpsons Chalkboard Gag, and Prezi Embedder. But, with all this power does come a downside…

As a site owner planning to use one of these plugins, you either have to read every line of code from the plugins you are planning to to use (and understand the code enough to spot any possible security vulnerabilities), or you have to trust that the plugin developer has made the code secure. If the developer was careless, your site could quickly be compromised (hacked!).

Continue reading “The Downside of the WordPress Plugin Directory”

Removing COM Module from Honeywell UtilityPRO

A few weeks ago, my apartment complex installed new UtilityPRO programmable thermostats in all of the units. As part of the CPS Energy Peak Saver program. Quite a convenient little gadget, but a little creepy too. With this thermostat, CPS has the ability to modify the temperature remotely without me knowing. Although I personally couldn’t care less, I spoke with some friends who didn’t like this idea. So that got me thinking — I wonder how difficult it would be to disable this remote access.

A little googling revealed that the thermostat I had was a Honeywell UtilityPRO, which handles remote communication through the ZigBee protocol. With this knowledge in hand, I did what any good computer scientist would do and ripped the damn thing apart (carefully, of course). What follows are the steps that I followed in removing the ZigBee COM module, which in turn disabled any remote communication with the thermostat.

Continue reading “Removing COM Module from Honeywell UtilityPRO”

OS-Agnostic Ghostscript Detection

In developing one of my more popular WordPress plugins, Document Gallery, I was faced with a challenge that stumped me for a bit. Recently, some of my users had asked for additional functionality to support auto-generation of thumbnails for the documents in their gallery. These documents could be any number of filetypes, but for this particular post we’ll be discussing PDF thumbnail generation.

PDFs are actually quite simple to thumbnail and the process is well-documented across the web. If I had been content with simply using the PHP Imagick extension to wrap my access to Ghostscript (GS), which does the actual conversion for PDFs, then the process of generating a thumbnail would have been quite straightforward, but I was not content with this solution, the primary reason being inefficiency. The way that Imagick processes PDFs before passing it down to GS is very slow. Thus, I went about setting up a framework to directly execute GS from PHP’s exec() with Imagick as a fallback in the event that I was unable to find a GS executable.

This all should have been quite simple, but it wasn’t. In order to accurately find and execute GS in a platform-agnostic way is actually quite challenging, and surprisingly poorly documented on the web. First, one must determine whether exec() is available. This is quite a convoluted thing to determine in-and-of-itself, but thankfully I found someone else’s solution for this and did not have to develop it myself. Finding the GS executable, however, was another story entirely. Continue reading “OS-Agnostic Ghostscript Detection”