Speed Test

I wanted to test my Internet speed at the office. I asked Verizon to provide the data but since they have time and more then 3 month later I'm still waiting, I started playing with some online tools.
I picked speakeasy's speed test as my testing tool and found some puzzling results...

I used my PC to test different browsers (FireFox 3.5.4 & IE6) using different path (firewall as default gateway & ISA 2006 proxy server with GFI WebMonitor 2009).
I have 2 T1s coming from Verizon on a multilink router. I'm not going to start calculate what I should expect, why and how. If you're into this data use a Bandwidth calculator.

First let's see how IE6 did with Proxy enabled

This is theoretically the slowest path since it's not only routed through another server before hitting the firewall but it is also processed by WebMonitor.

Disabling the proxy settings and running the same test on IE6 resulted in the following

While the upload speed was significantly faster I was surprised by the download result. Just can't be!

I had to test FireFox and compare the results

And once more just to make sure I'm getting valid numbers

At this point I realized that this test is worthless. Other then the local factors like other users and services that download & upload, the measurement is flexible and depend on so many factors that I just can't trust it.

Phones get static and drop calls

We're using an AVAYA S8400 server going out to 3 lines:
2 PRIs for local and long distance calls
T1 for International calls

2 weeks ago users started to complain about static on the line on local and long distance calls. At first when it was one or two users I thought it might be the phone though you expect a failed hardware to fail at all times and the static was random.
Then more and more users complained and some even added a new complain about drop calls.

In order to isolate the problem I have routed 2 heavily used area codes to the T1. Since no one reported problems on International calls it was a great way to find if the problem is local, equipment or infrastructure on my end or is it PRI\Verizon related issue.

Verizon which run both PRIs tested them after hours and reported back that both lines are clear.
At the same time no one complained about static at the 2 isolated area codes.

As the weekend passed we came back Monday morning to a clean line. No static. No drop calls. I changed nothing and yet the problem seem to be resolved...

Before I close this case I had to switch the 2 area codes back to the PRIs and make sure they are still clear. 4 days later the lines are clean.
To be on the safe side I asked Verizon to test the local equipment (that would be the demarc in my server room). Last night when the technician finished testing he confirmed it is clear.

While no one can explain it there is one last possible option, something we'll never know but can always blame: The building lobby is under renovations.
We used to have our office downtown where the infrastructure is old and fail whenever it's raining. Whenever we saw a Verizon truck within few blocks we anticipated phone or Internet problems. Unfortunately we where right most of the time. Taking this experience under consideration it is more then reasonable to think this option is possible. But as I said, we will never have an real answer.

Just Traceroute

Yesterday, just after 3pm our Internet connection had some major problems.
While browsing and most IM clients died, inbound mail still flowed and MSN Messenger was live at all times.
Testing the network I started with the immediate suspect for any Internet failure - the Proxy server. Seconds later, when I realized that Proxy was up and even when I bypass it I can't browse, I started testing my firewall and DMZ environment.
The way we're configured for Internet is a primary dual T1 connection going to Verizon's MFR router and a secondary T1 connection, both connecting to a DMZ switch which is also where my firewall hook in.
Using the trace command I was able to get all the way to the Verizon router beyond the locally installed MFR router. That indicated a Verizon side problem which is good on one side - My part of the network is working, but really frustrating on the other side - the side that need Verizon's help which is always a long process.
While on the phone with Verizon's support I was told it's a general outage and as we all know by now, it was impossible to get an explanation and\or a time frame. Luckily the problem was solved in 30 minutes and we didn't suffer significant damage.
While troubleshooting this situation I came across a cool useful website Just-traceroute that provide a platform to trace an IP from 4 different locations (USA, France, Singapore & Netherlands). This is handy to isolate many of the possible options. It also provide a built-in "send" button which you can use to email the output to yourself or any admin you're working with.

Verizon – a mystery even they can’t solve

Did you ever work with Verizon? It is not an easy task as Verizon has many departments which are not synchronized with each other so you’re routed from one place to another.
Here is a story I deal with for more then 3 month and the twist from this morning (don’t worry, it is just the beginning)
One of my traders has clients in Europe. He is dialing different countries all over West Europe and its working fine. We also have clients in South America, no problems there. Meddle East is also good and occasionally Asia and no problems there too.
On my end all these call are leaving my AVAYA PBX via Verizon LD (that is Verizon Long Distance, a department within the huge Verizon)
3 month ago this user started complaining that his calls to one country in Europe can’t go through. We dial over 10 times each time before it’s connected and when it finally connect the quality of the line is low.
I opened a ticket with Verizon every other week for 3 month and finally got a call back from their repair department (yes, a different department that has nothing to do with the LD guys)
I was so excited receiving the call that I wasn’t prepared for the surprise that was waiting: the Verizon technician told me that he cannot make the call via the Verizon network too!
Now I’m supposed to be happy that the problem is not on my end, right? WRONG!
He said they couldn’t make the call via Verizon LD and when he switched to a different LD carrier it worked fine. Then he sent me to call Verizon LD and most likely switch my LD service to another vendor!!! Unbelievable

Now part 2 of this story started and if it ever get a closure I’ll be back with all the details