Freaky Metal Words

Haha, so long Matt Leinart! I guess the joke is on the Cards who spent a bunch of money on your whining and beer bongs.

It is very satisfying to see the alert pop up after tweaking the last bit of JavaScript on the XSS that just didn’t want to reveal itself.

Every so often, I think I’m about to escape, but I get trapped again.

Let’s start from the beginning…

A couple of years ago I was brand new and fresh out of quitting teaching (this post is about a figurative IT nightmare, I still have literal teaching nightmares).  My job was performance testing the applications at my company.  In order to accurately model the load, I had an obvious question of “What does the usage in production look like?”  The answer was “Nobody knows.”  The company is neither small enough nor new enough for that answer to make any sense, but I had to dig anyway “How can I find out?  Can I get some logs or something?”  I was then told to request access to the Google Analytics account, which supposedly held the answers.

I logged in a peaked around to find that it was in a state of serious neglect.  There were no filters in place.  Traffic was randomly segmented into different profiles using different tracking codes not for any logical reason but they just added the JavaScript at different times and didn’t know what they were doing.  Most of the traffic was recorded as the non-helpful “(other).”  I’d later discover that the JavaScript was even worse.  There were pages with multiple, sequential setVar() calls.  It appeared in some places that they were also trying to use setVar to differentiate multiple views on a single page.

I needed some real numbers and this seemed to be the only way to get them, so I spent a day or two reading some Google Analytics documentation.  I found some things that I thought would clean up the data a lot so I started shopping them around.  Every person I talked to told me to talk to somebody else.  Eventually I found a person who worked with the usability group.  She wanted numbers as well.  She didn’t have any experience as a Google Analytics administrator, but she had admin access to the account.  Through her I was able to get some simple filters in place (everything to lowercase, removing the query strings, identifying default pages being reported twice as the directory root).

As things began to clear up, more problems would become apparent.  Eventually my admin pal got tired of me telling her to add filters, so I was granted access so that I could add them myself.  Certain problems took longer to figure out than the initial issues, so I did more reading.  You probably see my mistake already, but I was new, I didn’t.  I got some help from Justin Cutroni by posing some questions in the comments on his site.  I read Brian Clifton‘s book.  I read Avinash Kaushik‘s book.  I also got my manager to send me to a Google Analytics Seminar for Success.  My primary work responsibility was still testing software, but tasks came around sporadically at that time, so I didn’t mind filling in the down time with the analytics stuff.  I was a math nerd before I was a computer nerd, so the wall of statistics was interesting.

My Google Analytics pal from the usability team and I went through and cleaned up the data as much as we could.  Many new profiles were created, lots of filters were added.  We pushed the development teams to correct the JavaScript.  We figured out what a deceptive little goon setVar really was.  Seriously, what the hell, Google?  It caused interactions to be recorded which completely subverted the filtering on different profiles.  The visit numbers would show thousands of visits and no page views.  Through testing we figured this out, but the documentation provided no good explanation about the side effects.

Then people started coming to us with questions about obtaining more data.  How should they implement the JavaScript?  Could we filter X?  On the technical side, this was perfectly easy.  I was capable of telling anyone how they could obtain (if it could be done) the data they wanted using Google Analytics.  The problem for me was/is that I am at the complete bottom of the org chart here.  I don’t make business decisions with the data.  I know people do.  I’ve heard them talk.  I cringe when I hear GA numbers being thrown around.  Numbers that are clearly wrong.  Numbers that don’t mean what they think they mean.  Since I am not in touch with how the business wants to use these numbers, or what data is relevant to them, it’s difficult to answer GA questions which will determine the data available for everyone.  It’s tricky because there’s no going back.  The data cannot be parsed again or filtered differently.  Also, our site spans several applications and multiple domains and sub domains.  One development team’s poor handling is capable of wrecking the numbers for the others.

I’ve been trying to express this.  I’m a tester.  I’m not even a performance tester anymore.  I have no need for the numbers myself.  I have no direct line of communication to any important decision making people, but I control the data.  The numbers they see are the ones I configured it for them to see.  I wrote a whole bunch of regular expressions that move a bunch of strings around.  Nobody else verified them.  Nobody else knows what they do.  I documented them and pointed people to them.  I tried to do presentations to explain what I’d learned so that someone else would be capable of maintaining the account.  At the very least they should double check the things that I’ve done.  The result of the presentation was pretty much “It sounds like you know what you’re talking about.  How about you just keep it?”  D’oh!

As my new testing responsibilities have been more consistent than my old ones, I’ve had no time for analytics.  I tried to bring it up, but I was told that they’d be switching to Coremetrics because GA doesn’t do what they want (like they have any clue what GA does).  Allegedly there exists someone in the organization qualified and willing to be the steward of Coremetrics.  Months, then years go by and people keep dumping more and more into Google Analytics.  Coremetrics is still not a significant part of their data collection.  I peak in every now and then and see the once tidy organization, falling back into the state of chaos which existed back when I first found it.  In one place they are using it as a security audit log.  I hope your eyes didn’t roll right out of your head when you read that.  Mine almost did.  The head of product development is ok with that though.  I spoke with him directly about my data concerns.  They’re switching to Coremetrics… I don’t need to worry about it.

Recently I got a new manager.  On my behalf he found a new home for Google Analytics maintenance.  Supposedly, I’ll finally be handing that responsibility to a more suitable person in the near future.  As though they sensed it coming, one of the dev teams I’m supposed to do testing for has requested a bunch of changes to the filtering in GA.  I haven’t had any time to do the knowledge transfer, so I’ll have to do it.  Just when I thought I was one step from done, the end moves back from me.

Nobody wants to own it.  Everyone wants the numbers.  Nobody knows what they mean.  I can’t believe this is a real business and not The Office, Office Space or Dilbert.

Motivation:

Recently I decided that it was time that I give Selenium RC a shot.  I was attracted to the idea of recording scripts with Firefox and then working with them in a language of my choosing.  All the cool kids at Defcon seem to think Python is the answer to everything, but since that’s still on my ToLearn list, I figured I’d default to the more familiar Java where I’m already comfortable.  The option to do that was a big selling point for me.  Since I’m looking at a JavaScript (using Strophe.js) jabber client, I was also happy to let Firefox deal with that.  Another coworker is developing LoadRunner scripts for performance tests on the ejabberd server.  Essentially his scripts must duplicate the basic functionality of the JavaScript code in LoadRunner’s C.  Gross.  Initially I thought I might borrow his scripts and tweak them for my own testing, but I realized that I could jump well ahead of him if the Selenium idea panned out.  No redoing the JavaScript logic.  No C code.  Win.

A few days of puzzled stares at Eclipse and a hundred Google searches later, I think I’m still winning.  It was not quite the effortless leap I had envisioned however.  What follows is my adventure in getting Selenium to a point where I could start producing some actual tests.  I found answers to many of these problems all over the place.  When I can recall where I found them, I will link to the page, but I viewed countless pages wandering around and pleading to the Googles for help and don’t plan to retrace all of that.  Hopefully someone else on a similar quest comes across this post and explains to me how I’m a total idiot, where to find these answers clearly written in one place, and how I could’ve accomplished the same thing in a smarter, easier way.  If not that, perhaps some other lost chump like myself will be grateful to find a bunch of solutions in one place.

Problem:

The problems started early.  I recorded a simple script.  I converted it to Java (please don’t bother to tell me that this was my problem).  I tried to run it with Selenium RC.  At this point I found that Firefox opened up in a strange configuration, barfed a bunch of popups at me and then the test killed itself.  Some poking around revealed that *chrome as the browser mode was at fault.  Certain extensions were being loaded that just didn’t want to play nice and the profile would not remember the login for our company’s Blue Coat Proxy.

Solution:

Selenium allows for a *custom browser mode.  This allows you to get your browser configured manually.  http://www.borngeek.com/firefox/profile-tutorial/ me helped get my Firefox profile straightened out.  I created a profile with minimal extensions (Firebug and McAfee’s SiteAdvisor were particularly rude to my Selenium tests).  The profile (named selenium) is also configured to use the Selenium RC server as a proxy (localhost:4444).  I made sure to disable warnings, popup blocking etc as these features may interfere with the test scripts.  Since it’s a separate profile, I can easily return to my normal, more securely configured settings by restarting Firefox in it’s regular profile.  The browser string in my Selenium setUp call now looks like: “*custom C:\\Program Files\\Mozilla Firefox\\firefox.exe -P selenium”

Problem:

Having defeated the popups, extensions and proxy settings I hit some SSL certificate issues.

Solution:

First, I added the cybervillainsCA certificate to Firefox as a trusted authority (only for the selenium profile!).  That didn’t quite do the trick.  It turns out the Selenium RC server doesn’t care for the certs in our test environment.  Some more Googling revealed that restarting the selenium server with -trustAllSSLCertificates.

At this point I’m starting the server with

java -jar selenium-server.jar -trustAllSSLCertificates

That’s actually in a .bat file which has a shortcut in my Quick Launch bar so that it’s started with a single click.

Problem:

I figured by this point I’d done the configuring that I’d need to and now the coding party would start.  When I recorded the script I had done a simple login, navigated to a second page, logged in to the test chat client and then stopped.  This failed and I noticed that the script had not recorded the second open, so it was looking for the second login form on the wrong page.  That was weird I thought, but I added it in.  The test still broke.  Here I learned that I was not allowed to start on one domain and then open another.  I had logged in at sub1.testenvironment and the second page was at sub2.testenvironment.  The problem relates to cross domain restrictions browsers enforce on JavaScript.  Selenium controls the actions with JavaScript so it has a meltdown here.  It turns out that the *chrome browser mode operates in a way that avoids this restriction.  Here “chrome” is a Firefox mode, not a reference to Google’s browser.  That’s a bitch when you’re searching to understand what it is and how it works.  I already found back at the beginning that the *chrome mode didn’t work for me, so then what?

Solution:

Some google wandering turned up this very useful document: Selenium Tutorial.  It indicates that this problem can be addressed by running the Selenium server in proxyInjectionMode.  Ok, fine.  Now I’m starting the server with:

java -jar selenium-server.jar -proxyInjectionMode -trustAllSSLCertificates

Problem:

That victory celebration was cut very short as I learned that this mode was broken.  Before making that change I was able to have my script open the browser, load a page, login, load a page and then fail on a cross domain navigation attempt.  After the change, the whole test failed right at the start.  A step back!  Bummer.  ”onXhrStateChange.bind is not a function on session”  WTF does that mean?  A couple resources essentially said “it means you’re using proxyInjectionMode, don’t do that.”  Well shit, I kinda have to don’t I?

Solution:

http://groups.google.com/group/selenium-users/browse_thread/thread/bc7030cd44730d4f

A clue!  Put “true” as an extra argument!  There’s a slight problem with this however…  Java doesn’t really care for additional arguments.  You can’t go around forcing extra values on methods.  Previously I was just using the provided selenium-java-client-driver.jar.  Thankfully, the source jar is also there, so digging into that we get down to DefaultSelenium.java.  There’s an array that I can put “true” into.  Let’s see if it works:

public void open(String url) { commandProcessor.doCommand(“open”, new String[] {url, “true”}); }

Booya!  Cross domain problem is gone for me.

Problem:

Now the script is logging in on sub1.testenvironment, navigating to sub2.testenvironment and apparently loading the chat client page, but the script cannot find the elements on the page.  Actually worse than that, it claims it can’t find the window.  Somehow after the second selenium.open() call, it completely loses track of the window and can’t do anything else.

Solution:

Rather than open() for the chat test page, I used openWindow() which allows me to name the new window which Selenium is now able to find.  This seems to be a rather inelegant solution, to just start popping up new windows, but the second should be as far as I go so it’s not a huge deal.

Problem:

IT IS STILL NOT SUCCESSFULLY SUBMITTING VALUES ON THE CHAT PAGE!!!!!

Solution:

Timing.  You can’t take anything for granted about Selenium’s timing.  While to my eyes, I’d see the browser window load with all of the necessary fields, Selenium would claim that they were not there.  I used waitForPopUp() after openWindow(), but that does not ensure that the JavaScript client has successfully executed it’s initialization.  Some additional code was added to wait for Selenium to find the element before trying to type in it.

System.out.print(“Waiting on login form”);

while(!browser.isElementPresent(“username”))

System.out.print(“.”);

browser.type(“username”, “testUser2″);

Sometimes it spins for awhile, other times it goes almost immediately, but it always succeeds now.  After all of that I don’t have any actual tests to show for it, but in my naive view, I’ve finally cleared the hard part and can actually throw in the test cases from here.

Selenium seems awesome in theory but every time I solve a problem, I find another one.

I bought my plane tickets for a trip to Madison, Wisconsin.. OK, Sun Devils, hurry up and learn how to play football.

So much reading to do, so little attention span. C’mon brain! There’s too much shit I don’t know to keep wandering off like that.

I argued with people all day today. It was the best day of work I’ve had in a long time.

Thanks for making an Anderson Silva fight worth buying, but seriously Chael, learn a little BJJ for fucks sake. You were asking for that.

Certainly there are appeals to follow and a nasty homophobic backlash on the way, but at least for now the gay marriage ban in California has been rejected.  I have a lot of opinions about politics, and I feel very strongly about many of them, but I there are no issues which concern me more than those regarding basic human rights.  I feel more strongly about this issue because I’m convinced that out of all the issues being debated in our society, it is the least balanced argument.  There are different sides to most issues, and usually both sides bring some logical support for their view. I oppose the wars our country is engaged in, but I concede that decisions are made based on information that I don’t have.  I can piece together what I’ve got and draw conclusions from that, but I’ll admit there’s a possibility that I’ve missed something.  Likewise, there are smart people who’ve studied economics who have conflicting ideas about fiscal policies.  Again, I have my opinions, but I can’t say that they are fixed and absolute.  I could be wrong.  Like anyone else, I don’t think that I’m wrong, so I vote accordingly.  One issue that I’ve actually turned around on is gun control.  I’m still not a fan of firearms, but my view of the second amendment isn’t nearly as negative as it used to be.

On the gay marriage debate, I don’t see any reasonable case for restricting the rights of other people.  I don’t accept a possibility of being wrong in this case.  There’s treating people fairly and then there’s blatant discrimination.  That’s it.  Fairness == good.  Discrimination == bad.  Consenting adults of any gender, sex, race, creed, fan club, ethnicity or what have you should have the same options for entering into a partnership.  There’s another argument about the name of this partnership and whether the government should have anything to do with it at all, but that’s separate and distracting.  Marriage is a convenient term for it, so that’s what we go by.  Those who claim that they are “protecting marriage” or supporting “traditional marriage” work with fictional concepts regarding what they think marriage should be.  The history of marriage contains many more issues than the modern “traditional” view cares to account for which really defeats the whole “traditional” concept right from the start.

The other list of reasons to legislate inequality include:

- Gay relationships are unproductive, they can’t have kids. So?  Plenty of straight couples don’t have kids.  Not allowing them equal rights is not going to make them quit being gay and begin procreating.  Besides, there are plenty of kids who need to be adopted…

- The children!  What about the children!? Children of gay parents are not worse off than the children of straight parents.  Delusional religious people would have you convinced that this is true, but they have no evidence for this.  In fact, a recent study showed that kids with two mommies were actually more well adjusted than kids from the “sacred” or “traditional” families.  Gay parents won’t make kids gay either, straight parents make the gay kids, see the previous argument.

- God said it’s bad. Really any other arguments are desperate grabs to cover up for this one.  Even though they argue (falsely) that America is a Christian nation, and was founded as such, many understand that this doesn’t constitute a rational argument.  Plenty of other sins are legal and will not see any significant legal strife.  Businesses are open on Sundays, straight people can get divorced, gambling is legal and of course the United States already attempted and failed to ban booze.  Since “God said it’s bad” doesn’t work for those, it won’t work here either.

- It’s yucky and gross! Any doubts they have about God are covered by this.  You know what’s yucky and gross?  Church.  Church : Freaky Metal Kid :: Gay Bar : Protect Marriage Assholes.  I don’t wanna be in church, so I don’t go.  I don’t like religious people telling me their ghost stories, so I don’t hang out with them.  If you don’t want to make out with another dude, don’t do it.  If you don’t like hot girl on girl action, stay off the internet.

I’m tired from work and losing my train of thought, so I’ll end with: don’t be a dick to people just because they’re different from you.  If you don’t like them, leave them alone to do their thing and you can go do yours and we’ll all be happy.  It’s not that fucking complicated.